The Particle-In-Cell (PIC) method is widely used in relativistic particle beam and laser plasma modeling. However, the PIC method exhibits numerical instabilities that can render unphysical simulation results or even destroy the simulation. For electromagnetic relativistic beam and plasma modeling, the most relevant numerical instabilities are the finite grid instability and the numerical Cherenkov instability. We review the numerical dispersion relation of the electromagnetic PIC algorithm to analyze the origin of these instabilities. We rigorously derive the faithful 3D numerical dispersion of the PIC algorithm, and then specialize to the Yee FDTD scheme. In particular, we account for the manner in which the PIC algorithm updates and samples the fields and distribution function. Temporal and spatial phase factors from solving Maxwell's equations on the Yee grid with the leapfrog scheme are also explicitly accounted for. Numerical solutions to the electrostatic-like modes in the 1D dispersion relation for a cold drifting plasma are obtained for parameters of interest. In the succeeding analysis, we investigate how the finite grid instability arises from the interaction of the numerical 1D modes admitted in the system and their aliases. The most significant interaction is due critically to the correct representation of the operators in the dispersion relation. We obtain a simple analytic expression for the peak growth rate due to this interaction.