We present a modified outer gap model to study the phase-resolved spectra of the Crab pulsar. A theoretical double peak profile of the light curve containing the whole phase is shown to be consistent with the observed light curve of the Crab pulsar by shifting the inner boundary of the outer gap inwardly to $\sim 10$ stellar radii above the neutron star surface. In this model, the radial distances of the photons corresponding to different phases can be determined in the numerical calculation. Also the local electrodynamics, such as the accelerating electric field, the curvature radius of the magnetic field line and the soft photon energy, are sensitive to the radial distances to the neutron star. Using a synchrotron self-Compton mechanism, the phase-resolved spectra with the energy range from 100 eV to 3 GeV of the Crab pulsar can also be explained.