We present an experimental and theoretical study of the absorption and emission spectra of Yb atoms in a solid Ne matrix at a resolution of 0.025 nm. Five absorption bands were identified as due to transitions from the 4f145d06s21S0 ground-state configuration to 4f145d06s6p and 4f135d16s2 configurations. The two lowest-energy bands were assigned to outer-shell transitions to 6s6p3P1 and 1P1 atomic states and displayed the structure of a broad doublet and an asymmetric triplet, respectively. The remaining three higher-frequency bands were assigned to inner-shell transitions to distinct J=1 states arising from the 4f135d16s2 configuration and were highly structured with narrow linewidths. A classical simulation was performed to identify the stability and symmetry of possible trapping sites in the Ne crystal. It showed that the overarching 1+2 structure of the high-frequency bands could be predominantly ascribed to crystal-field splitting in the axial field of a 10-atom vacancy of C4v symmetry. Their prominent substructures were shown to be manifestations of phonon sidebands associated with the zero-phonon lines on each crystal-field state. Unprecedented for a metal–rare-gas system, resolution of individual phonon states on an allowed electronic transition was possible under excitation spectroscopy which reflects the semiquantum nature of solid Ne. In contrast to the absorption spectra, emission spectra produced by steady-state excitation into the 1P1 absorption band consisted of simple, unstructured fluorescence bands.