In this paper we demonstrate computer simulation results obtained for the coherent mode division multiplexed (MDM) 5x5 QPSK transmission using principal modes (PMs) of the stepped-index few-mode fiber (FMF) as a basis of independent signal carriers. The output signal recovering and the fiber propagation matrix determination are considered to be carried out in optical domain by means of reconfigurable multibranch diffractive optical elements (DOEs). Both the cases of Gaussian and Nyquist raised-cosine pulse shaping are considered for optical signal modulation. The simulation results show, that the transmission in the basis of PMs in strong coupling regime allows the reliability of the coherent MDM system to be fundamentally improved. As a result, utilization of the optical signal processing for MDM transmission could minimize substantially the DSP circuit complexity required for the real-time recovering of the transmitted signal.