Lithium iron phosphate LiFePO4 triphylite is now one of the core positive electrode (cathode) materials enabling the Li-ion battery technology for stationary energy storage applications, which are important for broad implementation of the renewable energy sources. Despite the apparent simplicity of its crystal structure and chemical composition, LiFePO4 is prone to off-stoichiometry and demonstrates rich defect chemistry owing to variations in the cation content and iron oxidation state, and to the redistribution of the cations and vacancies over two crystallographically distinct octahedral sites. The importance of the defects stems from their impact on the electrochemical performance, particularly on limiting the capacity and rate capability through blocking the Li-ion diffusion along the channels of the olivine-type LiFePO4 structure. Up to now, the polyanionic (i.e., phosphate) sublattice has been considered idle in this case. Here, we demonstrate that under hydrothermal conditions up to 16% of the phosphate groups can be replaced with hydroxyl groups yielding the Li1-xFe1+x(PO4)1- y(OH)4y solid solutions, which we term "hydrotriphylites". This substitution has a tremendous effect on the chemical composition and crystal structure of the lithium iron phosphate causing abundant population of the Li-ion diffusion channels with the iron cations and off-center Li displacements due to their tighter bonding to oxygens. These perturbations trigger the formation of an acentric structure and increase the activation barriers for the Li-ion diffusion. The hydrotriphylite-type substitution also affects the magnetic properties by progressively lowering the Néel temperature. The off-stoichiometry caused by this substitution critically depends on the overall concentration of the precursors and reducing agents in the hydrothermal solutions, placing it among the most important parameters to control the chemical composition and defect concentration of the LiFePO4-based cathodes.