Transition-metal perovskite chalcogenides (TMPCs) have emerged as lead-free alternatives to lead-halide perovskites and have been currently of increasing interest for optoelectronic applications because of their suitable band gaps, high carrier mobility, strong light absorption, and high stability. Here, we systematically report a study on the effects of Ti- and Se-alloying strategies on polaron behavior and carrier lifetimes in nonradiative recombination. Although such alloying can effectively tune the band gap of BaZrS3, we observe localized small polaron formation upon Ti alloying and large polarons generating in Se alloying. Ti-alloying strengthens the electron-phonon coupling, leading to a reduced carrier lifetime. Remarkably, Se-alloying weakens the electron-phonon coupling and prolongs the nonradiative electron-hole recombination lifetime by up to 60% compared to that in pristine BaZrS3 material. The simulations rationalize the difference in carrier lifetimes in TMPC alloys and provide guidelines for further improvements in TMPC-based photoelectronic devices.