High-quality, ab initio potential energy functions are obtained for the interaction of bromine atoms and anions with atoms of the six rare gases (Rg) from He to Rn. The potentials of the nonrelativistic +2 and Π2 electronic states arising from the ground-state Br (P2) -Rg interactions are computed over a wide range of internuclear separations using a spin-restricted version of the coupled cluster method with single and double excitations and noniterative correction to triple excitations [RCCSD(T)] with an extrapolation to the complete basis set limit, from basis sets of d-aug-cc-pVQZ and d-aug-cc-pV5Z quality. These are compared with potentials derived previously from experimental measurements and ab initio calculations. The same approach is used also to refine the potentials of the Br- -Rg anions obtained previously [Buchachenko, J. Chem. Phys. 125, 064305 (2006)]. Spin-orbit coupling in the neutral species is included both ab initio and via an atomic approximation; deviations between two approaches that are large enough to affect the results significantly are observed only in the Br-Xe and Br-Rn systems. The resulting relativistic potentials are used to compute anion zero electron kinetic energy photoelectron spectra, differential scattering cross sections, and the transport coefficients of trace amounts of both anionic and neutral bromine in the rare gases. Comparison with available experimental data for all systems considered proves a very high precision of the present potentials.