The impact of bismuth incorporation into the epitaxial layer of a (Ga,Mn)As dilute ferromagnetic semiconductor on its magnetic and electromagnetic properties is studied in very thin layers of quaternary (Ga,Mn)(Bi,As) compound grown on a GaAs substrate under a compressive misfit strain. An addition of a small atomic fraction of 1% Bi atoms, substituting As atoms in the layer, predominantly enhances the spin-orbit coupling strength in its valence band. The presence of bismuth results in a small decrease in the ferromagnetic Curie temperature and a distinct increase in the coercive fields. On the other hand, the Bi incorporation into the layer strongly enhances the magnitude of negative magnetoresistance without affecting the hole concentration in the layer. The negative magnetoresistance is interpreted in terms of the suppression of weak localization in a magnetic field. Application of the weak-localization theory for two-dimensional ferromagnets by Dugaev et al. to the experimental magnetoresistance results indicates that the decrease in spin-orbit scattering length accounts for the enhanced magnetoresistance in (Ga,Mn)(Bi,As).