Level Crossings are among the most critical railway assets, concerning both the risk of accidents and their maintainability, due to intersections with promiscuous traffic and difficulties in remotely monitoring their health status. Failures can be originated from several factors, including malfunctions in the bar mechanisms and warning devices, such as light signals and bells. This paper focuses on the intelligent detection of anomalies in warning bells through non-intrusive acoustic monitoring by: (1) introducing a new concept for autonomous monitoring of level crossings; (2) generating and sharing a specific dataset collecting relevant audio signals from publicly available audio recordings; (3) implementing and evaluating a solution combining deep learning and transfer learning for warning bell detection. The results show a high accuracy in detecting anomalies and suggest viability of the approach in real-world applications, especially where network cameras with on-board microphones are installed for multi-purpose level crossing surveillance.