In the last years, there has been a growing interest in the emerging concept of Digital Twins (DTs) among software engineers and researchers. DTs represent a promising paradigm to enhance the predictability, safety, and reliability of cyber-physical systems. They can play a key role in different domains, as it is also witnessed by several ongoing standardisation activities. However, several challenging issues have to be faced in order to effectively adopt DTs, in particular when dealing with critical systems. This work provides a review of the scientific literature on DTs in the railway sector, with a special focus on their relationship with Artificial Intelligence. Challenges and opportunities for the usage of DTs in railways have been identified, with interoperability being the most discussed challenge. One difficulty is to transmit operational data in real-time from edge systems to the cloud in order to achieve timely decision making. We also provide some guidelines to support the design of DTs with a focus on machine learning for railway maintenance.