Coastal waters such as those found in the Baltic Sea already suffer from anthropogenic related problems including increased algal blooming and hypoxia while ongoing and future climate change will likely worsen these effects. Microbial communities in sediments play a crucial role in the marine energy- and nutrient cycling, and how they are affected by climate change and shape the environment in the future is of great interest. The aims of this study were to investigate potential effects of prolonged warming on microbial community composition and nutrient cycling including sulfate reduction in surface (similar to 0.5 cm) to deeper sediments (similar to 24 cm). To investigate this, 16S rRNA gene amplicon sequencing was performed, and sulfate concentrations were measured and compared between sediments in a heated bay (which has been used as a cooling water outlet from a nearby nuclear power plant for approximately 50 years) and a nearby but unaffected control bay. The results showed variation in overall microbial diversity according to sediment depth and higher sulfate flux in the heated bay compared to the control bay. A difference in vertical community structure reflected increased relative abundances of sulfur oxidizing- and sulfate reducing bacteria along with a higher proportion of archaea, such as Bathyarchaeota, in the heated compared to the control bay. This was particularly evident closer to the sediment surface, indicating a compression of geochemical zones in the heated bay. These results corroborate findings in previous studies and additionally point to an amplified effect of prolonged warming deeper in the sediment, which could result in elevated concentrations of toxic compounds and greenhouse gases closer to the sediment surface.