This article contributes to the discussion on how the Swedish labor market is changing: is it upgrading or polarizing? Drawing on the Swedish Labor Force Survey the study examines the overall changes in the occupational job structure in Sweden by exploring how women and men were distributed within the occupational prestige hierarchy at two points of time, 1997 and 2015. The results show that changes in the labor market have resulted in different patterns of how women and men are distributed within the occupational prestige hierarchy. Women have an upgrading movement and have entered high-prestige occupations, while men have been subjected to job polarization, with an increase in employment in low-prestige occupations, as well as high-prestige ones.