The energy retrofitting of existing buildings reduces the energy use in the operation phase but the use ofadditional materials influence the energy use in other life cycle phases of retrofitted buildings. In thisstudy, we analyse the life cycle primary energy implications of different material alternatives whenretrofitting an existing building to meet high energy performance levels. We design retrofitting optionsassuming the highest and lowest value offinal energy use, respectively, for passive house standardsapplicable in Sweden. The retrofitting options include the thermal improvement of the building enve-lope. We calculate the primary energy use in the operation phase (operation primary energy), as well asin production, maintenance and end-of-life phases (non-operation primary energy). Our results showthat the non-operation primary energy use can vary significantly depending on the choice of materialsfor thermal insulation, cladding systems and windows. Although the operation energy use decreases by63e78%, wefind that the non-operation energy for building retrofitting accounts for up to 21% of theoperation energy saving, depending on the passive house performance level and the material alternative.A careful selection of building materials can reduce the non-operation primary energy by up to 40%,especially when using wood-based materials