We analyse the life cycle carbon implications of a building, with structural frame of prefabricated concrete, prefabricated modular timber or cross laminated timber (CLT) elements, designed to meet the Swedish passive house criteria. The analysis covers non-biogenic carbon flows related to the building alternatives, over an assumed life time of 80 years. The building alternatives are all modelled to have the same housing service and operation energy demand. Substitution factors, showing the efficiency of CO₂ emissions reductions when wood alternatives are used instead of non-wood alternatives, are calculated for the CLT and modular alternatives with reference to the concrete alternative. The results show that the CLT and modular buildings give less carbon emissions to the atmosphere during production and when the buildings are demolished at the end-of-life. Moreover, the wood residues from the production and end-of-life activities for the timber alternatives far exceed that for the concrete alternative. The substitution factors differ slightly between the CLT and the modular alternatives, and are significantly lower when fossil gas is substituted by wood residues instead of fossil coal. In summary, the life cycle carbon emissions are significantly lower for the timber alternatives.