Yew (Taxus baccata L.) exhibits among conifers a unique macroscopic elastic behaviour. For example, it shows a comparatively low longitudinal elastic modulus related to its comparatively high density. We herein explore the microstructural origin of these peculiarities, aiming at the derivation of microstructure–stiffness relationships. We measure stiffness properties of yew at different hierarchical levels and compare them to corresponding stiffnesses of Norway spruce (Picea abies [L.] Karsten). Cell wall stiffness is investigated experimentally by means of nanoindentation in combination with microscopy and thermogravimetric analysis. On the macroscopic level, we perform uniaxial tension and ultrasonic tests. Having at hand, together with previously reported stiffnesses, a consistent data set of mechanical, chemical and physical properties across hierarchical levels of wood, we discuss influences of microstructural characteristics at different scales of observation. Moreover, a micromechanical model is applied to predict trends of effects of the microstructure on the investigated stiffness properties. On the cell wall level, particularly, the amount of cellulose and its orientation – which was earlier reported to be distinctly different for yew and spruce – result in differences between the two considered species. On the macroscopic scale, model predicted effects of the annual ring structure on transverse stiffness and shear stiffness are found to be smaller than effects of the microfibril angle and mass density.