In this paper we consider scattering resonance computations in optics when the resonators consist of frequency dependent and lossy materials, such as metals at optical frequencies. The proposed computational approach combines a novel hp-FEM strategy, based on dispersion analysis for complex frequencies, with a fast implementation of the nonlinear eigenvalue solver NLEIGS. Numerical computations illustrate that the pre-asymptotic phase is significantly reduced compared to standard uniform h and p strategies. Moreover, the efficiency grows with the refractive index contrast, which makes the new strategy highly attractive for metal-dielectric structures. The hp-refinement strategy together with the efficient parallel code result in highly accurate approximations and short runtimes on multi processor platforms.