In this paper a high-order finite element method with curvilinear elements is proposed for the simulation of plasmonic structures. Most finite element packages use low order basis functions and non-curved elements, which is very costly for demanding problems such as the simulation of nanoantennas. To enhance the performance of finite elements, we use curvilinear quadrilateral elements to calculate the near-field from an impinging plane wave with second order absorbing boundary conditions. The magnetic field amplitude on the surface of one object is compared with a computation based on a multiple multipole expansion. Moreover, the convergence behavior of p-FEM with absorbing boundary conditions motivate an adaptive strategy of polynomial degree enhancement and enlargement of the domain.