We consider finite difference approximations of the second derivative, exemplified in Poisson’s equation,the heat equation and the wave equation. The finite difference operators satisfy a summation-by-partsproperty, which mimics the integration-by-parts. Since the operators approximate the second derivative,they are singular by construction. To impose boundary conditions, these operators are modified usingSimultaneous Approximation Terms. This makes the modified matrices non-singular, for most choicesof boundary conditions. Recently, inverses of such matrices were derived. However, when consideringNeumann boundary conditions on both boundaries, the modified matrix is still singular. For such matrices, we have derived an explicit expression for the Moore–Penrose pseudoinverse, which can be used forsolving elliptic problems and some time-dependent problems. The condition for this new pseudoinverse tobe valid, is that the modified matrix does not have more than one zero eigenvalue. We have reconstructedthe sixth order accurate narrow-stencil operator with a free parameter and show that more than onezero eigenvalue can occur. We have performed a detailed analysis on the free parameter to improve theproperties of the second derivative operator. We complement the derivations by numerical experimentsto demonstrate the improvements of the new second derivative operator.