We report on very high energy (>100 GeV) gamma-ray observations of Swift J164449.3+573451, an unusual transient object first detected by the Swift Observatory and later detected by multiple radio, optical, and X-ray observatories. A total exposure of 28 hr was obtained on Swift J164449.3+573451 with the Very Energetic Radiation Imaging Telescope Array System (VERITAS) during 2011 March 28-April 15. We do not detect the source and place a differential upper limit on the emission at 500 GeV during these observations of 1.4 × 10–12 erg cm–2 s–1 (99% confidence level). We also present time-resolved upper limits and use a flux limit averaged over the X-ray flaring period to constrain various emission scenarios that can accommodate both the radio-through-X-ray emission detected from the source and the lack of detection by VERITAS.