The motivation for a cosmic muon veto (CMV) detector is to explore the possibility of locating the proposed large Iron Calorimeter (ICAL) detector at the India based Neutrino Observatory (INO) at a shallow depth. An initial effort in that direction, through the assembly and testing of a similar to 1 m x 1 m x 0.3 m plastic scintillator based detector, is described. The plan for making a CMV detector for a smaller prototype mini-ICAL is also outlined.