Phase Spider, the Phase Wasp is called as such not because it can travel between dimensions, but because it can attack between dimensions. Your ordinary Phase Wasp is a creature of the Material Plane, a magical insect about a foot long (yikes), but without the venom that ordinary wasps have. Instead, their stinger shoots bolts of force, as per the magic missile spell, which allows it to hit creatures whose bodies are in the planes which are overlaid over the material. The wasp also has the natural ability to see invisible creatures which, again, helps it defend itself against etheral creatures.
The source of the Phase Wasps' magical powers are not explained, but I have my own theories. Like ordinary wasps, these dudes make their nests out of chewed wood pulp and paper. They have a special fondness for making them out of the papers of spell books, which makes a Phase Wasp infestation especially bothersome for mages.
So my theory is that Phase Wasps at some point in the past were ordinary wasps just looking for materials for their nests. A couple of conveniently available magical libraries you have a new breed of giant magical super-wasps. Thanks a lot, wizards.