I heard back from Vans yesterday regarding the rivet I drilled out. They said it wasn't a big issue and to put a rivet in that hole and drill another hole beside it, mid way between the two rivets, and put another rivet in that hole.
Today I drilled the new hole for the new rivet and then set about drilling out the other two rivets that needed to be drilled out. Both of the other rivets I ended up drilling most of the way through with a #40 drill bit. Once I had drilled fairly deeply I was able to use a punch and punch them out. They both came out with no hole damage.