This all works surprisingly well and easily. 1) mark out the holes 2) drill out as much of the hole as you can (but don't get carried away) 3) Use a corse square file to finish the hole out (remember, 1 direction (push) only)
the filing only takes about 3 mins per hole.
Layout tip - to keep things looking professional and "machine made" , layout a an evenly spaced grid for the holes, then only drillout and file the holes you want. If later you need to add a hole, everything remains space correctly. In any case, it look CNC-like even though you used a lousy file.
drilled out holes with file ready to go
Switches inserted. They friction fit within your filing tolerances - no problem