I needed to add one more wheel pant screw. I removed the pant, removed the wheel and bearings, drilled a hole in the brake backing plate, installed a rivnut, then put it all back together.
With that done I could re-attach the brake cable ends to the brake arms.
