[Perfectlap]

Mine were not a perfect alignment either, although I wouldn't say its extremely difficult either.
I used the template but still managed to get the hole punch not perfectly centered before I went in with the drill. The result is that I have to jiggle the rack into the receivers to get them to seat so that the holes for the security pins line up. But that's kind of a good thing because they'll have that friction to keep them in place and take some of the load off the pins.

Have you tried applying some white grease (lithium lube). I've seen it in the 3M spray cans.