Countersink might be a bit steep of an angle for the pin to fill sometimes. You could try a tapered ream and lightly taper the ends of the handle material as well. Some people are starting to use adhesives to hold the pins in now. To do it this way, use coarse sandpaper and rough your pin up with coarse sandpaper and maybe put tiny notches in it (I do it lightly with hacksaw). Wash the pin and dry it, or degrease with acetone, and then squeeze a little bit of Epoxy or JB-Weld into the holes, and rub some on to the pin and push the pin in, and let it cure before working with it.
If you want to use bolts or rivets, drill your pilot hole (1/8"), then use a counterbore or step drill to counterbore a larger diameter through some of your handle material.