Sounds like a similar problem that I had. You should check that the inline resistor networks are soldered in the correct orientation, and that the 74HC165’s are in the right way around.
Get yourself over to the sammichSID wiki page and download the test app, that will give you some clues as to what’s going on.
If it’s doing things just when your hand is near, sounds like some pins on the 74HC165 are floating (not connected) so your hand is acting like an antenna, channeling interference into that pin. It’s a bit like the buzz you hear when touching an audio plug that’s plugged into an amp.
Check there’s no shorts between encoder/switch pins and the heatsink… check all the solder joints there, resistor network orientation, parts in the right place, etc.