Maybe you are playing one or more keys on an external keyboard at the same time?
Because pattern changes are not only controlled by the matrix buttons, but also by received MIDI Note On/Off commands.
To doublecheck that this behaviour isn’t related to a hardware issue, I would propose to unplug the MIDI IN cable after you’ve ensured that there is no actively played MIDI key.
Alternatively you could select another MIDI channel and/or reduce the keyboard zone for the drum instruments (ENS->INS)
Thereafter press and depress the PLAY button - this will initiate an internal “all notes off” - just to ensure.