Bicycles (and other two-wheeled vehicles like motorcycles) constantly need righting while riding. That is done mostly by the rider. If the bicycle starts to tip to the side, you automatically steer to the same side so you induce an opposite effect. This is helped by the head angle, the wheel trail and the gyroscopic effect, but active steering plays the biggest role. If a bicycle leans to one side for a longer time, it starts to turn or it falls over. (In fact, this is how you steer a bike in a corner: you steer in the opposite direction to make it lean into the corner and then you just keep it leaning until you've taken the turn.)
If you're cycling on the right side of a cambered road, the part of the road you're riding on slopes to the right. On a normal bike this isn't a problem since you can ride upright regardless of the slope.
Compared to this, a bicycle with training wheels is a completely different vehicle. It doesn't need righting and it doesn't need to lean into corners. In fact, if you try to ride it like a normal bike it leads to the experience you described. Your training wheels make the bike lean to the right, perpendicular to the road surface. Your natural reaction to the bike leaning to the right is to steer to the right to counteract the leaning. You then trick your brain into thinking the bike is already upright by leaning left relative to the bicycle.
If you would lift the training wheels a little, you would create a vehicle that behaves like a bicycle when it is upright but makes it impossible to lean into corners and actually throws you off if you try to go into a corner at any speed.
The solution is to remove the training wheels completely. Training wheels suck and they don't teach you how to ride but they also hinder your progress.