I got this bike secondhand, and so I don't know its history/what could have made it like that. And I mean they wobble while I'm holding them, not while I'm riding hands-free only. I've ridden a few bikes before, and they didn't have this wobble, so I don't think it's just a case of me not being strong enough to hold the handlebars straight.
Different bikes can have different geometry. The head tube (the tube that continues the line of the front forks up to the handlebars) angle has a big effect on how the bike handles.
A steeper head tube makes the bike more "twitchy". You can feel like you're all over the place. A bike like this would be a more serious road bike, and if you are used to using flat handlebars then it probably has narrower handle bars (drop bars) also.
Both of those make it harder to control, until you get used to it.
For more reading see
As PeteH commented, possibly a bad headset (could be worn headset or bearings need maintenance or replacement or the bearing may simply have been overtightened) As we ride, small corrections are constantly made, left, right,, left, right... When the handlebars/fork can't turn smoothly, these corrections become exaggerated - too much to the left, then too much to the right. Grasp the frame and lift the front wheel off the ground; very gently turn the handle bars all the way left and right. If you feel any roughness, catching, binding, etc., then you need to take a look at the bearings and headset.