That's scary, you shouldn't be able to do that. It sounds as though you can straighten it easily too. If you straddle the front wheel facing backwards can you twist the bars in the fork easily?
If so, definitely tighten it up. Firmly. You should not be able to move that!
Who put the bike together? If it's new, take it back and talk to the owner of the shop - this is something they should be aware of so they can (re)train whoever assembled/serviced the bike. Of course, if it's a big box shop paying schoolchildren $2/bike to take them out of the box and assemble them they won't care, but you shouldn't be buying bikes from them for exactly that reason.
(edit: answer second part of the question) Over-tightening it could break things, yes. But that's normally quite hard to do. If it's a new bike get the shop to do this. If it's your bike it's easy enough to do.
For a modern threadless headset there will usually be two allen key bolts to tighten, and the best solution IMO is the Rolls-Royce one* - use a multi-tool with a short handle, so you don't have enough leverage to overtighten it. Do the bolts up firmly, test by trying to twist the handlebars. If you can, do it up a little more firmly. Then stop, because you don't want to strip the stem and there's probably something else wrong.
For a quill stem there will be a single hex head bolt going down into the headset (rather than across the top). But the procedure is the same, tighten it firmly and test.
In both cases, if that doesn't fix it you should open things up and find out why. But I'll leave that for another question.
- Henry Royce made each spanner of such a length that when tightened firmly the relevant bolt was correctly torqued. There's a description here in the paragraph beginning "It is worth recounting that the ‘S’ series".