If the crack is in the handle bar, the failure mode is as follows:
One side of the handle bar breaks off.
You fall forward, one hand pushing the broken off handle, the other hand pushing the still connected handle.
This falling motion will steer your bike sharply to the side with the broken off handle, effectively kicking your bike out from underneath your butt.
You will make contact with the road at the full speed you were riding at.
The resulting damage depends on your exact speed and road conditions, but I've had this type of accident once at about 20 km/h on a smooth surface (no sharp edges or bumps from tree roots, uneven sidewalk tiles, and the like), and it's still the third most severe accident I've ever had.
Note that the energy that goes into bruising you will go up with the square of your speed. So, at 30 km/h you have to expect more than twice the injuries than at 20 km/h. Do this at 40 km/h, and you'll have to hope that you are found before you bleed to death.
A crack in the stem itself may give a similar failure mode, or it may break directly forward. I don't know what would happen in this case, but my intuition tells me that this would be much more dangerous than falling sideways at full speed.
Bottom line: Your fork, your stem, your handlebar, and your steering tube are too critical for your security to ignore cracks. If any of these fail, there is no backup, and the failure modes are scary. You won't have any chance of mitigating the consequences of the crash.