I have a single speed bicycle that has some slack in the chain and threatens to come off. How can I reduce this slack?
If you have horizontal dropouts (the wheel axle slides into the frame from the back of the bike) the dropouts have long slots which will allow you to pull the rear wheel back and take slack from the chain. Be careful to ensure when you torque the wheel nuts down the wheel doesn't slip from being aligned straight, or the chain is running at a slight angle.
If the chain is so stretched to the point where it's falling off, you really should consider replacing the chain. You can get away with removing a link to remove the slack but a worn chain will be prone to sticky links and popping off unannounced.
If you don't have a tool to measure chain stretch then a normal ruler can work. The pins on links should line up at the beginning and 12" mark.
I change my chain 2 or three times a year based on the distances I'm riding. Chaning chains regularly means that your gearing doesn't quite wear enough to fit a single chain and it's less likely that you'll have to purchase some new drivetrain components when you get a new chain.
Many/most single speeds also have the rear dropouts set up so you can slide the wheel back a bit. At least last time I looked at them.
The best method I've found for adjusting tension without using a Surly Tuggnut or similar tensioner is to walk the axle back and forth. Generally, just holding the wheel at tension is not enough to get the tension right without undue frustration.
NOTE: while this method works for singles and fixies, it doesn't work with quick releases. (It will also work with eccentric hubs where dropout tensioners don't help.)
The idea is that you install the wheel in the dropouts with the chain routed correctly and then tighten the axle nuts a bit beyond finger-tight. Then, loosen the right axle nut and pull the wheel (using the leverage from the rim) to tension the chain and tighten that nut. You want to get the chain a bit too tight and then gently tap the nut to back off the tension until it is just right. Repeat the process on the left axle nut to get the wheel aligned, keeping one or the other axle nuts tight. Proper tension should allow the bicycle to pedal smoothly without binding and with as little chain droop as possible (the big risk of a droopy chain is that it will derail and that would not be healthy.)
By using the wheel as leverage, you don't need to rely on physically holding the wheel perfectly straight and taught. After you've tensioned and aligned the wheel (the rim should be centered between the stays) be sure the axle nuts are tightened to spec to keep the wheel from slipping forward.
(Additionally, if as you're adjusting your chain tension you find that the tension wobbles - i.e. it is tighter at some spots in the pedal rotation - then you need to center your chain ring!)
Your chain has probably stretched.
An amusing anecdote. I took my 1996 Litespeed Tachyon (20K miles or more on it) into the shop for a tuneup and asked them NOT to change the drive train unless absolutely neccasary, since it would mean changing everything. Its DuraAce so that would be a big chunk of change.
Anyway, turns out my chain had stretched a complete link! It was clearly stretched, but one complete chain link, so he popped out a link and reconnected the chain.
Short of that you will have to decide what you want to do. See if the chain is stretched. They have a tool for this, see if you can borrow one.