I've been looking at used bikes but I'm worried that I will get taken advantage of because I don't know a lot about bikes. Are there key questions or things I should look for when shopping for a used bike?
With any bike the most important thing is "fit" -- is the bike too tall, too short, too much "reach", too little?
For a bike to have a good basic fit you should be able to stand over it flat-footed and have "comfortable" clearance at the crotch. For bikes with lower than normal top bars (eg, many mountain bikes), you should pick a bike that would be comfortable even with the top bar at the "normal" height. And for "real" off-roading you want more actual crotch clearance than for more normal riding. But too much crotch clearance is a signal that the bike may be too small for you.
The bike should also be tall enough that you can raise the seat high enough to have your legs nearly completely extended at the bottom of your pedal stroke.
When riding, the "reach" should be comfortable for your style of riding. For casual riding you want a shorter reach and more upright posture than for racing or aggressive off-roading. However, for really tall, long-torso people a "casual" reach can be too short for even casual riding, forcing the rider to curl his torso too much, and short people, on the other hand, may have to use a bike that's a hair smaller than ideal to get the most comfortable reach.
When you ride it is it stable (but is it also responsive -- there's a trade-off). And obviously you want a bike that suits your needs -- multispeed or not? "Road" vs "mountain"? Upright or "aero"? Can carry a load with a rack, or is stripped-down for lightness and speed?
Checking the bike over:
Everything should be parallel (except, perhaps, the chain on a derailleur bike).
Stand straddling the front wheel looking toward the back of the bike. With the front wheel centered you should be able to look down and see the tire "disappear" under the frame tubes, with it not peeking out more on one side vs the other (which would indicate a warped frame). Similarly, the front and rear wheels should be exactly in line, not one more to one side than the other.
Look at the derailer. When the arm of the derailer is moved it should stay parallel to the wheel. Look at the chainrings. When the crank is turned, the chainrings shouldn't wobble.
Standing over the top bar, grip the rear brakes and move the bike forward and back. There will be some "spring" in rim brakes, but there should be no "rattle" sensation. Do the same with the front, only also feel for any looseness in the headset bearings. There should be no detectable looseness in the headset.
Off the bike (and preferably with the bike upside down or suspended in a stand) shake the wheels, feeling for any looseness in the wheel bearings. Likewise shake the crank.
And, obviously, check for serious rust, things that are seriously dinged, scratched, etc. A few scratches on a well-used steel frame is not a reason to reject the bike, though (unless you're looking for a "perfect" specimen).
One important thing is to give the bike a test ride, and feel it. You can tell a lot that way.
Check the frame over well, look for dents, scratches in the paint (and rust, if the frame is steel) make sure the wheels are sound. The components, everything. Just give it a good look all over. Anything broken should be fairly obvious.
Also ask a lot of questions
Only one thing to add to the other answers. Be aware of a frame with a brand new coat of paint. From experience I have found that this is done to cover defects in the frame. In my case the down tube had a slight crack in it that the new paint covered over and after a quick test ride it the crack was still not visible. A couple of months later we found that the crack had opened up and the tube had to be replaced.
Lift the bike so one wheel is off the ground and spin the wheel. Look at the rim where it meets the brakes, and see how much it wobbles. Ideally, the wheel should be perfectly round, but this is pretty rare on used bikes. Deviation of 2 mm or less is pretty good; 5 mm means it should get fixed; 15 mm means it's beyond fixing, and the wheels are garbage. (This assumes the bike has rim brakes; with disk brakes, the tolerances are a little more generous.)
Check for loose spokes. Grab a spoke between two fingers and gently shake it - if it's installed properly, it shouldn't move at all, if it feels like it's not firmly attached, then you have a problem.
Check the tires. Worn-down tread is not necessarily a problem (unlike car tires), but watch out for cracked, brittle-looking rubber.