The best advice I'm seeing here is to go to your bike shop and try out a few bikes. When you have a few test rides behind you, then you'll have a better idea of what questions to ask.
What this boils down to: Test-ride a few bikes, buy one, have fun with it. Everything else is details.
Bike types
I'd concentrate my search on hybrids and hardtail mountain bikes. ("Hardtail" means without rear suspension. Unless your budget is quite generous, you won't find good rear suspension in an "introductory" price range.) Mountain bikes are comfortable, easy to use, and agile. Hybrids are similar, but more suited to riding on asphalt. Both avoid the complications and expense of offroad machines and dedicated super-light road bikes.
Cruisers are an attractive alternative if you're okay going a little slow, but it's hard to fit one of those machines into a car trunk or back seat.
Saddles
It's unintuitive, but wide, cushy saddles are not the most comfortable. Unless you're getting a cruiser or other bike where you sit bolt-upright, these will chafe and become uncomfortable in short order.
On the other hand, hard, narrow racing saddles aren't appropriate for recreational and casual riding like you have planned. So I'd look for something in-between the two.
Tires
Most mountain bikes come with knobby tires, the ones that have square protrusions sticking out of the rubber. These are designed for riding on dirt, and aren't great for riding on the road. Fortunately, tires are fairly inexpensive. You can ride on the road on knobbies if you're okay with the speed hit. (I did my first 50-mile ride on knobbies.)
Folding bikes
I'll mention this since you brought it up as a possibility. Folding bikes are more expensive than their non-folding counterparts of comparable quality. The market for them is smaller, driving up costs, and the hardware that allows them to fold also costs money.
Folding bikes can take a little getting used to, particularly the ones with tiny wheels. (Say, 16" and under.) They feel a little "twitchy" when you first ride them, and I suspect that this is why people say they're all bad bikes. However, you not only get used to this but you discover that these tiny-wheeled bikes are agile and fun. Folders can become an addiction quite easily, but they're usually not a good first bike. And the really bad ones (generally anything under $400US) are on a par with the worst big-box bikes.
If you're willing to pay a little more, I recommend these bikes (I own two of them and love them) but be aware that you can get a better bike for the same money if you're willing to forgo the folding.
As has been mentioned, most bikes will fit in the rear seat of a car by simply removing the front wheel. But it sure is nice to fold up a bike and put it in your trunk, or the passenger seat. Or under your desk at work.
Accessories
Buy a helmet and tuck your right trouser leg into your sock. Anything else can wait until you need it. But here's what you may want to get later on.
Riding for more than half an hour at a time? You might want some cycling gloves. If you're doing this in the cold, they also make winter cycling gloves.
Pants leg getting torn up? You can splurge on a pants clip.
Planning to ride in traffic? You can either wear visible clothing (bright colors) or get yourself a reflective jacket or safety vest. Gonna ride in the rain? They make reflective rain gear.
Want to haul stuff? A backpack will do the job, or you can get a rack.
Riding at night? It's not as scary as it sounds. Make sure you have that reflective gear on, and pick up some lights.
Speedometer? GPS? Cycling shorts? If you need them, you'll figure it out, but none of this is needed when you're getting started.
