Clean the bike with warm water and soap. Car soap works great but dish soap is also a good option. Don't be afraid to clean those really greasy and black areas thoroughly.
As for the rust on the crank and wheels, this suggests those parts are chromed (one-piece crank, perhaps?) therefore the rust you see is most likely surface rust; a layer of minerals containing oxidized iron on top of the chrome since chrome doesn't rust. For this I suggest using fine steel wool like SOS steel wool soap pads or Bull Dog steel wool pads. I find SOS works best due to the soap it contains as it cleans the parts at the same time, but you could add soap to Bull Dog pads of course. You can also use aluminum foil; make a ball and start rubbing your chromed parts. In all cases, you want to use something than is harder then the rusty minerals stuck on the chrome but not as hard as chrome so you won't scratch it. And use plenty of soapy water, don't go at it dry!
While you're at it, clean the spokes the same way; they're likely stainless steel and dull (unless they're painted, of course.) You can go harder on the spokes.
If you can, take off the wheels to clean them; it'll be a lot easier than when they're still on the bike.
Now about the chain, it is also probably surface rust. But due to all the links and pivots, it can be hard to get rid of all the rust without giving it a bath in acid. Just get a new chain, they're cheap and you know chain wear won't be an issue.
For the really greasy parts on the chainrings, derailleurs and sprockets, I like to use Varsol paint thinner or mineral spirits. A toothbrush can come in handy.
Cleaning the exterior of the bike is step one. If you like riding it, next step would be to disassemble the crankset, hubs and pedals (if they allow it). Degrease the ball bearings, cups and cones (paint thinner) and re-grease them with new fresh grease :) If you don't have the tools and/or know how/interest to do it yourself, bring the bike to your local bike shop. Since you already cleaned most of the bike, they shouldn't charge you a lot to work on the bearings.
Last thing; if the bike is very old, the tires and inner tubes are probably due for a change as well. Even if they seem to hold a good pressure, don't go too far without a complete puncture kit that includes a tire boot. Look it up.
Spending time and money on restoring old bicycles is about fun and acquiring experience. Unless it's a very valuable, sought after bike, whatever money you spend on an old bicycle won't add to its value. But if you do it for pure fun as a hobby or to learn the trade of bicycle mechanic, it can be all worth it. Just beware of spending a fortune on tools like I did!