A pannier is the best way to carry weight on a bike. It keeps the center of gravity low. While a few people prefer backpacks, most will find a pannier (or two, to keep things balanced) makes for an easier ride (uphill, downhill, or on the level), though at the expense of eliminating that sexy wet spot on the back of your shirt.
You can also use a rack-top "caboose" pack, but it has a higher center of gravity and makes it much harder to swing your leg over the rear of the bike (if that's how you mount/dismount). A "saddlebag", attached to the back of the seat, also has a high center of gravity and has limited capacity, but is just fine for a few items. Slightly less ideal is a handlebar bag -- best used only for light stuff as weight there affects the bike's handling adversely.
Oddly, though, bike handling is generally better with weight in front panniers vs rear ones, though front panniers require an additional rack (which must be stiffly mounted) and good-quality (though small) panniers, so that the load doesn't bounce around -- a loose load on the front is not good to have, especially going downhill.
For rear panniers you need to be slightly concerned with heel clearance, but this isn't likely to be a big issue unless you have a very short wheelbase bike. For front panniers you likewise need to consider toe clearance. For the most part, once you've figured out how to mount your panniers (ie, how far forward/back on the rack) and adjusted things accordingly you'll have no trouble with these issues, except perhaps on that short-wheelbase bike.
Note that there are a number of panniers designed to convert to backpacks (and back). These are generally neither ideal as panniers nor ideal as backpacks, but they may be a good compromise for your use.