Bearing
A bearing is an interface that enables the axle to rotate inside some fixed part. The axle rolls on the bearings. For example, consider your hubs, pedals, the cranks, and your front fork.
Sheldon Brown has a longer discussion on bearings. Most bearings are steel balls.
Preload is important term related to bearings. For your hubs, preload is how much the hub is getting squeezed in from the side, ignoring the quick release skewer. All cup and cone hubs have inherent preload adjustment, but not all cartridge bearing hubs or other devices do. Too much preload will make the bearings grind and wear out. Too little preload can leave play in the hub (i.e. side to side movement when the quick release skewer is clamped) or other device (e.g. crankset, headset will rock back and forth).
Bearings will feel gritty when they are worn or dirty. Note that too much preload can also cause a gritty feeling. You can feel this by spinning the wheel in your hand, holding near the axles, or otherwise working the component by hand. In some cases, e.g. you submerged the bike during a river crossing, you can wash out the grease from the bearing, and cause it to sound dry (a whirring sound) but not feel gritty.
Cup and cone bearings

This type of bearing, pictured above, has the balls rotating between the cup and the cone. The locknut tightens the assembly together, and can be loosened with a standard sized wrench.
Cup and cone bearings require periodic service. Service involves cleaning out the old grease, which may have had some dirt mixed in with it, and adding fresh grease. If the balls are worn, you can measure their size with a caliper and buy replacements. Alternatively, manufacturers' technical documentation will frequently state the size and number of ball bearings; take note that sizes for front and rear hubs may be different. Eventually, the cones should wear, and manufacturers should maintain these as replacement parts as well. The cups may not be replaceable, as they are frequently integrated into the hub or pressed in extremely tightly.
Cup and cone bearings are more traditional and are frequently present on older equipment, but some companies persist in making high-quality cup and cone bearings (notably Shimano and Campagnolo).
Cup and cone hubs all require preload adjustment. Best practice is to tighten the locknuts so that there is barely perceptible play when the hub is off the bike, and zero play when the quick release or thru axle is tightened.
Cartridge bearings
Most manufacturers have moved to cartridge bearings. In many contexts, cartridges are sealed units that can contain things like printer ink, film, or, in bicycles, ball bearings, seals, and races (roughly equivalent to the cone and the cup respectively). Sometimes they are called sealed cartridge bearings, and most of the bearings in bicycles have rubber seals, but this is potentially a misnomer. Bicycle bearings might be unshielded at one end, and cup and cone bearings can have seals built in.

To replace cartridge bearings, you will need a bearing extractor to pull or push the bearing out, and a bearing press to push it back in. You can substitute a block of wood and a hammer as a makeshift press, but the risk is that you push the bearing in at an angle, damaging it or the bore. Related to presses, bearing drifts help align the bearings as they are going in. Bike stores frequently have these items.
The bearings themselves are available commercially from many suppliers. All cartridge bearings have a designation printed on the seal. The numeric part of this label represents the size of the bearing, although this may not always be immediately obvious because the numbers may be an “arbitrary” code. For example, the common 608 bearing used for skateboards is 22mm in diameter. There may be additional alphabetical suffixes providing further information, e.g. about the sealing arrangement. For example, while hard to read, the bearing pictured above is an MR 2237 LLB, which refers to a bearing with 22mm inner race diameter, 37mm external diameter, and 8mm depth.
LLB is a suffix denoting the seal type. It may be specific to Enduro Bearings, a major supplier for bicycle bearings; in this case, it means light-contact rubber seals. More generally, “2RS” refers to having rubber seals on both sides, “RS” refers to a single-sealed bearing, and “Z” or “ZZ” refers to single- or double-shielded. Bearings with full-contact seals are also available. Seals generally trade off protection against dirt intrusion and friction. Do note that bearings with less-protective metal shields exist, as well as unshielded bearings. Those should generally be avoided in bicycle applications if you are buying from a supplier that is not bicycle-specific.
Seals contact the inner bearing race in some form, while shields have a gap. Shields are only intended to guard against large mechanical contamination. Liquid or ultra-fine contaminants may still enter a shielded bearing. Seals are not immune to contamination, but the continuous pressure exerted by the seal helps keep it out.
Most cartridge bearings are designed not to be serviced. They are replaced entirely. If manufacturer instructions do not mention regular service, you can assume it is not needed. Some higher-end manufacturers (e.g. Chris King, Kogel) design their bearings to be serviced periodically, which entails gently removing the rubber seals, cleaning the grease, and adding fresh grease. Some seals may be hard to remove without damaging them.
Many cartridge bearing hubs do not have a preload adjustment mechanism, so there is no need to worry about preload unless manufacturer documentation says so. SRAM cranks require you to use a collar to preload the system. For Shimano cranks, you just tighten the crank cap to finger tightness, then tighten the pinch bolts. A similar process applies to headsets.
Generally, you can rotate the axle by hand to check the status of the bearings. If they feel gritty, the grease is likely to be contaminated, and a serviceable bearing should be serviced. If the preload is too tight, that can also cause the bearing to feel gritty, or possibly to feel notchy - that is, as if there are several notches on a dial.
Durability
Many factors may influence the durability of the bearing. If you ride in wet conditions, this will reduce durability. If you pressure wash your bike, take care not to point the hose straight at the bearings (in your hubs, bottom bracket, rear suspension pivot). As stated earlier, some bearings are more tightly sealed than others, trading off higher friction.
In a press fit bottom bracket, if the bottom bracket seats are not parallel to each other or if one is out of round, that can damage the BB over time. Bearings also have a load capacity. If a piece of equipment was designed with bearings that are too small to take the loads applied to it, that would also impact bearing life. Additionally, all else equal, the more narrowly spaced the bearings are, the higher the loads on them will be. This does affect some press fit standards, like BB30 and PF30, which have their bearings more narrowly spaced than others. Last, the material quality of bearings and the races they run on has an impact. This would include the specific steel alloys used as well as their pureness, and how smooth they are.
Miscellaneous
Ceramic bearings are sometimes offered as an upgrade on high-end bikes. They have marginally lower rolling resistance than high-quality steel ball bearings. In many industrial applications, ceramic bearings are thought to be more durable, but this may or may not transfer to bicycling applications. In addition, ceramic bearings are extremely hard, and they must be paired with a high-quality steel race, or else they can dent the race when the bike goes over bumps. Many cheap ceramic bearings may skimp on the quality of the races, and buyers should beware.
Bushings (link to Wikipedia) are another form of bearing. These involve a sleeve rotating over the axle. Bushings are often used in rear derailer jockey wheels and in mountain bike suspension.