Cage/Bottle Cage/Bottle Holder
Water bottle cages hold water bottles on a bike frame. They can be made of steel, aluminum, carbon, or titanium. Most cages attach to the frame via preinstalled threaded holes that are usually called bosses or mounts.
Most modern frames have at least two bottle cage mounts, one on the seat tube and one on the down tube. Some gravel bikes have a third mount on the underside of the down tube. Some full suspension mountain bikes may have only one mount, as the suspension system can take up some room in the bike's main triangle. Some dedicated cyclocross racing bikes may lack cage mounts entirely, as cyclocross races are short and one may need to shoulder the bicycle during a race.
On frames without mounts, bottle cage adapters might be able to strap around the tube. Alternatively, you may be able to drill a hole into the frame and put in a threaded insert called a riv nut, although this will void your warranty and it will create a stress riser. This raises the risk of later failure. It may not be an issue with relatively thick steel tubing, however.
A couple examples of cages are shown:

For time trials and triathlons, there are also cage adapters that attach to the seat and provide mounting points for extra water bottle cages and spare tubes/inflators. A few new mounts place a water bottle between the extension of an aerobar set. (Image credit to Slowtwitch.com, a triathlon forum.)

Hydration vests like those made by Camelbak are an option for bikes without any cage mounts or to carry additional water. These may be less comfortable in hot weather. Hydration bladders can also be carried in frame packs that are strapped to the top tube; these packs frequently have exit ports for the hoses. Last, stem bags will strap to the stem and handlebar, and these can carry a bottle also.
Some bikes, mainly gravel bikes, may have mounts on the outsides of the fork blades. These multipurpose mounts can fit standard bottle cages or larger cargo cages. Some gravel and triathlon bikes may have a pair of bosses on the top tube near the stem. These may have the spacing typical of water bottle cages, but they are used for bolt-on boxes (aka Bento boxes) that hold snacks and equipment.