This answer is an attempt to provide more details on the differences in handling between the 3 classes of bikes. It is a surprisingly hard question to answer, because each segment named in the original question and in @gschenk's very good update have evolved over time.
There is a primer on bike geometry, and its impact on handling, at the end of the article. This is a shorter version is as follows. Bikes can strike a balance between agility and stability. Agile bikes will be more responsive to steering input, which you might want in a road race or criterium. Stable bikes will want to put themselves back upright after you turn. This is a desirable trait in endurance bikes designed for non-racers, or perhaps even in bikes designed for stage road races. Additionally, putting loads (e.g. panniers or frame bags) onto most bikes will affect their handling, but the bike can be designed specifically to handle well with loads. Last, there is a difference between low- and high-speed handling. Cyclocross race bikes are designed to be agile at relatively low speeds (in a cyclocross race, corners are taken more slowly than in a road race). Many modern mountain bikes and some gravel bikes are designed to be stable on fast, technical descents, but they may not steer as readily into tight corners at speed.
One parameter that governs handling is trail. Higher trail will tend to increase the bike's stability. Typical trail values for road bikes are 55-60mm. Gravel bikes and MTBs tend to have more trail, but loaded touring bikes have less trail for reasons discussed later. This parameter is usually not found in bike geometry charts, but it can be calculated from a given head angle and fork rake. Note that bike handling is complex, and trail is not the only parameter involved.
Endurance vs Performance Road Bikes
Road bikes are quite diverse. At risk of stating the very obvious, they tend to be designed primarily or exclusively for operation on paved roads. They are not typically designed to carry loads. They are often capable of riding on dirt roads, but their tires and handing may make them inoperable on rougher gravel.
The previous answer alluded to one point of distinction between endurance road bikes and racing or performance road bikes: Endurance road bikes tend to have shorter reach (i.e. rider is less stretched out; the bike in the first picture has 367mm of reach) and more stack (i.e. rider sits up higher; illustrative bike has 563mm stack) than performance bikes in the same size.
Moving away from geometry, endurance road bikes probably have tubing or carbon fiber layups that are designed for more vertical compliance, and thus a more comfortable ride, than performance road bikes. To my knowledge, endurance road bikes may be designed with clearance for slightly wider tires than performance road bikes. To my knowledge, even endurance road bikes may not typically be equipped with rack and fender mounts. Maximum tire sizes may differ somewhat among road bikes. My impression is that it is usually not a major point of differentiation among current road bikes. Most should be able to take at least 28mm tires. I believe that most current road bikes cannot take gravel tires of 35mm or larger. However, see the discussion on all-road bikes in the last section.
Bikes designed for loaded touring may also be known as randonee bikes. Such bikes probably have stack and reach parameters similar to same sized endurance road bikes, or even less aggressive (i.e. more upright, shorter reach). I am not familiar with typical tire sizes, but I suspect touring bikes can accommodate larger tires than endurance road bikes - possibly much larger. They will also be designed to accommodate racks and fenders. This includes both the mounts, as well as tubing that can handle an extra load. I believe they tend to have longer chainstays and overall wheelbases. This increases stability and gives clearance for fenders.
In terms of geometry, weight over the front wheel makes it harder to turn a bike. Thus, as stated in the Cyclingtips article, loaded touring bikes may actually have lower trail than many road bikes. This would make for quick steering when unloaded. Why not put weight over the rear wheel? This 2009 article in Adventure Cycling by Jan Heine makes the case that (low) front loads are better than rear ones.
Touring bikes, then, are designed for very long rides on (I think) mostly paved roads. Their geometry may be optimized for stable handling while loaded. Secondarily, they need the appropriate mounts, and their frame material has to be able to accommodate the additional weight. I suspect this biases them towards steel or titanium tubing in thicker diameters than a performance road bike of the same material would use. I am not certain what typical tire clearances are.
Some CX bikes are race-oriented, with quick steering. These performance CX bikes often have lower BB drop to increase ground clearance. This makes it harder for a rider to strike a pedal in a corner or to strike the chainring on a low obstacle, but it makes for a higher center of gravity and faster handling. I would expect their wheelbases to be shorter, subject to their need for tire clearance.
Secondarily, these bikes might lack even water bottle cage mounts, let alone fender and rack mounts, and they might have clearance for 33mm tires at maximum. Note that UCI rules stipulate that CX tires must be no more than 33mm for UCI sanctioned CX races (however, not all races are UCI sanctioned, and even those that are may not enforce this rule for lower-level amateurs). So, CX racing bikes are designed for short criteriums on grass and mud.
This article by Rotor appears to corroborate my answer above. However, note that it claims to address the differences between cyclocross bikes and gravel bikes, whereas my read is that it focuses on cyclocross race bikes versus gravel bikes.
Cyclocross bikes require high maneuverability around tight turns and over barriers and other obstacles, so their designers prioritize speed and agility over comfort.
A typical gravel bike will have longer chainstays, a longer wheelbase, a slacker head tube angle, a slacker seat tube angle and a lower bottom bracket than a typical cyclocross bike. Gravel bikes tend to have more mountain bike-like geometry while ‘cross bikes tend toward a racier, road bike geometry.
However, before the widespread advent of gravel bikes, many CX bikes were designed as general purpose bikes that might be able to commute, go on light touring, or ride on gravel trails and dirt roads. The general purpose CX bikes probably have geometry and handling characteristics more similar to modern gravel bikes. In fact, I have one such bike, a Gunnar Crosshairs. I recall Gunnar's site at one time described it as being a bit like a stage race road bike (this was before endurance road bikes were a distinct category).
Endurance road vs. MTB geometry
The general purpose CX bikes I mentioned above might be the prototypes to modern gravel bikes. However, gravel bikes are diverse, because their intended uses are diverse. Gravel bikes appear to have a spectrum of geometries. One end tends towards endurance road geometry, the other end towards mountain bike geometry. These handle differently. They also may accommodate different use cases, with the road end more suited to gravel racing, and the MTB end more suited to backpacking or adventure riding. In particular, gravel bikes influenced by MTB geometry will have slack head tube angles and high trail. They may also have relatively long top tubes, and will be designed for short stems (e.g. under 90mm in most sizes, which is much shorter than road cyclists would run).
I have a Parlee Chebacco, which is geared more towards gravel racing, and has geometry and handling similar to many (endurance) road bikes. This review by Velonews indicated that the Chebacco was
... overmatched when the gravel gave way to bumpy singletrack, or less maintained dirt roads. It felt similar to a cyclocross bike in these situations, and it became immediately clear where the with the road end more suited towards gravel racing, and the MTB end more suited to backpacking or adventure riding, limitations lie.
Other gravel bikes may be much more inspired by mountain bike geometry. These bikes are designed to handle much more technical terrain, where one is making tight turns at low speed. This blog article by Tekné phrases it thus:
My analysis of geometry charts from different brands' gravel bikes reveals that some lean toward a mtb crowd, and others toward the cyclocross end of the spectrum in terms of fit. The former is more upright, and the angles are slacker. This is to provide more stability at speed on loose surfaces.
This article by Slowtwitch.com (a triathlon forum, funnily enough), discusses the geometry and handling differences between a more MTB-oriented gravel bike (the Lauf True Grit) and a more road/CX racing-oriented one (the 3T Exploro). The writer described the Lauf as very stable and very good at long straight lines. He said he preferred descending on the Lauf rather than the 3T. He described the 3T as more nimble. He had to be much more careful about choosing the right line on descents. However, he could climb more easily on the 3T because he could maneuver around obstacles much more quickly than on the Lauf. This review of Evil Bikes' Chamois Hagar states similar handling characteristics to the Lauf, and it additionally says that:
The one place I noticed the Chamois Hagar struggled is exactly where I thought it would: tight corners, especially at speed. Like early 29er mountain bikes, the Chamois Hagar does tend to overshoot corners. That’s the downside to the long, slack, mountain bike-esque geometry here, and that was evident on our test rides: any time a tight turn came up, the lines of tire tracks leading right off the trail seemed to pile up.
Also, some gravel bikes are designed for bikepacking - which is basically off-road touring. Riders can carry luggage in bags slung from the seatpost, the handlebars, the top tube, and each of the front fork legs. I do not think that rear rack mounts for panniers are as common for these bikes, but I do not know why this is so. I am not familiar with geometry changes for this design, but I suspect they have to account for the fact that the bike will usually carry loads, which may include front loads. That may mean relatively low trail, as with touring bikes. I suspect their bottom brackets may be lower to lower the rider's center of gravity, countering the fact that a lot of the luggage is mounted relatively high on the bike.
The clearest generalization we can make about gravel bikes is that they are designed to operate mainly on unpaved roads. Within that broad and obvious generalization, gravel bikes span a spectrum of intended handling styles. Some are more suited for longer-distance gravel racing. Some are more suited for technical terrain. Some gravel bikes incorporate low-travel suspension, which is not characteristic of CX race bikes and most road bikes - although some endurance road bikes have such suspension.
Gravel and All-Road Bikes
All-road bike may be an emerging term. From my read of the industry (NB, I don't work in it), some manufacturers may sell all-road bikes that sit between gravel and endurance road bikes, or they may sit at the road-like end of the gravel spectrum. My For example, this review by Bicycling Magazine of the Liv Avail Advanced Pro 1 Force bike stated that it was capable on pavement and dirt roads. It came stock with 32mm tires, and it had clearance for 35mm tires. The Trek Domane discussed under Endurance Road bikes, could also be considered an all-road bike with its 38mm tire clearance. My impression is that most gravel racers might prefer a 40mm tire in 2020, but 35mm may have been the standard for gravel racing around 2016/17.
However, other manufacturers use the term all-road interchangeably with gravel bikes. This 2017 article in Outside Magazine confirms this. Firefly Bicycles, a custom titanium bike maker, stocks only road and all-road bikes, and the latter appears to have similar specs to gravel bikes. Jan Heine uses the term all-road to refer to road-oriented gravel bikes, and adventure bike to refer to MTB-oriented gravel bikes. Carl Strong, a custom frame builder, draws a distinction between all-road, gravel, and adventure bikes, but on close reading, what he thinks of as an all-road bike (with clearance for 37-40mm tires) would be considered a gravel bike by many observers. He seems to merely be classifying the continuum into 3 categories, whereas Heine uses two.
It remains to be seen if all-road bikes emerge as a distinctive category from endurance road bikes. Alternatively, endurance road bikes might grow to accept wider tires and subsume this category. Or perhaps the categories of endurance road and road-oriented gravel bikes merge (which seems consistent with Heine's vision of the all-road bike).
Gravel vs. Cyclocross Race Bikes
Riders can likely race their gravel bikes in cyclocross races. CX races usually have many tight turns, which are taken at slow speed (relative to road races). A gravel bike not optimized for this type of course may fall behind. @gschenk stated correctly that CX courses tend not to have immense elevation changes, and thus CX racing bikes probably come stock with relatively narrow gearing, e.g. a 36t/46t crankset and 11-28 cassette, or an equivalent 1x system. Gravel bikes tend to come with both lower gears and a wider gear range, as their riders may have to endure extended slow climbs. For example, my gravel bike came stock with a 50/34 crankset and 11-34 cassette, and this is already considered high gearing. I don't find this gearing optimal for cyclocross races, and I suspect competitive cyclocross racers would agree. The small chainring provides too low gears, and the big chainring has too high gearing. However, a skilled rider would likely be able to work around the handling and gearing issues.
I close with a picture of the Cervelo Aspero, a very race-oriented gravel bike, being raced in a professional cyclocross race. The image is courtesy of Bike Rumor. To my knowledge, the Aspero has relatively nimble handling (it has less trail than my Chebacco in my size). This would make it suitable for CX races. If interested, read the article for information on how the team involved reduced the Aspero's stock BB drop, which is low compared to performance CX bikes, to increase the rider's ground clearance.
This is an optional outline on the basics of bike geometry. In general, some bikes are more stable. They need greater rider input to turn. In situations where the rider needs to make many tight turns, for example in a criterium race, a cyclocross course, or trails that many riders would use mountain bikes on, stable bikes may be at a disadvantage. More maneuverable bikes will be readier to turn. However, this may make them feel nervous when descending. A bike's stability is influenced by various aspects of the geometry including trail and head tube angle. The bottom bracket drop plays a role also, with higher drops placing the rider's center of gravity lower, and increasing stability. Also, there are some parameters that may affect a bicycle's maneuverability at low speeds more than its overall maneuverability/stability. I am not as familiar with this, but these play into the design of mountain bikes and some gravel bikes.
You can skip to the next section if you don't want details. If you do, this is drawn from an article by Cyclingtips. The photo below is from that article.
Unfortunately, this too is a complex subject, as the linked article illustrates. Here are the important parameters to focus on:
- Head tube angle: the angle formed between the head tube and the floor. The illustrative bike has a head tube angle of 72.5 degrees. A sharper angle tends to mean the bike is more easily turned, but also less stable. The bike's speed matters: a steep head angle is easily steered at low speeds but may feel too twitchy at fast speeds.
Trail is depicted in the image above, but trail does depend on the head tube angle (as well as fork rake/offset, which is 50mm in the first picture), and wheel size (note that this depends on tire size as well). Slacker head angle means higher trail. Higher trail will tend to increase the bike's stability (i.e. the front wheel will tend to want to remain centered and upright, as opposed to being quick to turn at the lightest input). Typical trail values for road bikes are 55-60mm. Gravel bikes and MTBs tend to have more trail, but loaded touring bikes have less for reasons discussed later. This parameter is usually not found in bike geometry charts, but it can be calculated from the given head angle and fork rake.
Wheel flop governs the bike's responsiveness to low-speed steering. The higher the wheel flop, the quicker the bike will steer at low speeds. Wheel flop depends to a large extent on fork rake and head tube angle. Keeping head tube angle constant, wheel flop decreases with fork rake. Keeping rake constant, wheel flop increases with head tube angle. This parameter is usually not found in bike geometry charts, but it can be calculated from trail and fork rake.
Bottom bracket drop is the vertical distance between the center of the bottom bracket and the center of the bike's dropouts. The bike in the first picture has 75mm of BB drop, and road bikes usually average 65-75mm. The more the BB drop (i.e. the lower the bottom bracket), the lower the rider's center of gravity and the more stable the bike. However, the ground clearance is reduced.
Wheelbase is the center of the front axle to the center of the rear axle. It isn't labeled in the example bike. Longer wheelbases are more stable but harder to steer in tight turns. Geometry charts will usually give the chainstay length rather than the full wheelbase. Chainstay length is part of the wheelbase. Jan Heine contends that wheelbase doesn't affect handling very much.