A large part of it has to do with the engineering constraints on a shifting system and what hydraulics are actually good at.
For a shifting system to work reliably, the shifting mechanism has to index to exactly the same place every single time. Some limited drift is OK, but it’s got to be small enough that you don’t need to re-adjust before every ride, or even every hundredth ride, and it also has to be easy for the person using it to correct as needed.
Provided you’re using good hardware (especially a good bowden cable), the traditional mechanical arrangement rather trivially manages those constraints. Drift is limited to slippage or stretching of the cable (and thus is always in the same direction, so you can even offset it a bit ahead of time), and ‘fixing’ drift just requires minor adjustments that anybody could do with one or possibly two relatively cheap tools.
Some more advanced systems, like Rohloff’s Speedhub, go a step further and essentially eliminate the drift issue by doing all the indexing internally and using a cable loop for a push-pull arrangement to actuate things. Cable slippage is a non-issue for these, cable stretch just makes the sifter a bit sloppier, and the internal indexing mechanism will generally last longer than the rest of the hub.
Wireless shifting also sidesteps these issues by moving the indexing to the derailleur or hub and then eliminating the mechanical aspect of transmitting ‘commands’ from the shifter to the drivetrain (and thus removing cable slippage and stretch as issues completely).
But hydraulics really aren’t like that for a couple of reasons:
- They rely on the incompressible nature of the hydraulic fluid, but it’s impossible to make one that truly never leaks and can reliably transmit motion from one end to consistent motion at the other. And if it leaks, air will get into the system, and air is not incompressible, so it will significantly impact how the motion is transmitted. A (small) bit of air in the lines is not an issue for brakes because you can always just apply more pressure, but it is a major issue for something that needs precise positioning control. In purely mechanical terms, you can think of it like having a small spring replacing a section of the bowden cable. A bit of creative thinking should reveal all the issues that would cause trying to shift. Even outside of cycling, it’s very unusual to see hydraulics used in cases that really do need exceedingly precise motion control (the closest you get is hydraulic steering on some off-road vehicles and boats, but that also falls into the ‘just apply more force’ case when air gets in).
- Unlike mechanical systems, proper maintenance of hydraulic lines for a bike requires a workstand, specialized tools that aren’t much use for anything else, additional materials (you need extra hydraulic fluid to properly bleed a system), significantly more time (by at least an order of magnitude, and additional skills beyond basic mechanical knowledge. This is not something most people are ever going to consider doing themselves.
- Because such high precision is required, maintenance would have to happen all the time. This is compared to brakes, where bleeding the lines is often only needed after more than 1000 km of usage because, as mentioned above, you can always just apply more force to your brakes. For a shifting system it would probably be 100-200 km of usage, far too frequently for most people to want to deal with it.
Part of the issue could possibly be mitigated with a two-line system in a push-pull arrangement, but that would involve twice as much work for maintenance, would still need regular maintenance rather frequently for consistent behavior, and probably would not behave like most people expect their shifter to behave.
