While working on my bikes (a road bike with Ultegra 6600 groupset and a MTB with Shimano deore XT M8000) I noticed that the upper pivot on the Deore does not freely rotate. Looking at the exploded part fiches I confirmed it, there is no B tension spring and screw. Why is that? Why was it needed before?
There are no recent Shimano rear derailers without b-tension (angle) adjustment, other than maybe very bottom-end ones, but there are many where the look and location of it is different from older (namely pre-Shadow) models. RD-M8000 absolutely has a b-tension screw, although in the exploded view document they call it an "End Adjust Bolt" for no good reason.
Here is my understanding of the design goals and tradeoffs involved with sprung upper pivots:
The one and only reason you would want a sprung upper pivot is to maintain chain wrap and pulley/cog distance when shifting in front on a derailer where the guide pulley and cage pivot are offset (as opposed to coaxial as on most current MTB derailers, and including M8000). If you can make the cage pivot and the guide pulley coaxial, there is no movement of the pulley as the cage moves from shifting the front derailer. The reason for wanting the offset also has to do with getting sufficient chain wrap given the space you're working with and the gears you want access to. Were it not for this, all the spring would do is add chain slap, resistance, weight, and complexity.
The Shadow design, the larger pulley sizes now used, and the new emphasis on single or double as opposed to triple ring setups all create a dynamic where Shimano has decided they're better off without sprung upper pivots on MTB RDs. SRAM has done the same. There may also be an aspect to it where the sprung pivot plus offset guide pulley design gets impractial with very wide range clusters - I'm not sure of that but I believe it starts requiring a lot more offset to work.
It's nothing new. Despite being out of favor for most of the past 4 or so decades, there have been a lot of unsprung upper pivot rear derailers. The spring's necessity is something manufacturers have been historically on the fence about.
Here's what Shimano officially says:
The new DURA-ACE Di2 rear derailleur sees a ground-up re-design which takes advantage of Shimano’s Shadow RD architecture first introduced for mountain bikes. This low profile design delivers a sleek appearance while improving [its] ability to withstand crash damage. The new front derailleur delivers unmatched shift quality, for quick and smooth performance even under high pedaling torque. Combined with Shimano’s Synchro Shift customizable programming, DURA-ACE Di2 delivers a level of intuitive shift performance never before seen.
and also this:
- Shimano Shadow RD low-profile design
- Pulley maintains consistent distance to cassette for increased shift stability
- Less damage in a crash
- Maintains compact and smooth appearance
Unless a Shimano employee or someone else with insider knowledge can provide actual reasons, we can only speculate.
tldr: I think it's Shimano playing it safe by reusing current well-proven MTB tech to support wide range cassettes and 1x.
The quotes above have been used for ten years to market "shadow" rear derailleurs, but I very much doubt that crash damage or appearance were driving factors all. My hunch is that Shimano reused already existing RD design approach used for MTB RDs to improve shifting over wider range cassettes, Ultegra and Dura-Ace didn't have an 11-34 cassette before, the maximum was 11-32. I think that the fact that M9000 was initially launched with 11-42 cassette max which was claimed to be "specifically designed with a wide gear combination and carefully balanced ratios", but several years later 11-46 M8000 suddenly becomes acceptable only proves that Shimano tends to play it safe and might be inert towards modern trends that competitors capitalize on, specifically 1x. Maybe future plans play a role too (wider body, more cogs, XDR-like drivers, 1x road/CX) and Shimano decided to use more suitable design now and refine it later.