Looking into an IGH it seems the biggest disadvantage is having a huge amount of rotating mass. I've seen one handmade bike where the IGH sits above the bracket, inside a carbon fibre shroud around the place some bikes have rear suspension. There are two chains, one driving the IGH and another to drive the rear wheel.
Why not have an internally geared bottom bracket where the cranks drive the input shaft and the crank ring rotates at a ratio selectable by the gear lever?
Then the rear wheel could have a single drive wheel driven by a chain or belt and a lightweight hub just solid enough to mount the spokes and disc. Then you could have mud-free gearing while reducing rotating mass even below what you have with a conventional cassette... and given that you have thrown out the bottom bracket the overall weight gain would be less.
Quite a job for the home builder to adapt an existing hub I guess but it should be possible.