One issue is heel clearance. You can normally sort this out but not with large panniers especially with big feet.
A bigger issue is fitting a rack. Your options are very limited if you have rear suspension (including low weight limits); even disc brakes are a restriction, though not as much in the last few years.
Panniers also add quite a bit of width where you don't want it on narrow trails. Typical bikepacking bags are narrower and often higher.
On steep uphills you don't want too much weight at the back - the front wheel will lose grip or even contact. This is worse on MTBs than even the same trails on a bike with a longer wheelbase, no suspension, and harder tyres. Steep descents are less of a problem in that you wouldn't normally do them fully loaded.
I suspect that the MTB touring you've seen isn't as heavily loaded as the road/trail touring you've ridden.
These factors combine to influence another: habit. People copy what they've seen, often applying this copying to milder conditions (e.g. using solutions that are necessary on a full-sus, but on a hardtail).