Clip-less (and before them, clips) came out of the road riding community where the emphasis is on power transfer and efficiency. The most efficient position is the ball of the foot on the spindle (+/-1 a few mm). if you look at all early, and most modern, road shoes and cleats, its clear they are not suited to mountain biking and are designed for one task - the no compromise transfer of as much power from the rider to the bike as possible.
When MTB'ers started to use clipless, the advice remained "ball of foot" position, however by the late 1990's it was shown for MTB riders this is flawed - that the stability and control offered with the foot further forward (In the extreme, spindle under the arch) provides a far better outcome when riding. How far forward/back is always a trade off between efficiency and control/balance. With flats, its not an issue - move your foot forward fro the gnarly downhill, back for an uphill grunt.
It took a long time for the ideas of those pioneers to been widely acknowledged. Its only in the last few years (its taken this long) MTB shoes are being manufactured with the slots for cleats further back than previous models - I knew elite riders in the late 1990's who drilled holes and mounted their cleats behind the slots of shoes available at the time.
You will still hear a lot of people repeating the dogma that the ball should be over the spindle, they are very wrong, and the more extreme and technical the Mountain biking, the more wrong they are.
Another question that needs to be asked by every Mountain Biker who rides clip-less is 'Why, what advantage do they give over flats'? - the reason I mention this is if the roadies use clip less for one thing, improve power transfer, and the cleats back position we use on MTB is a compromise, whats the gain we are getting.
