Again - Following up with an opinion from a self confessed conspiracy theorist. Its about the money - follow the profit tree to its roots
When there were only 26" wheels a small LBS needed to have one wheel set in stock, and one sized tire ( maybe 5-10 variations). They would special order specific gear the the customer wanted and could easily stock enough to supply of the shelf something suitable for most customers most of the time. The LBS had a range of MTB in stock - a few (maybe 10 bikes) would cover the requirements for 95% of his customers, and he would target a particular but broad market (Typically either Family and weekend riders, or 'cashed up cost doesn't matter wannabes and elite cyclists).
Now, the small LBS needs to stock 3 wheel sizes, in disc and rim brake style (6 wheel sets). They also need about 50 different tires on the shelve. The number of bikes required to be stocked by the LBS has also increased. He needs at least twice as many on the floor to remain competitive, but that much stock will drive him out of business. This is beyond the capability of a small LBS to do and still remain profitable. Mum and Dad now need to stock 3 different spare tire and tube sizes at home (Dads 29'er, mums 26'er and sons 27.5er').
Then there is the hype - 26'er is old, 29'er is so much better. Followed by "well, you need a 29'er for XC but AM and downhill a 26'er is better - to be a serious cyclist you have to have one of each (with spares of course). Now, we have 27.5 - So, that 29er I sold you last year really is to big, and you know that 26 so last decade, well, have I got the bike for you.... It's only $$$, but I'll throw in a discount cause you are such a valued customer.......
In effect it is being pushed by the larger retailers, who can afford the stock level, to push the small LBS out of business. The elite custom-builders where pricing is less of a concern will do very nicely out of it as well (No competing LBS focused on "reasonable pricing" will drive some people to them).
The net effect of all this choice is that the cost to us will skyrocket as the competition is shut down. We won't have competing LBS that offer personal service (except those that provide for the high end market) and will have to deal with the large, impersonal chain stores. The idea of talking to the mechanic will be seen as quaint, as the salesmen run the front counter. "Why talk the the mechanic, buy a new bike.... "
I believe the changes are mostly (99.99%) profit motivated to get more bikes moving from shop shelves to unused garage ornaments.