Once again, I spent some quality time debugging a slow leak and found out it was a loose valve core. I tightened it with pliers, but it raises the question, is there a situation where it is useful to be able to remove the valve core? Also, have replacement valve cores been available at some point, and do the cores have a standard thread or is it manufacturer-specific?
As others have mentioned one reason is to make it easier to installed slime/sealant into a tube or tire. Typically you'll take the valve off, and use an injector to put the slime into the tire.
For tubeless tire setups, removing the valve core allows a higher volume of air to quickly get into the tire when you are first blowing it onto the rim.
Some types of valve extenders used by people with deep profile rims thread into the valve core threads.
Another reason to have removable Presta valves is they tend to be somewhat fragile so being able to replace a busted valve vs. having to replace the whole tube could potentially save a little bit of money (or allot of money if we are talking about breaking the valve core on a $150 tubular tire.)
The valve cores should be consistent from maker to maker.
I think the big benefit of a removable core is that it makes it possible to add "self-patching slime" to the tube. With a core in place the slime would likely foul the valve and make it inoperable. Otherwise, I think the removability is more an artifact of the manufacturing process – and as you've noted, not necessarily a feature.