A lot of mystique grows up around the regulation of many sports. The key (as Neo says) is to get to the source ...
UCI regulations say
Section 2: bicycles
Bicycles shall comply with the spirit and principle of cycling as a sport. The spirit
presupposes that cyclists will compete in competitions on an equal footing. The principle asserts the primacy of man over machine.
Freewheels, multiple gears and brakes are not permitted for use on the track during competition or training.
See UCI CYCLING REGULATIONS GENERAL ORGANISATION OF CYCLING AS A SPORT PART 1 GENERAL ORGANISATION OF CYCLING AS A SPORT (pdf).
The main reason behind this rule is safety. Crashes are, and always have been, common in track racing. Anything that can drop onto the track is forbidden. Freewheels and gears can drop oil, or otherwise fail, causing a crash. Gears (currently) always have some loopy control cable that could entangle another competitor.
Brakes are unnecessary because of the closed environment and the use of fixed wheels. Levers and cables can fail, and brake blocks can come out. In any case they would add weight to the bike, which would cost time in sprints.
So in general it's all about minimising equipment failure possibilities, and maximising the competitor vs competitor aspect.