There is a good "common language" guide to the bike specific regulations in the Highway Code here.
The relevant paragraph reads:
At night your cycle MUST have white front and red rear lights lit. It MUST also be fitted with a red rear reflector (and amber pedal reflectors, if manufactured after 1/10/85).
So, while I am not a barrister, nor a UK citizen, the answer appears to be that you have to have a rear reflector somewhere. However, there may be some wiggle room since the law specifies at night. It seems unlikely that if you only ride during the day that you will run into a problem if you remove it.
Most jurisdictions in the US and other countries have similar laws. While safety experts generally agree that a rear light is much more effective (and UK law seems to recognize this by requiring not only the light, but that it be lit), you can usually get away with a rear light that is also made out of red or amber reflective material.