I bought a new floor pump and it pumps everything perfectly fine, the problem I have is that it only shows a pressure reading while I am pumping and the moment I stop, the gauge drops back down to zero. I wanted to know if this is normal. I will note that I have not tried pumping anything with high psi, only tried it for things less than 12 psi.
Does the pressure reading slowly drop to zero from non zero values when you stop pumping? If so, it means that the air is leaking from the gauge, and generally from the pump. There are three places where it can happen.
From the tube itself, meaning it is damaged. It has to be a pretty huge hole for the depressurization to be immediately apparent. Basically, the tube won't hold air at all. In such case, fix or replace it.
From the pump, meaning it is damaged. Given that it is a new pump, it is unlikely, but if this is the case, your best solution would be to replace it by warranty claim.
From the connection between the tube and the pump, specifically at the chuck/valve. This means the pump head does not sit tight on top of the valve. Disconnect the chuck and reattach it again, it may take several attempts to achieve a tight enough connection. Be sure to put it squarely and press lightly on it when closing.
12 psi is also a quite low value. Traditional floor pumps have their gauges going up to 120 psi at least. As it is for basically any mechanical gauge, the lowest 10% of its range are not accurate at all (I mean relative error of 50% — 100%) because of friction in the gauge itself. The readings are basically random, especially on cheaper pumps. Besides, the arrow will jump around simply because you apply varying pressure to handle when pumping. The readings will become more stable once you reach values of 30 psi.
If you really need to pump your tires to a very low value (< 20 psi), you are in need of a separate precision pressure gauge. When I pump my fat bike tires (desired pressure is between 10-15 psi), I ignore the pump's gauge readings and purposefully slightly over-inflate them. Then I use a separate digital gauge which has precision of 0.1 psi, and actually deflate the tires back until I obtain desired pressure values.