I was planning on getting this old steel frame powder coated, but the guy doing the powder coating noticed some small holes in the top tube after he stripped it. Does this indicate the top tube is rusted out? Is there a way to check?
Get the end of a pick into a hole and gently experiment with the strength of the surrounding material. A rusted out tube will feel weak and may even break away.
Also the tone of a decayed tube when you tap it with a tool will be flat and damped compared to unrusted pipe which will sound ringy and bright.
Theres probably no way to get a pipe cam inside the tube short of drilling a hole though you could check inside the head tube and seat tube for access.
A XRay might reveal the inner condition but that’s kind of out of scope.
It’s an old bike so it’ll be made with straight gauge steel tubes and will be reasonably strong. In the absence of any obvious signs of decay, my instinct would be to carry on with the restoration and keep riding it.
You need to do some exploratory work - don't be shy here.
Stab into it with any poking-tool and define where the bad metal ends. It may be ten times worse than what you see now, but if you don't find it then your mind will worry every time you ride this bike.
As per WeiwenNg's idea, a borescope can be handy though be aware they may not have the best colour reproduction.
You should be able to access the inside of the top tube through the seattube or via the head tube. One of those should have a vent hole for hot gasses when the frame was originally welded. If neither does, then hot fumes may have been the root cause or a contributing factor, contaminating the top tube on the inside and allowing it to rust from the inside out.
If you don't have a vent port from top tube to seat tube, do take the opportunity to drill one at some point during this repair.
Repair options - you need someone who can weld steel nicely. Ideally a frame builder, but if someone that specialised is not available/affordable then a skilled welder should be able to cut open that area, remove all the damaged/thin metal, and fill it in with fresh metal from a welder.
The maximum area for filling might be up to 10-15mm and if your rust goes further than that, you need a plug. I'd suggest getting a steel tube of the same grade and outer diameter, and once the holes are cut in the top tube, cut a matching patch and weld that in place.
If the rust is all the way around the tube, you may have less work to cut out the entire top tube and replace it with a fresh tube. This would require releasing the old tube from the lugs and fishmouthing and fitting the new tube then brazing or welding it in place.
Another option is to leave stubs of the old top tube and use some kind of in-line coupler to butt-join the old and new sections together, though this adds weight and will look "weird"
Fancy answer is to cut the downtube as well, and add a travel connector there as well, making your bike into something that can be disassembled into a suitcase, or "rinkoed" for travel.
The pump peg in your photo is going to get in the way - if you don't ever plan on using it, cut it off and store it for future rebuilds. If you want to keep it, again store it until the main repair is done.
In practical terms - this repair may cost more than a good used bike. You could save the forks and all other components, and transfer them to a better frame. Its not ideal especially if this frame has sentimental attachment, but sometimes money is the deciding factor.
You might choose to rattle-can spray this frame and hang it on a wall somewhere as art and a memory.