So I forgot about the correct procedure and removed the expander plug on my carbon fork before unfastening the stem's bolts (carbon-specific stem, 5 Nm, unscrewed stem right after removing plug, and followed procedure after that, no cracking sound or visible signs of damage).
Should I be worried about structural damage? To what degree? Immediate? Long-term weekness?
I have no idea, and could not find anything on the web about this scenario.
There are various designs of expander plug on the market, both those supplied by fork manufacturers and others made by hardware and other component companies such as FSA. They reach various depths into the fork steerer, but usually not so deep as to fully cover the area the stem clamps on.
Bicycle manufacturers (TREK as a specific example) demand that at least one spacer is always present above the stem to ensure the steerer tube is not damaged, but you are allowed to put more than one spacer above the stem -- to the point that the expander plug is not involved in your clamping area at all.
Therefore, unless your stem is mounted too high on the steerer tube to the point that it represents a risk of popping off if the carbon deforms (I have a fork at the shop to which this has happened -- and the expander was present nor did it stop the steerer being damaged) you cannot damage the steerer tube by clamping a suitable stem under normal forces where the full contact area is in play unless the manufacturer specifically forbids this in a note with the bike/frame. Cervelo would be a specific exception to this, but they use a bonded metal sleeve with a star nut.
Modern carbon is a resiliant material and the steerer tube is specifically constructed, like a carbon seatpost, to take a clamping force within a specfic limit. I also have a carbon seatpost at the shop that has a crack running all the way up the back of it. It has been ridden by a light rider, who was totally unaware, for a significant period of time. Despite my best effort, I wasn't able to snap this part off either. The point is that modern carbon parts don't just snap off. See Danny McAskill riding some carbon rims to get an idea of what's possible.
A carbon steerer is part of a high performance bike and the fork and headset should be stripped and checked at regular service intervals both for good running and to check for defects. If you were to notice a propblem at one of your service intervals, then would be the time to consider replacement.
There's some distant hypothetical chance of damage from this, and it should be avoided, but it's almost certainly fine as long as no damage is apparent (cracking, delamination or separation of layers, etc). It's pretty common for it to happen by accident or as a result of the plug being installed loose and causing problems with headset adjustment. For most forks it would probably take the bolts being way overtorqued for damage to happen.
I would quantify the risks differently from Nathan. Remember that the fork steerer is hollow. The compression plugs support the walls of the steerer from inside against the stem's clamping force, which is actually a requirement because the carbon isn't strong enough to be clamped unsupported. Also, the potential issue is that initial damage to carbon can be invisible, and it can propagate with time. Hence, I'm not sure we can say the chance of damage is distant.
That said, components are all designed with a safety margin. For example, people may not have perfectly calibrated torque wrenches, or may tighten the bolts in the field, or they may mix the sequence of operations up as you did, etc. Engineers know this, so we generally should not expect the steerer to crumble to dust the instant you remove the plug.1 Also, you exposed the steerer to the unsupported clamping force for a brief period.
Unfortunately, the ideal case is that you or someone would inspect the steerer with ultrasound or some other advanced imaging. You presumably don't have the equipment at hand or the skill to interpret the readings, or you wouldn't be asking the question. While this answer may be vague and not fully satisfactory, I'd just urge watchful waiting. This involves periodic inspection. You would be looking for cracks showing on the steerer, or changes in feel at the front end. You could look at the steerer every week for a few weeks, then less frequently thereafter. You could take a reference photograph now so you know a surface feature was there a few weeks ago.
Footnote 1: Ultralight items are generally going to have much thinner safety margins. I assume you don't have one of these. Anyway, it's worth mentioning that the advice would lean much closer to replacement if this were an ultralight item.
