A simple explanation I've heard is that metal tubes are isotropic - their material properties are identical in all directions (tube shape and thickness being constant). Carbon fiber can be made fundamentally anisotropic, i.e. their stiffness and strength differ in various directions.
I am not an engineer, but this brief synopsis was given by Josh Poertner on his Marginal Gains podcast. He has extensive engineering experience in the bicycle industry.
I also suspect he was simplifying for a non-engineering audience. For one, we are not limited to round and only round metal tubes of equal thickness throughout. For quite a long time now, we have had butted metal tubes, i.e. the tubes are thicker at the ends where you do need more strength because you're welding that tube to others, but they're thinner towards the tube center; for spokes the butting is external, but for frame tubes it's internal so you can't see from outside. For another, you can shape the tubes. For example, I know that at least some metal down tubes are oval where they join the bottom bracket - the greater lateral cross section makes for a stiffer bike. More complex shaping than that is available, as the comment below alludes to. Within material limits, you can change the diameter of the tubes as well, where larger and thinner walled tubes are stiffer than smaller and thicker ones. Somewhat related to that, variations in metal alloys can enable slightly different material properties. For example, 6/4 vs 3/2.5 titanium, steel with niobium in the alloy mix may have enabled very thin-walled (and thus large diameter) tubes that are stiffer, various alloys of aluminum. So, metal may be isotropic if you hold the tube dimensions constant, but there are still numerous ways to adjust the frame properties of metal bikes.
For another, it may be more correct to say that most carbon fiber structures can be quasi-isotropic, although you can also make them close to anisotropic - it might be more technically correct to talk about degree of isotropy. You can, I believe, make an isotropic carbon structure if desired.
Regardless of the specifics, it appears you can vary the material properties of a carbon structure more, and I believe much more, than a metal structure of the same shape. With carbon, you're not limited to changing the shape and wall thickness of the tubes involved. There are more degrees of freedom to vary the properties in.
One last bit as to the notion that carbon offers a more compliant ride. It is true that carbon can damp vibrations. However, a lot of your vibration damping comes from your tires. Even performance road bikes these days can take relatively large tires. You can safely run larger tires at significantly lower pressures than we all used to, and you are likely to have less rolling resistance, not greater (this is actually a complex topic, and that statement is an oversimplification). Also, steel and titanium have long been regarded as comfortable rides; they may feel qualitatively different than a carbon frame, but none of those materials may be clearly superior in overall subjective comfort. For that matter, aluminum frames have improved considerably since the 2000s, even though that material has been overshadowed by carbon. It's possible to make a comfortable aluminum bike as well. It is not pure marketing to state that carbon can be made not isotropic, so a carbon fork can be more compliant vertically (i.e. it damps vibrations) than it is laterally (i.e. it doesn't flex side to side when you pedal). It is not marketing to state that carbon bikes can be designed that way as well.
For fork material, it does appear that carbon has come to dominate. Steel forks can also be comfortable, but they do have a weight penalty. While the amount of weight in question actually only has a minor impact on performance in most cases, it is a pretty visible parameter, and anyway steel forks don't seem to be a mass production thing for performance bikes. (FWIW, I have a custom steel bike with a steel fork.) I'm pretty sure aluminum forks were always harsh. I know that titanium forks never made headway, in part due to the cost, and possibly because some of the earlier ones had failures. I'm not sure if that was a quality control issue or something inherent to material properties.