Carbon wheels are strong enough for gravel and mountain bike riding now, after advances in construction techniques and materials. It’s true that carbon wheels used to be regarded as too delicate. For sure, I just listened to a podcast (Marginal Gains) where they recapped the history of carbon wheels at Paris Roubaix. There was one year in the late 2000s when Zipp lost a lot of carbon wheels to breakages. This was very much influenced by the narrow tires and high pressures (relative to today) that people were running. Then, one year, Fabian Cancellara won on Zipp 303s, and subsequently everyone adopted them.
If you want a wheelset to potentially pull double duty on road and off road, then you might want to look for something with 23-25mm internal width. Gravel wheels are, I believe, settling on a 25mm width. Some but not all road wheels are going this route. For example, Campagnolo’s Bora WTOs have a 19mm width, and Shimano’s Dura Ace 9270s have a 21mm width. These are better for 25-28mm road tires. Zipp and Enve use 25mm in their latest road wheels, and those are probably best for 28-32mm road tires, as well as being able to accept gravel tires.
I believe that the aerodynamic gains of carbon wheels are muted by gravel tires. For one, the big tire changes the overall shape of the leading and trailing edge of the wheel away from a teardrop shape. Also, tread adds turbulence, which equals drag. This doesn’t mean the gains are zero, it just means that it’s less clearly worth it to buy a carbon wheelset for a gravel bike. There are carbon wheels designed for gravel racing, but they’re much wider, e.g. 3T’s design.