I recall that Josh Poertner, who owns Silca, runs the Marginal Gains podcast, and previously worked at Zipp, discussed this at one point. I can't recall which episode this was mentioned in, and thus I can't cite it at this point.
My recollection is that he has said that skin, even shaved, is not aero compared to fabric - provided the fabric is reasonably tight and does not have wrinkles. I believe Poertner mentioned that the shoulders and hips are critical wrinkle areas. It seems intuitively obvious that wrinkles would add drag, but I'm not sure if he discussed why shaved skin had higher drag than smooth fabric. Do also note that fabrics differ in drag among themselves, as one of the comments indicates.
For examples of how performance cycling clothing has evolved, consider the photo below from the 2002 Paris-Nice race, posted on Cyclingnews and featuring Alexandre Vinokourov:
By comparison, the current peloton is in skinsuits (may alternatively be called speed suits), even for road race stages. In the 2000s, skinsuits were worn in time trials alone, and they usually didn’t have enough pockets. This is from the 2020 Strade Bianche race, featuring Annemiek van Vleuten at the front:
In summary, performance road gear has gotten much more form fitting over the years. Poertner may have mentioned on the podcast that the sleeves and bib lengths have got a bit longer as well, because fabric is faster than skin. The socks have gotten higher as well, up to the UCI's limit for sock height.