I think there is a case to be made that you will be less visible in dark colours. Anecdotally whenever I look down the road at a group of cyclists, it's the ones in large blocks of bold colours (not necessarily fluorescent) whom I can spot first. Bright but mixed patterns of colour are also less visible from a distance. I'm not saying anyone should be berated for wearing black - plenty of cars are black/grey/dark blue and certainly it's not fair to blame them in the case of an accident, but I do believe there's an increased risk. For something better than my opinion, consider the accident statistics for black cars.
During daylight hours, black cars were up to 12 per cent more likely
be involved in crashes than white vehicles, while at dawn and dusk,
the figure rose to 47 per cent.
It's not a huge stretch to extrapolate this to cyclists.
As others have pointed out, how easily others can see you depends on the lighting conditions. Any time cars will have their lights on you should have as much reflective material (Scotchlite or similar) as possible. This isn't limited to clothing, you can buy tape for your bike in a variety of daytime colours (including black!).
Finally, whatever colours you wear, I would suggest using been-seen-by* lights at all times. Yes, even during the day. In many situations I believe this will make more difference than the colour of your clothes. Lights can be noticed at a glance and instantly identify you as a vehicle moving on the road, rather than a pedestrian or part of the scenery. However, be aware that even lights which provide side-visibility are often not bright enough to make you more visible from the side in daylight. For side visibility in daylight the best thing is to wear large blocks of bright colour.
* Be-seen-by lights - not necessarily bright enough to allow you to see the road ahead, but to make you visible to other road users)
In addition to the advice above, I want to make clear that I would not blame you for wearing black clothes. It may be sensible to wear large blocks of bright colours and it may reduce your chances of being in a collision with another vehicle. However, unless you're breaking the law, say by not having lighting after dark or reflectors, then I don't think you should be blamed for wearing black. In my view it is the responsibility of all road users to avoid collisions, even with dark objects like fallen trees, wandering cattle or cyclists in black. If the conditions mean they can't see well enough to be sure of avoiding a collision, they need to slow down. I say this as I hear a lot of victim blaming in the media about cyclists wearing black.
I'd suggest that black will show the dirty less easily than lighter colours. Most of my bright-yellow clothing is spotted with muddsplatters and the occasional oil stain which would hardly show on black. Either way Goretex does require careful washing to keep its waterproof and breathable properties. I prefer not to use these additional products and just rinse in clean cold water instead.