Obviously the brakes do not place any wear on the rims. But are there other factors that could lead to the rims needing to be replaced? I'm excluding damage from accidents.
Aluminum alloys have a zero or very low endurance limit (amplitude (or range) of cyclic stress that can be applied to the material without causing fatigue failure).
Aluminum rims will therefore eventually fatigue and crack, most likely from nipple holes as this is where force is applied and stress is concentrated.
Spokes can gradually deform their spoke holes at the rim. That would eventually necessitate replacement.