There are a lot of more potentially useful avenues to investigate first, because you feel that chamois pad quality is not the culprit next most obvious choice is the saddle:
- Stock saddles are often not very high quality. They are also typically too soft (aka not supportive) which will cause problems on longer longer rides. Adding to this, if you still have the original saddle, it is 4 years old now. The padding may have simply become compacted and unsupportive. Being a heavier rider you would likely benefit from a saddle that uses more dense/supportive foam, as well as one that has a stiffer body to better support weight.
- Is the saddle angled correctly. The Allez should have come stock with a Specialized body geometry saddle. It should have explicit instructions on how to angle it correctly. If the saddle is not tilted correctly, your pelvis will not be angled correctly and you may not be distributing your weight as evenly across the saddle as it could be. This can cause hot spots, and other problems
- How is your position on the saddle? Saddles are often designed for a particular body angle. Some are designed for a more upright posture, while others are designed for a more aggressive body position. This affects pelvis position/tilt and how you interface with the saddle. For example, a saddle designed for more aggressive body positions may be too narrow for a more upright riding position which can cause hot spots and discomfort.
- How is your pelvis position? Ideally, your pelvis should be rolling forwards like a pitcher pouring water (especially for endurance riding). Many riders due to bike fit issues find themselves rotating the opposite direction (sometimes to protect the prostate, due to a poor saddle) which can change how you bear weight on your sit bones and cause high pressure areas and discomfort.
- Is the saddle the correct width for you? Saddles are like shoes, one size does not fit all. There are tools to measure the width of your sit bones in order to ensure that you are matched to the correct width of saddle. Too narrow and you will not be supported properly, too wide and it can interfere with pedalling and cause chafing and hotspots.
Expanding on (4) pelvis position (@gaurwraith)
This passage from "Bike Fit" by Phil Burt (lead physiotherapist for British Cycling and a consultant physio for Team Sky) describes pelvis position.
POSTURE
Posture is the maintenance of a certain body position and requires appropriate joint mobility, joint/muscle coordination and muscular endurance. Limits in any of these elements can result in postural irregularities. Good posture on the bike required good flexibility through the hamstrings and the glutei muscles: this allows the pelvis to roll forward, keeping the back in a straight position while reaching the handlebars.
One major factor limiting the back's ability to remain relatively straight while on a bicycle is thoracic immobility: lack of movement in the middle of the spine normally results in the spine flexing too much. Excessive spinal flexion while on a bicycle will limit breathing and compromise your ability to stabilise your spine for torque production to the pedals
I personally when through a episode of mid-to-upper back pain caused in part by the issues described above. I was posteriorly rotating my pelvis (rolling towards the back rather than forward), which also affected how I sat on the saddle. My Brooks saddle developed very deep sit bone impressions and modern saddles quickly became uncomfortable. As I regained my back flexibility, I noticed I could roll my pelvis much farther forward evening out sit bone pressure on modern saddles, which made them comfortable again.