As you increase the rotor's diameter, the shearing forces on the bolts attaching the calipers to the fork or frame are only one of the limiting forces.
Beyond the usual diameters, you will need to pay special attention to the rotor's overall rigidity, starting by its thickness.
We can take a hint from cars, where the rotors consist of two solid pieces, themselves connected to provide a larger lateral rigidity.
The problem is that once you apply Euler's critical load formula on the rotor, you will find you need to increase its rigidity to the point where it has an unacceptable weight for a bicycle. Even though, as juhist and Burki argue, the frictional force remains the same for the same surface materials and caliper opposing forces, the length of the arc distance along which the rotor needs to transmit that force to the hub (opposing the force at the calipers) increases (by π).
This is the reason why increasing the rotor's diameter will result in oscillations under heavy braking, as MaplePanda reports. To see this, think of a rotor made of rubber. The rubber will stretch behind the calipers, which is not a problem for metal rotors, but it will also be pushed in front of the calipers, which, whether the rotor is made of rubber or metal given enough force, will cause the rotor surface to bend unless it's sufficiently rigid. It is this bending that the rider will feel as an oscillation.