I went on two bike tours in 2017 and 2018 around North America, for 3000+ miles each, and this is what I did to secure the panniers to the bicycle:
That said, I did take some precautions.
Whenever possible I locked the bike in a highly visible location, preferably with some security cameras pointing at it, and preferably visible inside the restaurant or shop.
Bicycle in sight outside Arby's, Altoona, IA, USA
On several occasions I decided not to stop at a place where I couldn't find a place to lock the bike where I could keep an eye on it or where there wasn't a camera pointed at it.
Once or twice I noticed someone showing much more than the normal curious interest in the bike, and all it took was for them to realize someone was watching them and they moved on.
I always brought the bike inside wherever I stayed at night, including directly into my hotel room.
Bicycle in hotel room, Niagara Falls, ON, CA
Most of the really valuable stuff (laptop, etc.) I had in the backpack strapped to the bike above the panniers. When I had to leave the bike, I was able to easily remove the backpack and take it with me. It also had a change of clothes and some other emergency stuff in case something did happen to the bike.
I never had anything stolen, though I admit I might have just been lucky.