Usually shoes for walking/running and cycling shoes are constructed quite differently: shoes you walk on are usually far less rigid, most have a much softer sole to absorb shocks/bumps, and of course do not have cleats.
As this is ergonomics and people are quite different, there is probably no general answer: I find walking in SPD Trecking shoes uncomfortable, but not enough that I would not use them every day.
For your specific problem, I guess changing liners could help. However you'd need very stiff liners that actually distribute unevenness. I have not yet seen those.
Assuming you have static workplace: maybe you could arrange having a second pair of shoes specifically for walking there?