You're right, Andre -- it's partly a matter of what you're used to living with, the strategies & reactions you need and acquire. I've trekked through the Naukluft in Namibia, for example, but still feel a bit out of place in such settings, no matter their beauty & grandeur. My wife, from the US, has lived in New Mexico, Utah and such dry places, but still feels a little ill at ease with Canadian forests and winters, even from the inside of our very reliable Subaru station wagon. Mind you, I still carry sleeping bag-shovel-sand-metal traction aids-extra food & drink on winter journeys. My kids think I'm daft, of course, a relic of rural Ontario in the 1950's. (I just shrug and say, "Well, if you ever have a breakdown at 25 below and the sun's going down, better make sure it's on a well-travelled road.")
After a while, it also becomes a matter of what you want to do, much like your choice about not sailing in wild and dangerous waters: I turned down a chance to go paddling for two weeks this summer in North-Central Quebec, remote & beautiful though it is, because I'd rather go pedalling in Scandinavia.