In my youth, my cousins and I wandered the backwoods in the Appalachians in Western Maryland with .22 and 30-30 rifles, a .45 1911 and a Persian Mauser. None of us wore ear protection. None of us are actually deaf 60 years later, but I do suspect we suffered damage. Interestingly, I don't remember the sound being as painful, as it was last year when I forgot to replace my ear protection and fired the R9 at an outdoor range (in a berm cut into a hill).
Point to consider, i.e.: the case of the "car gun." I've been told firing a 9mm (high pitched sharp "crack", or a 357 will likely cause damage while a .38 might be a better choice. Nevertheless, in an extreme situation the risk of some hearing loss would be an easy choice.