Silence is not the same as quiet. Our office isn't silent. It's softly quiet.

How did you go about starting a culture of library rules though? Did it naturally start like that or was there a defining moment where it was addressed?

I think it's fair to say an environment that is so quiet you can hear a pin drop (i.e. ideal movie theater quiet) is not going to be welcome by most, for the reason you cited. But the same effect can occur in a noisy place as well, if a single individual is sporadically more noisy than the background noise.

So it's more about achieving some constant noise level, above which no sporadic noises rise. IMHO, the "library/whisper quiet" rule achieves this.

The problem for me is that it NEVER just becomes background noise that I can ignore. It is just constant, never-ending distraction that significantly reduces my productivity.

Wherever I've worked, no matter how quiet it's been, keyboards still make noise.

And the best keyboards are also the noisiest, :p.