They are not supposed to do that, but some clients have an option to enable it anyway.

There are also clients that make a second request to ask for a favicon. In the spirit of the protocol the "icon" is just a UTF-8 symbol, but that behaviour is still controversial.