Just type about:config into the address bar, and then filter by browser.chrome. You should see something similar to this:

Set the value of these two preferences to false by double-clicking on them.

You’ll have to restart Firefox, but the favicons will be gone entirely.

browser.chrome.site_icons

Personally, I love favicons, but this article was a request.