Just type about:config into the address bar, and then filter by browser.chrome. You should see something similar to this:
Set the value of these two preferences to false by double-clicking on them.
You’ll have to restart Firefox, but the favicons will be gone entirely.
browser.chrome.site_icons
Personally, I love favicons, but this article was a request.