Tons of new feed fetcher bots are classified as browsers

Avatar
  • updated
  • Fixed
Question: How does or should WLE classify feed fetchers?

After the Google Reader shutdown a whole lot of feed reader bots have popped up that WLE doesn't recognize yet. Google Feedfetcher itself is classified as a spider, but I'm seeing a ton of new RSS bots that appear under "Browsers". Here are the most active ones from my server logs:

Bloglovin/1.0 (http://www.bloglovin.com; x subscribers)
Superfeedr bot/2.0 http://superfeedr.com - (here come various quips)
repparser/0.1 (python) (no idea what this one is!)
Zemanta Aggregator/0.8 +http://www.zemanta.com
Reeder/1.5.7 CFNetwork/609.1.4 Darwin/13.0.0

There are tons more. The new AOL Reader has just entered the fray, for example, so this zoo is only going to get bigger for a while.

There are also hits from standalone feed readers like FeedDemon and RSSOwl. All of those had been historically classified as browsers, though, so I guess they can stay that way.
bots feed spiders
Avatar
Michael
We'll add support for new RSS bots in the next version of the program.

If it is possible, could you send a recent log from your site to us at support@weblogexpert.com? We'll use it to check new bots to be added in the next version.
Avatar
Christoph Nahr
Excellent, thanks. I'll send you last week's logs, they amount to ca. 750 KB gzipped. Let me know if you need more.
Avatar
Michael
We have added support for the most popular RSS bots in version 8.2 beta, you can download it from http://www.weblogexpert.com/download.htm.