Filter spider entries from log
Hi there, we have an exceptional number of 302 events related to spider crawling which appear in our downloaded files report. Can these be filtered from the collection and reporting process? An example follows. TIA, Kim.
2019-07-31 02:08:17 172.31.255.10 GET /Materials/Presentation_to_Integrity_Legal_John_Lightowlers.pdf - 80 - 157.55.39.202 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 302 0 0 250
2019-07-31 02:08:17 172.31.255.10 GET /Materials/Presentation_to_Integrity_Legal_John_Lightowlers.pdf - 80 - 157.55.39.202 Mozilla/5.0+(compatible;+bingbot/2.0;++http://www.bing.com/bingbot.htm) - 302 0 0 250
if you wish to exclude them from reports at all, you can use either the Spider filter (with value Bing Robot) or User Agent filter (with value *http://www.bing.com/bingbot.htm* ).