I have an app that checks your website for issues like broken links and SEO issues. Since a couple of weeks, the actions of this crawler show up in the Google Analytics accounts for the visited websites. I would like to prevent that, because it messes up the GA reports of my users.
The crawler is not executing any javascript and has this user agent:
Mozilla/5.0 (compatible; Appname Crawler; +https://www.nameofmyapp.co/crawler)
I would expect that GA would be smart enough to recognize that this is a crawler. It used to, but since a few weeks GA counts the visits of my crawler as actual visits, even when the Bot And Spider Filtering option is on.
How do I prevent this from happening?
2
Answers
Found it: my bot is using the Google Pagespeed API to check how fast the page is. Those requests end up in GA.
Under Admin > View settings you could add an Filter with an UserAgent you want to exclude.
Alternatively your Bot should block the GA Script, so your visit should not be recognised.