Documents & Resources
Welcome to the BotGuard Knowledge Base
BotGuard Knowledge Base
The BOT Traffic Research Initiative
Operated by BotGuard OÜ
Join website owners and leading researchers in the BOT Traffic Research Initiative. This initiative is aimed at gaining a better understanding of the nature of global web traffic and the effects of malicious traffic on eCommerce sites.
Who Is this Project For?
We welcome all businesses with websites, web servers, and eCommerce platforms to apply! The growing share of “bad” traffic poses many risks to online stores and other websites, from ad fraud, to denial of inventory, to credential cracking, the list is quite extensive. This project uses a number of novel detection techniques not currently available in most mainstream commercial products, granting you insight into your web traffic patterns and the types of visitors browsing your site.
Participation for website owners requires a website (or several websites) with significant web traffic.
Hosting companies and server owners that have a high amount of traffic, or that prefer the traffic not to leave the data center where the web servers are installed, will need to provide a server to monitor the data. Server requirements are quite moderate, a regular VPS will be sufficient.
How Does The Project Work?
During the course of this global research project, we collect the most complete data on global bot traffic and make it available to a wide range of academic researchers. The data is analyzed and the data contributors (Website owners and HSPs) receive personalized reports on their own traffic, as well as access to research results.
The project can be broken down into 3 stages:
Website owners and Hosting Service Providers (HSPs) work with BotGuard as a platform supplier to set up traffic monitoring. All data is anonymized by design. Only website owners have access to the full version of their data. We use a copy of the traffic data samples, which means that there is no impact to production and no security risk.
We provide several methods of integration including web server extension modules (for Apache or NGINX) and ingress traffic controllers, which can be installed on a separate virtual server and connected to the router to analyze a mirrored copy of the inbound traffic. We also provide detailed guidelines for choosing an integration method. No changes to the website code are necessary.
Anonymized traffic data is provided to researchers. Researchers use the real-world data to conduct any type of relevant research study or project that can benefit from this type of information. They also perform detailed analyses of web traffic for website owners.
All participants will have access to research results carried out on the basis of traffic monitoring. Website owners and HSPs will receive a valuable, in-depth analysis on the nature and sources of their own traffic, as well as access to the global eCommerce traffic data collected during the course of the project. All businesses participating in the project will receive the status of an official partner and the opportunity to use this status in their marketing activities.
Frequently Asked Questions
What type of industry does the data come from?
Inbound traffic data is provided by many different types of websites including media sites and eCommerce stores: lifestyle, electronics, fashion, sporting goods, groceries and food, health and medicine, beauty, etc. Virtually any type of business with an online storefront or website can contribute their web traffic data.
Where does the traffic data come from?
The traffic data is sourced from participating hosting providers and website owners from around the world, including Germany (Frankfurt, Falkenstein, Nuremberg), Finland (Helsinki), United Kingdom (London), Italy (Milan), Netherlands (Amsterdam, Meppel), Poland (Warsaw), Russia (Moscow, Saint Petersburg), Singapore, USA (NJ, NYC, Silicon Valley), New Zealand (Auckland), Australia (Melbourne, Ipswich), and Brazil (São Paulo). We continue to install the necessary hardware and software in more locations to ensure maximum global coverage.
Is there any cost?
No. Participation is entirely free and voluntary.
Supported by Leading Academic Institutions
Prof. Dr. Gholamreza Anbarjafari
Head of iCV Lab,
Institute of Technology,
University of Tartu,
Narva mnt 18, Tartu 51009
About The Project Operator: BotGuard OÜ
BotGuard recognises the need for more transparency online, and has a unique platform to support a traffic monitoring initiative. We operate within this project as a platform supplier to set up traffic monitoring for participants. We work together with the scientific community to provide a clear and up-to-date picture of the global web traffic.
Join Our Research Panel!
Meet the project coordinator and Head of Global Partnerships — Danica Gallon
To participate or find out more about the program, please contact the project coordinator directly at firstname.lastname@example.org.