In short: This bot scrapes and returns up to 500 organic search results from Google SERPs for a provided list of queries.
Have a look at this example output file to understand what the results will look like.
More examples:
- How to export TikTok channels by hashtag using this tool (see the first column for input)
- How to export publicly visible phones and emails from LinkedIn using this bot
Video tutorial
Take a quick look at how I'm launching this bot:
Now watch me show how the bot works in detail:
How to scrape search results from Google?
- Sign up for Botster
- Open the bot's start page
- Enter your search keywords (each keyword must go on a new line)
- Pick the number of search results to parse
- Select the operating system
- Select the target device type (desktop or mobile)
- Select your target language
- Select the region you'd like to scrape the results for (city, country or state)
- Click the "Start this bot" button
You're done!
How does Google scraper work?
The scraper will return the following data:
This bot analyzes the Google results page returned in response to a given keyword query and visualizes the SEO-relevant elements in a convenient text feed containing:
- Website’s position on SERP;
- URL;
- domain;
- title;
- description.
An SEO expert can use this data to run search analytics and identify their platform’s immediate competitors in the organic results.
Input data
Starting the bot will open a page containing several mandatory and optional fields.
The first step is to assign a unique name to the current query and to choose or create a folder that will store the output files. Afterward, you can proceed to the following settings:
- Region – narrow down the results to a certain geographical location, in which case the bot will filter out all irrelevant domains. Leaving the field empty results in a global search.
- Language – only show the results in a certain language. Leaving the field empty results in a global search.
- Top positions – the number of search positions you would like to be displayed (10, 20, 50, Device – filter the websites according to the types of devices they are optimized to run on (PCs, tablets, smartphones).
Input the keywords you would like to analyze in the next field. If there are several, a new line should be used for each.
The last set of options allows to opt-in for notifications via:
- e-mail;
- Slack;
- Telegram;
- JSON POST request.
How to interpret the results
You can use the results in:
- SEO
- Analytics and data analysis
- Competitive analysis
After identifying the resources that tend to receive preferential treatment from Google and land higher on the results page, an SEO expert can analyze the features they share and optimize their platform accordingly. This applies to design, tags, keywords, titles and so on.