Let’s say you are working on a project that needs to do web scraping but you don’t know websites on which scraping is to be performed beforehand instead you are required to perform a google search and then proceed according to google search results to a few websites. In that case, you need google search results for your different queries.
- One way of achieving this is using request and beautiful soup which has been discussed here in Implementing Web Scraping in Python with BeautifulSoup.
- Instead of putting so much effort into a trivial task google package has been made. It’s almost a one-liner solution to find links to all the google search results directly.
- Using python package google we can get results of google search from the python script. We can get links to first n search results.
Installation
google package has one dependency on beautifulsoup which needs to be installed first.
pip install beautifulsoup4
Then install the google package
pip install google
Required Function and its parameters
- query: query string that we want to search for.
- TLD: TLD stands for the top-level domain which means we want to search our results on google.com or google. in or some other domain.
- lang: lang stands for language.
- num: Number of results we want.
- start: The first result to retrieve.
- stop: The last result to retrieve. Use None to keep searching forever.
- pause: Lapse to wait between HTTP requests. Lapse too short may cause Google to block your IP. Keeping significant lapses will make your program slow but it’s a safe and better option.
- Return: Generator (iterator) that yields found URLs. If the stop parameter is None the iterator will loop forever.
Python codes on how to do a google search using python script
Example1: google_search.py
Python
try: from googlesearch import searchexcept ImportError: print("No module named 'google' found")# to searchquery = "GeeksforLazyroar"for j in search(query, tld="co.in", num=10, stop=10, pause=2): print(j) |
Output:
Let’s perform a google search manually and verify our result
Example 2: google_search.py
Python
try: from googlesearch import searchexcept ImportError: print("No module named 'google' found")# to searchquery = "A computer science portal"for j in search(query, tld="co.in", num=10, stop=10, pause=2): print(j) |
Output:
Let’s perform a google search manually and verify our result
Reference: Google python package
This article is contributed by Pratik Chhajer. If you like Lazyroar and would like to contribute, you can also write an article using write.geeksforgeeks.org or mail your article to review-team@geeksforgeeks.org. See your article appearing on the Lazyroar main page and help other Geeks.
Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above.
