I am working on a research publication and collaboration project that has a literature search feature in it. Google Scholar seems like it will work since it is an open source tool but, when I researched Google Scholar, I could not find any information about it having an API.
Please let me know if there is any API for Google Scholar that is valid.
TIA.
2 Answers
A quick search shows that others are trying to implement such APIs, but Google does not provide one. It is not clear whether this is legal, see for instanceHow to get permission from Google to use Google Scholar Data, if needed?.
There's no official Google Scholar API. There are third-party solutions like free scholarly Python package which supports profile, author, cite results but does not support organic results, or Google Scholar API from SerpApi which is a paid API with a free plan that supports organic, cite, profile, author results and bypasses all the blocks on SerpApi backend.
Example code to parse profile results using scholarly using search_by_keyword method:
import json
from scholarly import scholarly
# will paginate to the next page by default
authors = scholarly.search_keyword("biology")
for author in authors: print(json.dumps(author, indent=2))
# part of the output:
'''
{ "container_type": "Author", "filled": [], "source": "SEARCH_AUTHOR_SNIPPETS", "scholar_id": "LXVfPc8AAAAJ", "url_picture": "", "name": "Eric Lander", "affiliation": "Broad Institute", "email_domain": "", "interests": [ "Biology", "Genomics", "Genetics", "Bioinformatics", "Mathematics" ], "citedby": 552013
}
... other author results
'''Example code to parse organic results using Google Scholar Profile Results API from SerpApi:
import json
from serpapi import GoogleScholarSearch
# search parameters
params = { "api_key": "Your SerpApi API key", "engine": "google_scholar_profiles", "hl": "en", # language "mauthors": "biology" # search query
}
search = GoogleScholarSearch(params)
results = search.get_dict()
# only first page results
for result in results["profiles"]: print(json.dumps(result, indent=2))
# part of the output:
'''
{ "name": "Masatoshi Nei", "link": "", "serpapi_link": "", "author_id": "VxOmZDgAAAAJ", "affiliations": "Laura Carnell Professor of Biology, Temple University", "email": "Verified email at temple.edu", "cited_by": 384074, "interests": [ { "title": "Evolution", "serpapi_link": "", "link": "" }, { "title": "Evolutionary biology", "serpapi_link": "", "link": "" }, { "title": "Molecular evolution", "serpapi_link": "", "link": "" }, { "title": "Population genetics", "serpapi_link": "", "link": "" }, { "title": "Phylogenetics", "serpapi_link": "", "link": "" } ], "thumbnail": ""
}
... other results
'''There is a dedicated Scrape historic Google Scholar results using Python blog post of mine at SerpApi which shows how to scrape historic 2017-2021 Organic, Cite Google Scholar results to CSV, SQLite.
Disclaimer, I work for SeprApi
