How can we fetch google scholar citation through SERPAPI? - Jquery

RoshniTiwary
August 29, 2022
104 views
0 votes
2 Answers

I have a number of authors name list and their e-mail id and i want to fetch their citations programmatically . So how could i fetch their citations using SERPAPI ?

Answers

If I understand right, you already have the author’s IDs, if it’s true you need to do some loop to get citations from each author, here’s how you can do it (also check code example in the online IDE):

const SerpApi = require("google-search-results-nodejs");
const search = new SerpApi.GoogleSearch(process.env.API_KEY); //your api key from serpapi.com

const params = {
  engine: "google_scholar_author", // search engine
  hl: "en", //search languge
};

const authorsIds = ["EicYvbwAAAAJ", "hqAf_AgAAAAJ", "5M-bz3EAAAAJ"];

const getJson = async () => {
  return new Promise((resolve) => {
    search.json(params, resolve);
  });
};

const getCitations = async () => {
  const citations = [];
  for (id of authorsIds) {
    params.author_id = id; // set author id to request params
    const json = await getJson();
    citations.push({ author: json.author, citations: json.articles });
  }
  return citations;
};

getCitations().then((citations) => {
  console.dir(citations, { depth: null });
  // or do what you want to do next
});

Output:

[
   {
      "author":{
         "name":"George Patton",
         "affiliations":"University of Melbourne; Murdoch Childrens Research Institute.",
         "email":"Verified email at unimelb.edu.au",
         "interests":[
            {
               "title":"Mental Health",
               "link":"https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:mental_health",
               "serpapi_link":"https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Amental_health"
            },
            {
               "title":"Adolescent Health",
               "link":"https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:adolescent_health",
               "serpapi_link":"https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aadolescent_health"
            },
            {
               "title":"Psychiatric epidemiology",
               "link":"https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:psychiatric_epidemiology",
               "serpapi_link":"https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Apsychiatric_epidemiology"
            },
            {
               "title":"Psychiatry",
               "link":"https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:psychiatry",
               "serpapi_link":"https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Apsychiatry"
            }
         ],
         "thumbnail":"https://scholar.googleusercontent.com/citations?view_op=view_photo&user=hqAf_AgAAAAJ&citpid=3"
      },
      "citations":[
         {
            "title":"Global, regional, and national comparative risk assessment of 79 behavioural, environmental and occupational, and metabolic risks or clusters of risks, 1990–2015: a systematic …",
            "link":"https://scholar.google.com/citations?view_op=view_citation&hl=en&user=hqAf_AgAAAAJ&citation_for_view=hqAf_AgAAAAJ:EsEWqaRxkBgC",
            "citation_id":"hqAf_AgAAAAJ:EsEWqaRxkBgC",
            "authors":"MH Forouzanfar, A Afshin, LT Alexander, HR Anderson, ZA Bhutta, ...",
            "publication":"The lancet 388 (10053), 1659-1724, 2016",
            "cited_by":{
               "value":8808,
               "link":"https://scholar.google.com/scholar?oi=bibs&hl=en&cites=2908528395435776161,3055444908891106187,4467761493745102427,12996769275393345791,10219250234207102035,3883213508573811465,10824011199718854451,9798408928041897835",
               "serpapi_link":"https://serpapi.com/search.json?cites=2908528395435776161%2C3055444908891106187%2C4467761493745102427%2C12996769275393345791%2C10219250234207102035%2C3883213508573811465%2C10824011199718854451%2C9798408928041897835&engine=google_scholar&hl=en",
               "cites_id":"2908528395435776161,3055444908891106187,4467761493745102427,12996769275393345791,10219250234207102035,3883213508573811465,10824011199718854451,9798408928041897835"
            },
            "year":"2016"
         },
        ... and other citations
      ]
   },
   {
      "author":{
         "name":"Ann Kristin Knudsen",
         "affiliations":"Departement Director Norwegian Institute of Public Health",
         "email":"Verified email at fhi.no",
         "interests":[
            {
               "title":"burden of disease",
               "link":"https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:burden_of_disease",
               "serpapi_link":"https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aburden_of_disease"
            },
            {
               "title":"psychiatric epidemiology",
               "link":"https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:psychiatric_epidemiology",
               "serpapi_link":"https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Apsychiatric_epidemiology"
            },
            {
               "title":"sickness absence research",
               "link":"https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:sickness_absence_research",
               "serpapi_link":"https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Asickness_absence_research"
            },
            {
               "title":"parental alcohol use",
               "link":"https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:parental_alcohol_use",
               "serpapi_link":"https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aparental_alcohol_use"
            },
            {
               "title":"problem drinking",
               "link":"https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:problem_drinking",
               "serpapi_link":"https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aproblem_drinking"
            }
         ],
         "thumbnail":"https://scholar.googleusercontent.com/citations?view_op=view_photo&user=5M-bz3EAAAAJ&citpid=1"
      },
      "citations":[
         {
            "title":"Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990–2013: a systematic …",
            "link":"https://scholar.google.com/citations?view_op=view_citation&hl=en&user=5M-bz3EAAAAJ&citation_for_view=5M-bz3EAAAAJ:hFOr9nPyWt4C",
            "citation_id":"5M-bz3EAAAAJ:hFOr9nPyWt4C",
            "authors":"T Vos, RM Barber, B Bell, A Bertozzi-Villa, S Biryukov, I Bolliger, ...",
            "publication":"The lancet 386 (9995), 743-800, 2015",
            "cited_by":{
               "value":13443,
               "link":"https://scholar.google.com/scholar?oi=bibs&hl=en&cites=1961466636690701750,16526736009745336210,14070611524888519341",
               "serpapi_link":"https://serpapi.com/search.json?cites=1961466636690701750%2C16526736009745336210%2C14070611524888519341&engine=google_scholar&hl=en",
               "cites_id":"1961466636690701750,16526736009745336210,14070611524888519341"
            },
            "year":"2015"
         },
         ... and other citations
      ]
   },
   ... and othe authors
]

There’s a web scraping Google Scholar Author info blog post if you want to understand this topic deeper.

You could use my library, pyserpZotero. It not only scrapes and parses the results, it also offers to export them as Bibtex, upload to Zotero, clean any raw LaTex, and download the free PDFs of any of your results that are also on arXiv.org

#### Build a list of search terms:
TERMS = ['reinforcement learning', 'traveling salesman', 'nowcasting', 'propensity score']

MIN_YEAR = "2010" # Oldest year to search
SAVE_BIB = False  # Save a Bibtex file (.bib)?
USE_ZOT  = True   # Upload to Zotero?
CLEAN    = False  # Attempt to remove/repair broken LaTex and other formatting 


#### Load libraries
from box import Box

import pyserpZotero
import yaml
    
#### Import Credentials from Your YAML File
with open("config.yaml", "r") as ymlfile:
    cfg = Box(yaml.safe_load(ymlfile), default_box=True, default_box_attr=None)

API_KEY = cfg.API_KEY
ZOT_ID  = cfg.ZOT_ID
ZOT_KEY = cfg.ZOT_KEY

#### Instantiate a serpZot object for API management
citeObj = pyserpZotero.serpZot(API_KEY  = API_KEY, 
                               ZOT_ID   = ZOT_ID, 
                               ZOT_KEY  = ZOT_KEY)

#### Call the search method
for i in range(len(TERMS)):
    print(citeObj.searchScholar(TERM     = TERMS[i], 
                                MIN_YEAR = MIN_YEAR,
                                SAVE_BIB = SAVE_BIB))
    print("This should've returned 0 (sucess)")
    # Upload the parsed results
    print(citeObj.search2Zotero())


#### Clean Ugly Raw LaText (as Much as Possible)
if CLEAN:
    citeObj.cleanZot(ZOT_ID      = ZOT_ID, 
                     ZOT_KEY     = ZOT_KEY,
                     SEARCH_TERM = "\") # optional (defaults to all items)

#### Check Arxiv for Free PDFs of Papers and Attach / Upload Them To Zotero
citeObj.arxivDownload()

How can we fetch google scholar citation through SERPAPI? – Jquery

Answers