'Scopus Abstract Retrieval - Value and Type Error only when too many entries are parsed

I am trying to retrieve abstracts via Scopus Abstract Retrieval. I have a file with 3590 EIDs.

import pandas as pd
import numpy as np

file = pd.read_excel(r'C:\Users\Amanda\Desktop\Superset.xlsx', sheet_name='Sheet1')

from pybliometrics.scopus import AbstractRetrieval
for i, row in file.iterrows():
  q = row['EID']
  ab = AbstractRetrieval(q,view='META_ABS')
  file.at[i,"Abstract"] = ab.description
  print(str(i) + ' ' + ab.description)
  print(str(''))

I get a value error - Value Error

In response to the value error, I altered the code.

from pybliometrics.scopus import AbstractRetrieval
error_index_valueerror = {}

    for i, row in file.iterrows():
      q = row['EID']
      try:
        ab = AbstractRetrieval(q,view='META_ABS')
        file.at[i,"Abstract"] = ab.description
        print(str(i) + ' ' + ab.description)
        print(str(''))
      except ValueError:
        print(f"{i} Value Error")
        error_index_valueerror[i] = row['Title']
        continue

When I trialed this code with 10-15 entries, it worked well and I retrieved all the abstracts. However, when I ran the actual file with 3590 EIDs, the output would be a series of 10-12 value errors before a type error ('can only concatenate str (not "NoneType") to str surfaces.

Series of value errors and a type error

I am not sure how to tackle this problem moving forward. Any advice on this matter would be greatly appreciated!

(Side note: When I change view='FULL' (as recommended by the documentation), I still get the same outcome.)



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source