Errno 10054: An existing connection was forcibly closed by the remote host - Tweepy Error

Question

I am trying to retrieve information on followers from a Twitter account with almost 600K followers. I was able to use this code to retrieve around 280,000 followers on another account, with no issues. But for the bigger account I get this error:

[Errno 10054] An existing connection was forcibly closed by the remote host

I don't have two open connections, that I am sure of. And I am using the wait on limit parameter.

1. Do you know what I need to do to get the full 600K or so followers?

2. Is there a way to edit the code so that it writes the retrieved followers to a CSV in the event of that error? Or can I write the retrieved data to the CSV for each loop. Right now, once the code errors out, I lose everything it retrieved, and I know it retrieves hundreds of thousands of followers before it errors out. It would be helpful if I can retain this data.

The code I am working with was retrieved from [Retrieve Twitter Followers - Brianna Herold][1]:

import pandas as pd

# Twitter API credentials
consumer_key = '***'
consumer_secret = '***'
access_token = '***'
access_secret = '***'

auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
api = API(auth, wait_on_rate_limit=True, wait_on_rate_limit_notify=True)

screen_name = '***'
ids = []
for fid in Cursor(api.followers_ids, screen_name=screen_name, count=5000).items():
    ids.append(fid)

info = []
for i in range(0, len(ids), 100):
    try:
        chunk = ids[i:i+100]
        info.extend(api.lookup_users(user_ids=chunk))
    except:
        import traceback
        traceback.print_exc()
        print('Something went wrong, skipping...')

data = [x._json for x in info]
df = pd.DataFrame(data)
df = df[['id', 'name', 'screen_name', 'location', 'description', 'url', 'followers_count', 'friends_count', 'created_at', 'verified']]
df.to_csv(r' myfilepath)


  [1]: https://towardsdatascience.com/how-to-download-twitter-friends-or-followers-for-free-b9d5ac23812

Answer 1

I had the same issue.

Twitter seems to have restricted their TLS 1.2 cipher suites. The only available ones are now:

TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f) ECDH x25519 (eq. 3072 bits RSA) FS128TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030) ECDH x25519 (eq. 3072 bits RSA) FS256TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8) ECDH x25519 (eq. 3072 bits RSA) FS

Which means your server that runs the application needs these available and installed on it.