'How do I speed up my 130 second youtube-api python example? [duplicate]

This python snippet:

from datetime import datetime
from googleapiclient.discovery import build
import googleapiclient
import socket

with open("secret_youtube_api_key", "r") as api_key_file:
  api_key = api_key_file.read()

socket.setdefaulttimeout(30000)

youtube : googleapiclient.discovery.Resource = build('youtube', 'v3', developerKey=api_key)
print("youtube type " + str(type(youtube)))

request : googleapiclient.http.HttpRequest = youtube.channels().list(
  part='statistics',
  forUsername='PewDiePie'
)
print("request type " + str(type(request)))

print("\n" + str(datetime.now()) + " request.execute() start")
response = request.execute()
print("response type " + str(type(response)))
print(str(datetime.now()) + "\n request.execute() end")

print(response)

gives the following output:

youtube type <class 'googleapiclient.discovery.Resource'>
request type <class 'googleapiclient.http.HttpRequest'>

2022-04-20 03:22:35.541216 request.execute() start
response type <class 'dict'>
2022-04-20 03:24:46.061762
 request.execute() end
{'kind': 'youtube#channelListResponse', 'etag': '01QNwqKcy80m97clcBBjfozMTV0', 'pageInfo': {'totalResults': 1, 'resultsPerPage': 5}, 'items': [{'kind': 'youtube#channel', 'etag': 'O5elmeewmtTIKBg8LuPaid7kDFs', 'id': 'UC-lHJZR3Gqxm24_Vd_AJ5Yw', 'statistics': {'viewCount': '28262284610', 'subscriberCount': '111000000', 'hiddenSubscriberCount': False, 'videoCount': '4472'}}]}

The timestamp for the actual call execution are quiite regularly 130 seconds apart. My control command using wget gives a hint

wget --timeout=300000 "https://www.googleapis.com/youtube/v3/channels?key=$APP_KEY&forUsername=PewDiePie&part=statistics" 
--2022-04-20 03:24:44--  https://www.googleapis.com/youtube/v3/channels?key=REDACTED&forUsername=PewDiePie&part=statistics
Resolving www.googleapis.com (www.googleapis.com)... 2a00:1450:4001:810::200a, 2a00:1450:4001:811::200a, 2a00:1450:4001:80e::200a, ...
Connecting to www.googleapis.com (www.googleapis.com)|2a00:1450:4001:810::200a|:443... failed: Connection timed out.
Connecting to www.googleapis.com (www.googleapis.com)|2a00:1450:4001:811::200a|:443... failed: Connection timed out.
Connecting to www.googleapis.com (www.googleapis.com)|2a00:1450:4001:80e::200a|:443... failed: Connection timed out.
Connecting to www.googleapis.com (www.googleapis.com)|2a00:1450:4001:80f::200a|:443... failed: Connection timed out.
Connecting to www.googleapis.com (www.googleapis.com)|142.250.185.170|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/json]
Saving to: ‘channels?key=REDACTED&forUsername=PewDiePie&part=statistics’

channels?key=REDACTED     [ <=>                                                                       ]     476  --.-KB/s    in 0s      

2022-04-20 03:33:28 (18,7 MB/s) - ‘channels?key=REDACTED&forUsername=PewDiePie&part=statistics’ saved [476]

This seems to be a common problem without a clear solution. There are related questions (link) that fall short in their answer.

It looks like IPv6 just generally fails. What could be wrong on my end. How would I fix IPV6 issues on my end? Given that my end may be unfixable - how would I need to modify my python code to exclude IPV6?

forcing ipv4 on wget returns instantly: wget -4 --timeout=300000 "https://www.googleapis.com/youtube/v3/channels?key=$APP_KEY&forUsername=PewDiePie&part=statistics"



Solution 1:[1]

This answer to a differently worded question fits here:

It answers how to force ipv4 - not how to fix ipv6.

from datetime import datetime
from googleapiclient.discovery import build
import googleapiclient
import socket

with open("secret_youtube_api_key", "r") as api_key_file:
  api_key = api_key_file.read()


#https://stackoverflow.com/a/50044152/536874
import socket
getaddrinfo_original = socket.getaddrinfo
def getaddrinfo_decorated(*args, **kwargs):
  # filter responses that are not ipv4
  return [response for response in getaddrinfo_original(*args, **kwargs) if response[0] == socket.AF_INET]
socket.getaddrinfo = getaddrinfo_decorated

youtube : googleapiclient.discovery.Resource = build('youtube', 'v3', developerKey=api_key)
print("youtube type " + str(type(youtube)))

request : googleapiclient.http.HttpRequest = youtube.channels().list(
  part='statistics',
  forUsername='PewDiePie'
)
print("request type " + str(type(request)))

print("\n" + str(datetime.now()) + " request.execute() start")
response = request.execute()
print("response type " + str(type(response)))
print(str(datetime.now()) + "\n request.execute() end")

print(response)

The updated code now outputs:

youtube type <class 'googleapiclient.discovery.Resource'>
request type <class 'googleapiclient.http.HttpRequest'>

2022-04-20 14:06:27.311410 request.execute() start
response type <class 'dict'>
2022-04-20 14:06:27.503453
 request.execute() end
{'kind': 'youtube#channelListResponse', 'etag': '_EY8e-LjjUX00BenLPIGIts8iS0', 'pageInfo': {'totalResults': 1, 'resultsPerPage': 5}, 'items': [{'kind': 'youtube#channel', 'etag': 'QmXjB1Hz-b5QX1ahMELzX1BTl6I', 'id': 'UC-lHJZR3Gqxm24_Vd_AJ5Yw', 'statistics': {'viewCount': '28263689871', 'subscriberCount': '111000000', 'hiddenSubscriberCount': False, 'videoCount': '4472'}}]}

Which is a 200ms call like it should be. A better answer would include how to find and fix whatever is wrong with ipv6 on my network.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Johannes