Google Analytics Data api (beta) returning incomplete data sometimes

Question

I'm working with data from Google analytics GA4 in python, using RunReportRequest call. The issue I'm facing is that sometimes the script returns complete data with all values, but sometimes dimensions come null, mostly custom dimensions. BTW in all API calls I still had available quota.

I've noticed this behavior even scheduling the script execution multiple times a day, and haven't been able to pinpoint what could possible be the cause since the script conditions are pretty much the same.

The script I'm using is similar to this one:

from google.analytics.data_v1beta import BetaAnalyticsDataClient, DateRange, RunReportRequest, Dimension, Metric
import os

client = BetaAnalyticsDataClient()
property_id = os.environ.get('PROPERTY_ID')


def download_ga_data(start_date, end_date, limit=int(1e6)):
    request = RunReportRequest(property=f"properties/{property_id}",
                               dimensions=[Dimension(name=x) for x in ['customEvent:myCustomEvent', 'eventName']],
                               metrics=[Metric(name="averageSessionDuration")],
                               date_ranges=[DateRange(start_date=start_date, end_date=end_date)],
                               limit=limit,
                               return_property_quota=True,
                               keep_empty_rows=False)
    response = client.run_report(request)
    return response

Thanks in advance