How to store multiple python dictionaries or merge in a loop and return?

Question

I am trying to get the pagination results of two pages but return is exiting loop and displays only one result from a page.

Is there a way to store or merge them?

  def incidents():
        m = True
        limit = 50
        offset = 0

        while m == True:
            url = f"{URL}/incidents"
            params = {
                "statuses[]": "resolved",
                "include[]" : 'channel',
                "limit"     : limit,
                "offset"    : offset,
                "total"     : "true",
            }
            r = requests.get(url, headers=headers, params=params)
            data = r.json()

            offset += 50
            print(offset, (r.text))
            more = False # Set deliberately for easier understanding

        return data

The offset, (r.text) output looks like -

50 {"incidents":[{"incident_number":1,"title":"crit server is on fire" ....


100 {"incidents":[{"incident_number":54,"title":"ghdg","description":"ghdg",....

Return only displays below and not the other one. There should be a way like use a generator for example? So we can merge them both and store in data variable so data can be returned?

100 {"incidents":[{"incident_number":54,....

Answer 1

I believe you could store the results in your own list:

incidents = []

and then

data = r.json()
for element in data['incidents']:
    incidents.append(element)

Edited for clarity - that way you're gathering all incidents in a single object.

Answer 2

I'm not sure because you just gave the very start of r.text (is there more than 'incidents' within the result?), but i expect the previous answer to be a bit short; i'd suggest something like

results = []

(before the while) and at the end

    data = r.json()
    results += data['incidents']

return results

(btw: in your original post, each run through while just set the var "data", so no wonder the return can only deal with the very last part retrieved. But i guess that is just an artifact of you simplification, like the "more=False" would even prevent getting a second page)