Extracting data from Pinterest using gRPC, Python

Question

I want to grab all chats of my Pinterest account
I have a Proto Service:

syntax = "proto3";


service Pinterest{
    rpc GetConversations (request_chat_info) returns (chat_response);
}



message request_chat_info{
    string conversation_id = 1;
    string csrftoken = 2;
    string _b = 3;
    string _pinterest_sess = 4;
}


message chat_response{
    string type = 1;
    string id = 2;
    string text = 3;
}


message chat_response_array{
    repeated chat_response messages = 1;
}

---

and this is my Pinterest Servicer:

# GRPC Service
class PinterestService(pb2_grpc.PinterestServicer):
    def GetConversations(self, request, context):
        conversation_id = request.conversation_id
        csrftoken = request.csrftoken
        _b = request._b
        _pinterest_sess = request._pinterest_sess
        chats = _exec(
            get_chat,
            {"conversation_id": conversation_id, "csrftoken": csrftoken, "_b": _b, "_pinterest_sess": _pinterest_sess}
        )
        
        return pb2.chat_response_array(messages=chats)

---

and main Program is something like this:

# ENDPOINTS
CHAT_API = "https://www.pinterest.com/resource/ConversationMessagesResource/get/"




# Execute Fucntion
def _exec(func, params):
    return func(**params)



# Make Requests here
def _get(url:str, cookies:Dict = None, headers:Dict = None) -> requests.Response:
    response = requests.request("GET", url=url, cookies=cookies, headers=headers)
    response.raise_for_status()
    return response


# Chat Parser Function
def _chat_parser(chat_dict: Dict) -> Dict:
    return {
        "type": chat_dict.get("type", ""),
        "id": chat_dict.get("id", ""),
        "text": chat_dict.get("text", ""),
    }


# Function to handle GRPC
def get_chat(conversation_id:str, csrftoken:str, _b:str, _pinterest_sess:str) -> Dict:
    options = {"page_size":25,"conversation_id":conversation_id,"no_fetch_context_on_resource":False}
    _cookies = {"csrftoken":csrftoken, "_b":_b, "_pinterest_sess":_pinterest_sess}
    query = {"data": json.dumps({"options":options})}
    encoded_query = urlencode(query).replace("+", "%20")
    url = "{}?{}".format(CHAT_API, encoded_query)
    msg_counter = 0
    while True:
        try:
            return _chat_parser(_get(url, _cookies).json()["resource_response"]["data"][msg_counter])
        except IndexError:
            break
        finally:
            msg_counter += 1

---

I need to get all CHAT and I Don't know how to do that!
The Response JSON in Pinterest is exactly Like this:

["resource_response"]["data"][0]
["resource_response"]["data"][1]
["resource_response"]["data"][2]
["resource_response"]["data"][...]

Based on Messages Count the last number change from 0 to any number
I don't know how to handle that!
does the fault in my proto or what?
should I use stream in proto, if yes, bidirectional stream or for Client, Server...
Thank you for Helping me.

Answer 1

I found the answer
it must be server stream and use for loop and yield