curl_exec returns unreadable text - cURL PHP

Question

I am trying cURL, but curl_exec() returns unreadable text like the screenshot below. I wrote cURL like below. I was wondering how to fix this issue.

$ch = curl_init("https://app.kajabi.com/login");
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
    'Host: app.kajabi.com',
    'Connection: keep-alive',
    'Cache-Control: max-age=0',
    'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="99", "Google Chrome";v="99"',
    'sec-ch-ua-mobile: ?0',
    'sec-ch-ua-platform: "Windows"',
    'Upgrade-Insecure-Requests: 1',
    'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Safari/537.36',
    'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
    'Sec-Fetch-Site: none',
    'Sec-Fetch-Mode: navigate',
    'Sec-Fetch-User: ?1',
    'Sec-Fetch-Dest: document',
    'Accept-Encoding: gzip, deflate, br',
    'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8',
    'Cookie: _kjb_session=795006a5538f30410ce2f56bd813ddb0; __cf_bm=7iLyh_LWPmJjzo07YdEJQaE_RT0LPS2R6NL1Hp3Li6g-1649142817-0-Ae4i2Gq5QTr+PktvLBJEV8MHcgGTw5ADVHkedUa3JTcVLHEDTyE01Nw6qsZtmjs7Quu+phKNOlCtu/8Cxpdwxec=; __cfruid=531ca052551b47923660c7b1832af0f2ea867981-1649142817; _kjb_ua_components=41e11a8e3c73294e1d2e0f1813e1f86d'
));
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);

if (curl_errno($ch)) {
    print "Error: " . curl_error($ch);
    exit();
}
echo $response;

Answer 1

I tried putting the response into a file, and it appears that the response is in gzip format. file_put_contents('temp.gz',$response)

I extracted the archive and found that it's a HTML document telling you that the access is denied.

You can show the response directly in the output of your php script, though:

$decoded_response = gzdecode($response);
echo $decoded_response;

And maybe you should check whether the content is actually gzip before attempting to use gzdecode; see this thread: php curl, detect response is gzip or not

---

Edit: You can let php automatically do the decoding by setting CURLOPT_ENCODING to '':

<?php
$ch = curl_init("https://app.kajabi.com/login");
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
    'Host: app.kajabi.com',
    'Connection: keep-alive',
    'Cache-Control: max-age=0',
    'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="99", "Google Chrome";v="99"',
    'sec-ch-ua-mobile: ?0',
    'sec-ch-ua-platform: "Windows"',
    'Upgrade-Insecure-Requests: 1',
    'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Safari/537.36',
    'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
    'Sec-Fetch-Site: none',
    'Sec-Fetch-Mode: navigate',
    'Sec-Fetch-User: ?1',
    'Sec-Fetch-Dest: document',
    'Accept-Encoding: gzip, deflate, br',
    'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8',
    'Cookie: _kjb_session=795006a5538f30410ce2f56bd813ddb0; __cf_bm=7iLyh_LWPmJjzo07YdEJQaE_RT0LPS2R6NL1Hp3Li6g-1649142817-0-Ae4i2Gq5QTr+PktvLBJEV8MHcgGTw5ADVHkedUa3JTcVLHEDTyE01Nw6qsZtmjs7Quu+phKNOlCtu/8Cxpdwxec=; __cfruid=531ca052551b47923660c7b1832af0f2ea867981-1649142817; _kjb_ua_components=41e11a8e3c73294e1d2e0f1813e1f86d'
));
curl_setopt($ch, CURLOPT_ENCODING, '');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);

if (curl_errno($ch)) {
    print "Error: " . curl_error($ch);
    exit();
}
echo $response;
?>

Answer 2

You are getting un-handled GZIP from Curl because you are manually setting the Accept-Encoding: header in your header array, rather than letting Curl handle it. Curl then gets an unexpectedly-encoded response and goes "I dunno, you deal with this".

You're telling the remote side "I want things handled this way" but you're not actually telling the local side.

Easy fix: Remove the Accept-Encoding: header from your header array, optionally move those encoding specifications to the CURLOPT_ENCODING setting you added in your own answer, but I would say that this is unnecessary as curl will prefer compression anyway.

Other headers that you should likely not be manually setting:

• Host: unnecessary unless you need a value other than the hostname in the URL
• Connection: client needs to be aware
• Upgrade-Insecure-Requests: client needs to be aware, browser-specific