How to prime a zlib block for appending on compressed data?

Question

Here i have a test which takes in a zlib stream which is stored in the array called __long_zlib and the plain text stored in plain_text. The following code primes __long_zlib which allows the plain text compressed data to be appended on. However, this does not work as the output is corrupted. I believe the issue is with the bits that are being primed as the first couple of bytes are corrupted when decompressed.

Expected output when decompressing raw_stream.bin:

this file needs appending...
ayoo i have been appended with a newline

Actual output:

this file needs appending...
t:7�xx�e been appended with a newline

#include <stdio.h>
#include <zlib.h>
#include <string.h>
// zlib-flate -compress=9 < ./long > long.zlib
unsigned char __long_zlib[] = {
  0x78, 0xda, 0x2b, 0xc9, 0xc8, 0x2c, 0x56, 0x48, 0xcb, 0xcc, 0x49, 0x55,
  0xc8, 0x4b, 0x4d, 0x4d, 0x29, 0x56, 0x48, 0x2c, 0x28, 0x48, 0xcd, 0x4b,
  0xc9, 0xcc, 0x4b, 0xd7, 0xd3, 0xd3, 0xe3, 0x02, 0x00, 0xa1, 0x53, 0x0a,
  0x12
};
const char plain_text[] = "ayoo i have been appended with a newline\n";
 
int main()
{
    z_stream strm;
    unsigned char buf[4096] = {0};
    // copy the zlib file into the buffer and chop off the adler32
    int zlib_len = sizeof(__long_zlib) -  4;
    memcpy(&buf, __long_zlib, zlib_len);
    strm.zalloc = Z_NULL;
    strm.zfree = Z_NULL;
    strm.opaque = Z_NULL;
    if (deflateInit2(&strm, Z_BEST_COMPRESSION, Z_DEFLATED, -15, 8, Z_DEFAULT_STRATEGY) == Z_OK)
    {
        int first = 2;                   // zlib header is two bytes so skip that
        int back = 3 + (buf[first] & 7); // number of bits that the block start is back from the first length in a stored block
        deflatePrime(&strm, (8 - back) & 7, buf[zlib_len -1 -  (back > 8 ? 2 : 1)]); //prime the last bits from the previous stream?
        strm.next_in = &plain_text;
        strm.avail_in = strlen(plain_text);
        strm.next_out = (Bytef *)&buf[zlib_len  - (back > 8 ? 2 : 1) -1  ];
        strm.avail_out = sizeof(buf) - zlib_len;
        deflate(&strm, Z_FINISH);
        FILE *f = fopen("raw_stream.bin", "wb");
        fwrite(&buf, 1, sizeof(buf) - strm.avail_out , f);
        fclose(f);
        deflateEnd(&strm);
    }
 
}

Answer 1

You're just making stuff up. You need to first understand exactly what's going on with the deflate append code that you are attempting to adapt.

Your code, or really the code you're trying to copy, is assuming that there is an empty stored block at the end of the deflate stream that is being appended to. There is no such stored block in your zlib stream.

Answer 2

Db2 uses cost-based optimizer.
It probably decides, that TBSCANs are cheaper for certain queries.
You may try to get desired access plan with the corresponding hint (it's called optimization guidelines in Db2) like below:

select * from mytable t1 
left join mytable t2 on t1.id = t2.id
/*
<OPTGUIDELINES>
...
</OPTGUIDELINES>
*/
;

where optimization guidelines may look like:

(let Db2 choose some JOIN request like MSJOIN, NLJOIN, etc. using IXSCAN with particular index name):

<OPTGUIDELINES>
  <JOIN>
    <IXSCAN TABLE="T1" INDEX="UNQUALIFIED_PK_INDEX_NAME"/>
    <IXSCAN TABLE="T2" INDEX="UNQUALIFIED_PK_INDEX_NAME"/>
  </JOIN>
</OPTGUIDELINES>

(NLJOIN with IXSCANs using whatever appropriate indexes):

<OPTGUIDELINES>
  <NLJOIN>
    <IXSCAN TABLE="T1"/>
    <IXSCAN TABLE="T2"/>
  </NLJOIN>
</OPTGUIDELINES>

Or use whatever else acceptable guidelines to get the access plan desired.
Refer to the Optimization profiles and guidelines topic for more details.

Compare the total cost of each access plan to understand, why Db2 prefers some particular access plan (likely with the smallest total cost).