'Cannot read integer values saved into binary file with Java

I implemented a procedure for saving 32bit signed integer values, retrieved from PostgreSQL, into a binary file.

I used ByteArrayOutputStream and DataOutputStream

//..


ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream out = new DataOutputStream(baos);

//this cycle over my retrieved recordset
while (tdb.next()) {

   out.writeInt((tdb.getInteger(1)));  //getInteger method extract integer value from recordset

}

//convert into byte array
byte[] bytes = baos.toByteArray();

//create file for output
File fileout = new File("./myfilebin.bin");
//write data
Files.write(fileout.toPath(), bytes);

//...

My input data sample contains these values:

0, 0, 42812, 42822, 41483, 0, 0, ...

When I try to read my binary file, I will get these:

0, 0, 1017577472, 1185349632, 195166208, 0, 0, ...

For reading file I wrote a simple python script:

import struct

with open("./myfilebin.bin", "rb") as f:

     for chunk in iter(lambda: f.read(4), ''):
        integer_value = struct.unpack('i', chunk)[0]
        print(integer_value)

Any idea?

pythonjavagoogletestsuppress-warningsaddress-sanitizersanitizer


Solution 1:[1]

Probably your numbers are Big-Endian (BE) and you try to decode as Little-Endian (LE).

Try: struct.unpack('>i', chunk) to force reading as BE.

See also the docs for struct module on alignment and endianess.

FYI, DataOutputStream always writes in BE as stated in the official javadoc.

Solution 2:[2]

Ok, the problem was the different behaviour in bit reading order between python, that uses LE, and java.

I need to create a binary file that follows ISOBUS standards and I have to write it using LE.

So I change my code in this way:

FileOutputStream fout = new FileOutputStream("myfileout.bin");

            DataOutputStream out = new DataOutputStream(fout);


            while (tdb.next()) {
                //convert byte endian behaviour for ISOBUS software compatibility
                ByteBuffer bb = ByteBuffer.allocate(4);
                bb.order(ByteOrder.LITTLE_ENDIAN);
                bb.putInt(tdb.getInteger(1));
                //invert significant byte
                bb.flip();

                out.write(bb.array());
            }

            fout.close();
            out.close();

Solution 3:[3]

I am getting reports with info about stack-buffer-overflow, which I want to supress.

If you are going to just ignore errors, one solution is to not to test with address sanitizer at all.

I want to not use address sanitizer in this test.

If you want to suppress sanitizer reports for this particular test case, while keeping the sanitizer for other test cases within this source file, using disable_sanitizer_instrumentation attribute might do the trick.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1
Solution 2 N3tMaster
Solution 3 Employed Russian

Related Questions