Tensorflow-lite - Getting bitmap from quantized model output

Question

We are working on semantic segmentation application in android using tensorflow-lite.The '.tflite' deeplabv3 model used has input of type (ImageTensor) uint8[1,300,300,3] and ouput of type (SemanticPredictions) uint8[300,300].We were successfully able to run the model and get the ouptut in a ByteBuffer format with the help of tflite.run method.But we were unable to extract an image from this output in java.The model which is trained with pascal voc dataset and was actually converted to tflite format from the TF Model: 'mobilenetv2_dm05_coco_voc_trainval' .

The problem seems to be similar to the following stackoverflow question : tensorflow-lite - using tflite Interpreter to get an image in the output

The same issue which deals with float data-type conversion seems to be fixed in the github issue: https://github.com/tensorflow/tensorflow/issues/23483

So, how can we properly extract the segmentation mask from the UINT8 model output?

Answer 1

Try this code:

    /**
     * Converts ByteBuffer with segmentation mask to the Bitmap
     *
     * @param byteBuffer Output ByteBuffer from Interpreter.run
     * @param imgSizeX Model output image width
     * @param imgSizeY Model output image height
     * @return Mono color Bitmap mask
     */
    private Bitmap convertByteBufferToBitmap(ByteBuffer byteBuffer, int imgSizeX, int imgSizeY){
        byteBuffer.rewind();
        byteBuffer.order(ByteOrder.nativeOrder());
        Bitmap bitmap = Bitmap.createBitmap(imgSizeX , imgSizeY, Bitmap.Config.ARGB_4444);
        int[] pixels = new int[imgSizeX * imgSizeY];
        for (int i = 0; i < imgSizeX * imgSizeY; i++)
            if (byteBuffer.getFloat()>0.5)
                pixels[i]= Color.argb(100, 255, 105, 180);
            else
                pixels[i]=Color.argb(0, 0, 0, 0);

        bitmap.setPixels(pixels, 0, imgSizeX, 0, 0, imgSizeX, imgSizeY);
        return bitmap;
    }

It works for a model with mono colour output.

Answer 2

Something along the lines of:

  Byte[][] output = new Byte[300][300];

    Bitmap bitmap = Bitmap.createBitmap(300,300,Bitmap.Config.ARGB_8888);

    for (int row = 0; row < output.length ; row++) {
        for (int col = 0; col < output[0].length ; col++) {
            int pixelIntensity = output[col][row];
            bitmap.setPixel(col,row,Color.rgb(pixelIntensity,pixelIntensity,pixelIntensity));
        }

?