Android - How to get canvas object from my view?

Question

My app uses a tensorflow lite model to detect objects from an uploaded image .

Therefore I need to draw the bounding boxes of the object on the image that the user has uploaded.

This is my activity_camera.xml where an image is uploaded into.

<ImageView
        android:id="@+id/imageView"
        android:layout_width="281dp"
        android:layout_height="324dp"
        android:layout_marginTop="30dp"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        app:srcCompat="@android:drawable/ic_menu_gallery"
        tools:ignore="ImageContrastCheck" />

Here is where I am running the tflite model. (CameraActivity.java)

predictBtn.setOnClickListener(new View.OnClickListener() {

            @Override
            public void onClick(View v) {

                img = Bitmap.createScaledBitmap(img, 128, 128, true);

                try {

                    Android model = Android.newInstance(getApplicationContext());

                    // Creates inputs for reference.
                    TensorImage image = TensorImage.fromBitmap(img);


                    // Runs model inference and gets result.
                    Android.Outputs outputs = model.process(image);
                    Android.DetectionResult detectionResult = outputs.getDetectionResultList().get(0);

                    // Gets result from DetectionResult.
                    float score = detectionResult.getScoreAsFloat();
                    RectF location = detectionResult.getLocationAsRectF();
                    String category = detectionResult.getCategoryAsString();

                    drawDebug(canvas,location);

                    // Releases model resources if no longer used.
                    model.close();
                    // here we will print out the results of the object to text views based on the image that is inputted by the user
                    // we print out object type and its accuracy score

                    objecttv.setText(category);
                    scoretv.setText(Float.toString(score));


                } catch (IOException e) {
                    // TODO Handle the exception
                }

            }

                                          public void drawDebug(final Canvas canvas, RectF location) {

                                              final Paint boxPaint = new Paint();
                                              boxPaint.setColor(Color.RED);
                                              boxPaint.setAlpha(200);
                                              boxPaint.setStyle(Paint.Style.STROKE);

                                              canvas.drawRect(location, boxPaint);
                                          }
        }
        );

The drawing of the bounding boxes does not work as I need to get the canvas first to draw on.

What do I need to modify in this or create to get my canvas object from the view?

Answer 1

Views don't have Canvas objects. When a view is drawn in onDraw, it is passed a Canvas object to use. That is the only time that a Canvas exists for a View, and that is when all drawing must occur.

Instead, what you should do where you want to draw is invalidate() the View. This will tell the View it needs to redraw, which will call onDraw at some point in the near future. Then add your drawing code to onDraw. If necessary, store any state that you need to draw in member variables of the View so it has access to them at draw time.

Since you're using an ImageView, what you'd need to do is subclass ImageView. Add that new subclass to the xml. Then override the subclass's onDraw function to call super.onDraw(), then do any additional drawing you wish.

Answer 2

Answer modified after question changed.
You can draw the bounding boxes with Canvas. Override onDraw() in the view where you want to draw the rectangle. Attention, this codes only works for one bounding box. Otherwise you have to implement a for loop. You should not call the canvas onDraw function directly from the activity. Here is a complete example:

Your tracker class:

// Init your canvas view
DrawView mCanvasView = (DrawView) findViewById(R.id.canvasView);

// ...

predictBtn.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            img = Bitmap.createScaledBitmap(img, 128, 128, true);
            try {
                Android model = Android.newInstance(getApplicationContext());
                // Creates inputs for reference.
                TensorImage image = TensorImage.fromBitmap(img);
                // Runs model inference and gets result.
                Android.Outputs outputs = model.process(image);
                Android.DetectionResult detectionResult = outputs.getDetectionResultList().get(0);
                // Gets result from DetectionResult.
                float score = detectionResult.getScoreAsFloat();
                RectF location = detectionResult.getLocationAsRectF();
                String category = detectionResult.getCategoryAsString();

                mCanvasView.drawBoundingBox()                 

                // Releases model resources if no longer used.                                   
                model.close();
                
                // here we will print out the results of the object to text views based on the image that is inputted by the user
                // we print out object type and its accuracy score
                objecttv.setText(category);
                scoretv.setText(Float.toString(score));
            } catch (IOException e) {
                // TODO Handle the exception
            }
        }
});

Implement this canvas view:

import android.content.Context;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Paint;
import android.view.View;

public class DrawView extends View {
   Paint boxPaint;
   public static Canvas mCanvas;

    public DrawView(Context context) {
        super(context);
        boxPaint = new Paint();            
    }

    public void drawBoundingBox() {
        // Refresh the view by calling onDraw function
        invalidate();
    }

    @Override
    public void onDraw(Canvas canvas, AttributeSet attrs) {
         // Draw what you want
         boxPaint.setColor(Color.RED);
         boxPaint.setAlpha(200);
         boxPaint.setStyle(Paint.Style.STROKE);
         canvas.drawRect(location, boxPaint);
    }
}

Define your Draw view in xml of your tracker class

<your.package.name.DrawView
    android:id="@+id/R.id.canvasView"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />