fabric.js updating line coordinates

Question

I have set up an arrow with fabric.js using a line and a triangle rotated 90 degrees. I have set these up in a Group so that they will move together.

When I double click on the arrow, I have a circle being placed at each end of the arrow's line. When I move these circles I want the corresponding end of the arrow to be moved to wherever my pointer moves while the other end stays in place.

Currently, when I move one of the circles, the line of the arrow is shifted away from it's current position and then as I move the circle the line updates it's position from the place it gets shifted to. I'm guessing this shift is happening because I'm missing something with the group.

Here is my function that is handling when a circle is being moved:

handleDrag: function(obj) {
    const pointer = obj.pointer
    const target = obj.transform.target
    const line = target.line
    line.set({
        x1: line.x1,
        y1: line.y1,
        x2: pointer.x,
        y2: pointer.y,
    })
    line.setCoords()
},

The target would be the circle that is being moved and when I add the circle I give it the line that it is supposed to move so that I can use target.line to get the line and update its coordinates.

Here is a codepen so you can see what is happening now:
https://codepen.io/jrabes/pen/ExoJZgJ