Scaling custom cursor on hover over a link with GSAP is not working properly using mouse enter and leave functions

Question

I would like to scale up my custom cursor when the user is hover over a link, but this scales up and directly back to normal size.

It has something to do with the scale parameter inside GSAP function gsap.to, but I can't figure out how to fix this. Any suggestions what's going on, since I don't see an error message at all?

Live preview: https://codepen.io/Caspert/pen/dyJbWdy

Code explanation First I store the mouse position in a object.

let mousePos = {x: 0, y: 0};
let cursorSpeed = .25;

On mousemove I update the position of the cursor and transform the custom cursor, so it will follow the default cursor. I am not sure if the window.removeEventListener("mousemove", this); should be called.

document.addEventListener("mousemove", (e) => {
  mousePos.y = math.lerp(mousePos.y, e.clientY, .2);
  mousePos.x = math.lerp(mousePos.x, e.clientX, .2);

  const diff = e.clientX - mousePos.x;
  const acc = diff / window.innerWidth;
  const velo = + acc;
  const scale = 1 - Math.abs(5 * velo);

  gsap.to(cursor, cursorSpeed, {
    x: e.clientX, // update client X
    y: e.clientY, // update client Y
    opacity: 1, // show custom cursor
    scale: scale, // update scale
    ease: "smoothInOut"
  });

  // Execute scale function
  onMouseScale();

  // Clean up function
  window.removeEventListener("mousemove", this);
});

So as mentioned above, I would like to scale up the cursor when a user hovered over a html-link. So I call the function below for each link and transform the scale of the custom cursor to a little larger on mouseenter. On mouseleave I will smoothly scale down the cursor back to normal.

// Add event for hover links
let links = document.querySelectorAll("a");

function onMouseScale(){
  links.forEach((link) => {
    link.addEventListener("mouseenter", (e) => {
      // GSAP animation for scaling up cursor
      scaleAnimation(cursor, 2);
    });

    link.addEventListener("mouseleave", (e) => {
      // GSAP animation for scaling down cursor
      scaleAnimation(cursor, 0);
    });
  });
}

function scaleAnimation(cursor, scale) {
  gsap.to(cursor, 2, {
    scale: scale, // update scale
    ease: "Power3.easeOut"
  });
}

Unfortunately the above function not working as expected, because it will scale up and down each time I move the mouse 'inside' the link. I figured out when commenting out scale: scale in the mouse move event in the gsap.to function it will scale up and don't change until I 'leave' the link. The problem then is, that I scales up smoothly, but immediately back to normal (without smooth transition). Since both mouse enter and leave use the same function, it's strange that only enter is smoothly animated.

If there are any questions left, please let me know. Thank you in advance.