Non-user controlled TextareaAutosize and TextField goes into infinite loop

Question

How does one supposed to use controlled react-textarea-autosize with non-user input? Example:

import TextareaAutosize from '@mui/base/TextareaAutosize'

const text1 = 'Lorem ipsum...'
const text2 = 'Lorem ipsum dolor sit amet...'
const [text, setText] = useState(text1)

const handleChange = () => {
  if (text === text1) {
    setText(text2)
  } else {
    setText(text1)
  }
}

...

<TextareaAutosize value={text} />
<button onClick={handleChange}>change text</button>

...

This doesn't work and TextareaAutosize just goes into infinite loop and eventually throws error MUI: Too many re-renders. The layout is unstable. TextareaAutosize limits the number of renders to prevent an infinite loop. and doesn't change it's state.

Also the same is true for TextField with multiline prop which just uses TextareaAutosize as the base and according to the docs can be controlled.

Codepen with TextareaAutosize: https://codepen.io/kengho/pen/popGYvx

Codepen with TextField : https://codepen.io/kengho/pen/oNpmOZq