Compose Layout: Align relative to centered item (eg. toRightOf)

Question

Is there a way in Compose to align a composable next to a centered item without using ConstraintLayout?

I want to achieve this:

I could use a Spacer and Weights like this

Row(
    modifier = Modifier.fillMaxWidth(),
    horizontalArrangement = Arrangement.Center,
    verticalAlignment = Alignment.CenterVertically,
) {

    Spacer(Modifier.weight(1f))
    Button(...)
    Label(Modifier.weight(1f),...)
}

Problem is that I display the Label conditionally and if I hide the two elements with the weights, the button moves slightly.

Also not sure if using weights is producing more performance impact than the ConstraintLayout in the first place.