control over Vertex AI annotation

Question

In google Vertex AI platform the configuration of labeling the data set is decided by the vertex AI itself, ie. for example the color which is given to the bounding box while labeling is decided by vertex ai, what if I want to assign the color as I my preference? I couldn't see any option to change the annotation color and also tried to feed externally labelled files to the vertex ai and it shows error as there is no field named color;

{"imageGcsUri":"gs://cloud-ai-platform-5100f6e6-d2e6-4966-869e-ba22a09ef85a/bg_MAX_0002.JPG","boundingBoxAnnotations":[{"displayName":"abc","xMin":0.07510431154381085,"xMax":0.34492350486787204,"yMin":0.1022964509394572,"yMax":0.4384133611691023,"annotationResourceLabels":{"aiplatform.googleapis.com/annotation_set_name":"3765893295830466560"}},{"displayName":"pqr","xMin":0.6801112656467315,"xMax":0.9318497913769124,"yMin":0.1503131524008351,"yMax":0.6367432150313153,"annotationResourceLabels":{"aiplatform.googleapis.com/annotation_set_name":"3765893295830466560"}},{"displayName":"xyz","xMin":0.15438108484005564,"xMax":0.6620305980528511,"yMin":0.605427974947808,"yMax":0.906054279749478,"annotationResourceLabels":{"aiplatform.googleapis.com/annotation_set_name":"3765893295830466560"}}],"dataItemResourceLabels":{}}

Given above is an example label generated by vertex ai and it doesn't contain any color information even though I've labeled in multiple colors. So my question is how can we get control over the color feature of the labeling system in vertex AI?