Is there a way to get the accuracy metric value from deployed model in DataRobot using api code

Question

Through python Api's , I have uploaded and deployed an ML based custom model on DataRobot Platform successfully. Now how to get the accuracy metric for the deployed model.

NOTE: ACCURACY_METRIC used is 'LOGGLOSS'.

I tried accuracy_over_time.metric. it gave "LogLoss" as an output result. but how to get the metric value for this LogLoss metric.

I tried accuracy_over_time.metric_values. It says "there is no attribute as 'metric_values'."

Answer 1

You're looking for get_accuracy():

accuracy = deployment.get_accuracy(
    start_time=datetime(2019, 8, 1, hour=15),
    end_time=datetime(2019, 8, 1, 15, 0)
)

You'll need to define the time span if you want accuracy over time, like so:

rmse = deployment.get_accuracy_over_time(
    start_time=datetime(2019, 8, 1),
    end_time=datetime(2019, 8, 3),
    bucket_size=construct_duration_string(days=1),
    metric=ACCURACY_METRIC.RMSE
)