h2o Flow UI: Build Model but no ROC CURVE or AUC for multiclass?

Question

Reference to https://dzone.com/articles/machine-learning-with-h2o-hands-on-guide-for-data

I was able to follow the example and plotted the ROC and AUC curve but when I did the same thing with my data, I have "SCORING HISTORY - DEVIANCE" instead of "SCORING HISTORY - LOGLOSS" and my "ROC CURVE - TRAINING METRICS" did not appear.

My dataset has 2 classes, 0, 1 instead of yes and no from the example. What determines whether the ROC curve will be portable?

Updated 10 Jul 2019 reference to Maurever's reply: I have "convert to enum" for my response "label" and change distribution to "bernoulli" but I still can't plot ROC.

Update: For future users who has the same encounter as me. Seems like h2o cannot use response with class as "0" and "1" even if you convert to enum. My problem is solved by relabel of 0 and 1 to "Normal" and "Malicious" and the ROC curve will be plot.

Answer 1

My dataset has 2 classes, 0, 1 instead of yes and no from the example.

What will have happened is that H2O has decided that was a numeric column, rather than a categorical (aka factor) column. The fix is simply to use as.factor() on that column. Do this just after importing the data, but before using it to build a model.

Then it will know to build a classification model, not a regression model, and you will get the metrics you expect to see.

Here is an example: https://stackoverflow.com/a/41441578/841830 (or search for as.factor in the H2O docs, where you will find loads more)

In Flow, you first upload the file, then you click parse files, and it shows a list of the columns. The categorical/factor columns are called "Enum" there (yes, it is confusing). Your column is presumably showing as numeric, so go to the dropdown box and change it to be enum. Then start the parse. Then, after that, build your model.

Answer 2

My problem was fixed by changing the target variable from 0s and 1s to False and True values.