Using shap library for model understanding when model input is a list of tensors

Question

I'm fairly new to deep learning models, how to design them, and how to understand them. I'm trying to understand my model using the SHAP library, but having some issues with following this tutorial.

My model's forward function looks like this, where the input is a list of tensors.

def forward(self, batch, runValidation=False):
        [
            visual_features, token_ids, token_masks, labels, xpath_ids,
            class_ids, attr_token_masks
        ] = batch
        ...
        return loss, class_loss, pr_result, output, visual_features

But based on the tutorial and error messages I'm getting, it seems that SHAP expects the model inputs and outputs to be tensors. Namely, background and x_test_each_class seem to be tensors.

# select backgroud for shap
background = x_train[np.random.choice(x_train.shape[0], 1000, replace=False)]

# DeepExplainer to explain predictions of the model
explainer = shap.DeepExplainer(model, background)

# compute shap values
shap_values = explainer.shap_values(x_test_each_class)

My questions are:

1. Is this proper model design to have the input/output be a list of tensors?
2. If yes, then is it still possible for me to use SHAP on this model?

Answer 1

I ended up concatenating my list of inputs into one long tensor, then writing a wrapper around my forward function to parse those inputs back into their original format before feeding them into the forward function.

Answer 2

I plugin my hololens2 into my PC. and open the OpenXR Developer Tools for Windows Mixed Reality SW, but it show does not have a head mounted display.

The OpenXR Developer Tools for Windows Mixed Reality on PC is only worked for the WMR headsets instead of HoloLens. For HoloLens2, the OpenXR Developer Tools should be installed through the Windows Device Portal or Microsoft Store on HoloLens, and there is a demo scene to test OpenXR runtime.

For Device Portal way, please navigate to the "OpenXR" page and then click the "Install" button under "Developer Features".