'Detecting incorrect custom model - Compose model in Form Recognizer

I have created a compose model with multiple custom models. For one specific document where it was supposed to use model A for analysis, ends up using model B which is incorrect.

Here are few specific to know:

  • Applications could be completely filled or partially filled (So I added both to my training set)
  • Both the models in discussion are trained on application form totally different in format

Observations:

  • If the test document is tested on model A(correct model), the confidence is 80%
  • If the test document is tested on compose model, modelB is taken up for analyze and docType Confidence is 21% in this case

Q1) Isn't compose model supposed to perform analyze with best fit model? In this case model A is best fit. But model B is selected.

Q2) What's the best practices for training when composing huge set of models (say in the range of 40)

Q3) How can issues like this be fixed?

azure-cognitive-servicesform-recognizerazure-form-recognizer


Solution 1:[1]

I received a response from Microsoft team on this, The way compose model works seems to be refined on 3.0 version. I tried with exact same training data on Form Recognizer Studio (3.0) and the compose feature is working as expected!

Solution 2:[2]

Composed model selects the best fit model to analyze the document. Are documents in Model A and Model B similar in structure ? If the documents are similar you might get higher accuracy creating a single model for both documents. Try adding documents from model B into the training set of model A and compose without Model B and see if that improves the accuracy of the composed model.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Prajwal Ainapur
Solution 2 Neta

Related Questions