We’re in the Digital Transformation age and everyone are using mobile phones, computers, and tablet to solve business problems. We now automate business processes to work productively and efficiently. One of the main process that business users wanted to automate is the paper approval. Business users want to automate this because of the issue where you need to look for the approver in his/her desk to approve your form. If the approver is out of reach, your paper will just be in your desk waiting until the Approver arrives.
With all the digital transformation and automation, Power Platform released General Availability of AI Builder last October 2019 but some of its features are still in preview. And one of the features that I’m playing around is the Form-Processing (Preview).
While I’m doing some proposal for work, I did this scene where the user will create a list in SharePoint, attached an invoice form to the item and then submit. Once submitted, the Power Automate will trigger to get the attached file from the newly created item. It will then read the invoice form using AI Model Form-Processing. Once it is extracted, the data from the invoice form using the AI Model will then update the list item in SharePoint. The below image is the Power Automate that I’ve been working on. I will show you the detailed setup that I have for my Power Automate in my next post.
For now, below is the image for the flow.
Flow for picking up the attached file from SharePoint item
How Does it Work
Form-Processing extracts data fields and tables from the form. In your Power Apps environment, you can see the AI Builder in the left pane of the Web Studio. From there, you can build your Models like Form Processing, Object Detection, Prediction and other available AI productivity.
AI Builder in Power Apps
In building the AI Model using the Form-Processing, you need to have a minimum of 5 documents to train your model. The model can detect duplicate document. Since it is a duplicate, it will ignore and will not upload the document.
Information if the document uploaded has a duplicate.
After you are done uploading all the documents, you start to Analyze the document.
Analyzing the document uploaded
After analyzing all the documents, now you start to select the fields that you are going to extract from the document.
Selecting the fields that you want to extract from the document
Once done selecting the fields, you train the model.
Train the model
After training the model, you will then publish it. After publishing the model, you can now start using it in your Power Automate and Power Apps.
Your Form-Processing Model
This is only my own experience and comments with regards to AI Builder Form-Processing. As I went through my testing and configuration, some things need to address and this would be useful as well. As I have some experience with AI application for the extraction of the data through scanning of the documents. Probably this would be helpful as well for this feature.
- When you are done uploading the file and done with analyzing the document if you don’t want to use the document that has been analyzed and need to upload a new document. The new document will not be analyzed anymore. You need to start creating a new model again and start analyzing.
- There is no option for the trained model to manually re-train it. Let’s say for example you have more than 20 documents that already analyze but then, the document doesn’t properly extract the fields or the tables. You can’t re-train the model.
- There is no option for you to split the documents if there are multiple pages. I know part of the best practice is no multiple pages. If these can be extended to multiple pages, this could be a great value to the app. Especially when the form reaches even the two pages only.
- You can’t adjust the positioning of the extracted fields or tables. Let’s say, you have a four-column table. S/No, Description, Rates, Remarks are the headers. During the analysis of your model, extracted fields of the Rates are somewhat nearer to the fields of the Remarks. If the table of the document that you are going to extract is not the same alignment for the trained model. There will be a possibility that the Rates will be read as Remarks if the Description field is wider than the trained model.
- If there could be a manual training of the model, we can define the unique keyword of the document. From there, you can set manually the coordinates of the keywords. You can identify if the keyword that you are looking are either in the TOP header or in the footer.
We know that Form-Processing is still in Preview. There is still room for improvement. If this thing can be addressed, AI Builder Form-Processing can be a powerful tool for extracting forms. And because it is under the Power Platform, you can leverage the Power Apps and Power Automate more and it would be easy to build and can leverage more on the business process.
Featured image by Pixabay