Features of comparative analysis of models of intelligent document processing

In recent years, intelligent document processing technologies have undergone significant changes, providing businesses with more efficient and accurate automation of document processing. Businessware Technologies regularly tests various AI models to evaluate their performance in real-world conditions. As part of their AI benchmark, several popular IDP models were analyzed, including:

• Azure AI Document Intelligence;
• GPT-4o;
• Google Document AI and others.

Main testing criteria

The main testing criteria are as follows:

1. Recognition accuracy — an assessment of the model’s ability to accurately extract data from documents, such as field names, values, document layout and text blocks.
2. Processing time — the average time it takes the model to process one document.
3. Cost — the cost of processing 1000 pages and any additional costs.

Test results

In the invoice processing tests, the models showed the following results. The testing results of different IDP models show significant differences in recognition accuracy, processing speed and cost. The Amazon Analyze Expense API model demonstrated high invoice detection accuracy and fast processing, while delivering an average cost of $10 per 1,000 pages. Azure AI Document Intelligence demonstrated slightly lower accuracy and slightly longer processing times, while delivering a similar cost.

GPT-4o using third-party OCR delivered high accuracy but took significantly longer to process, while delivering a slightly lower cost of $8.8 per 1,000 pages. Google Document AI demonstrated relatively low accuracy, especially when dealing with document elements, but still delivered fast processing times, with a cost of $10 per 1,000 pages. These results highlight the importance of selecting the right IDP model based on specific business needs, such as accuracy, processing speed, and cost. The Businessware Technologies AI benchmark provides valuable insights for organizations looking to optimize their document processing processes. Understanding the strengths and weaknesses of different IDP models allows you to make an informed choice and implement the most appropriate solution for your needs.