AI - A New Frontier for Intelligent Document Processing? - Myth or Reality?

"If you do not know how to ask the right question, you discover nothing."
W. Edwards Deming

Sixty years ago, mathematician and philosopher, Norbert Wiener, published a book about the role of machines in society. The book ended with a warning: "We shall never receive the right answers to our questions unless we ask the right questions.

The question we ask in this article is: In what real sense is Artificial Intelligence (AI) and Machine Learning (ML) a reality in the context of Intelligent Document Processing (IDP)?

Here, Mark Edwards of Boss Equity leverages his experience and network in the software sector to discuss this question.

They will also be considering the extent to which Machine Learning solves today's problems in this technology sector and how much the industry is hampered by marketing hype?

What is (IDP) Intelligent Document Processing?

Along with the plethora of much-used 3 letter acronyms in the Information Management market over the past two decades, yet another acronym has gained recent attention - "IDP" or Intelligent Document Processing.

IDP is a digital process that enables the streamlining of document processing. It uses technologies such as Machine Learning (ML), Natural Language Processing (NLP), Robotic Process Automation or Intelligent Character Recognition (ICR) to process documents to reduce or eliminate tasks within a process that would normally rely on human intelligence. This could be in any organisation where they need to process high volumes of semi-structured and unstructured documents.

IDP could involve any software product or solution that captures data from documents, categorizes and extracts relevant data for further processing.

Although the key is around the word "Intelligent", the ability to capture, extract and analyse business-critical documents has been effectively deployed for the past 20 years. This has been achieved using OCR and IDR (Intelligent Document Recognition) technologies, most notably for the electronic processing of invoices and associated transactional documents. These have been widely deployed by several Document Capture software vendors, providing an on-ramp for a raft of business processes.

What Has Changed?

Document Definition - Documents now encompass any container of content, whether emails, blogs, SM messages and, increasingly, audio and voice communications. With the exponential growth of "content", spawned by the internet, Web 2.0, SaaS & Cloud computing and an increasingly remote and mobile workforce, the need for organizations to find the right content, at the right time to make the right decisions, has become paramount.

Whatever the format, the information represented can be extracted, classified and presented to drive internal business processes.

Technology advances
The application of AI and associated Machine Learning (ML) & Natural language Processing (NLP) technologies has now enabled the automation of routine business processes and better business intelligence, particularly by adding a layer of intelligent, contextual sentiment analysis so the relevant data can be extracted. Data extraction from documents no longer simply relies on templates and forms recognition.

As noted by Alan Pelz-Sharp, (Founder & Principal Analyst of Deep Analysis): “Although Machine Learning and NLP have been with us for some time, the new use of Deep Learning by Amazon, Google, Microsoft etc for understanding language are transforming this field. Others such as Corticol.io are using advanced neuroscience techniques (HTM) to do the same. In short, though NLP has been with us for decades, more modern techniques are delivering much more accurate, speedy and relevant insights. However, there are many start up’s coming into the Cognitive Capture sector – leveraging expertise in AI that have little knowledge of traditional approaches of many vendors in this sector”
Merging of Back Office & Front Office
A growing trend in recent years has been the drive to improve customer communications and associated marketing automation systems by more accurate real-time data analytics. However, these solutions still rely on extracting content from back-end, document-intensive, legacy systems in order to enhance key line of business applications.
Capture 2.0
As summarised by Harvey Spencer Associates (now VP of Software at Infosource): "We can now capture the key data from multi-channel inputs derived from many sources in multiple different ways. What is changing is the level of data understanding and integration to the back end processes, leading to "smarter" or maybe "Intelligent" Capture 2.0 automation services".

Infosource has measured the market for Capture 2.0 Services, currently at $32bn. However, considering the growth of RPA, this is undoubtedly much higher.

Impact of Cloud Capture
As reported by Infosource, Capture 2.0 Intelligent Capture Cloud-based services and Robotic Process Automation were the key drivers of revenue growth. In 2019, the worldwide market grew by 16.5%, reaching US$5.5bn in end-user revenues.

As highlighted by Harvey Spencer, “standard, containerised cloud services, combined with load management services, such as Kubernetes, now exist. This is a self-healing set of services that orchestrates and manages many services on multiple different cloud servers, including prioritisation”.

Robotic Process Automation (RPA) Capture Systems are contributing to strong growth with an increase of more than 60% year-over-year. Cloud services are forecast to continue to grow by more than 25% CAGR through 2024, to over $1.7bn.

Nevertheless, the beginning of true native Cloud Capture appears to be still in its infancy. As Alan Pelz Sharp continues: “Although various firms have cloud capture capabilities – the challenge here is volume. In industrial style mass scanning 100ppm etc physics gets in the way – you have lag and have to install a T1 connection of upload in batches. For these scenarios Cloud probably isn’t the way to go if you are truly handling huge volumes – or rather, you take a hybrid approach to capturing on premises – then pushing batches to the cloud for full processing

How much is hype versus reality? We would welcome readers’ views on this - particularly if you have examples of native cloud capture.

Rise of Low code/No code

"No-code" software is part of yet another increasing trend towards digital transformation, with Forrester Research finding that 84% of enterprises have already begun using low-code (limited coding required) and no-code software. Unlike legacy systems that may pose problems with integration and data access, low-code /no-code platforms are accessible by everyone without dependency on programmers.

There is a growing requirement for software to have a user-friendly interface, particularly with a more decentralised & mobile workforce. This is also a changing dynamic of the sales channel with the modifications to both the buying mechanism & target customer. IT departments are no longer the key decision makers.

Gartner predicts over 65 per cent of all app development functions will be built by low-code by 2024.

Impact of Covid-19

The dramatic increase in home working, caused by the pandemic outbreak has accelerated the deployment of collaborative IT software platforms and cybersecurity requirements. This has contributed to changes in centralized IT systems and infrastructures to address the overflowing support required from IT teams.

It has accelerated the digital revolution & deployment of technologies, many of which have been around for several years.
Most importantly, it has changed people’s work habits and mindset of how to use technology more effectively.
It has catapulted the application of IT software systems to the top of Board Room agendas.

At Boss Equity, we believe it has resulted in increasing the acquisition appetite of software tech companies, compared to other business sectors that are more vulnerable to the effects of the pandemic.

Graphical Presentment of Data – Next-generation BI
No doubt many will recall the much-hyped term, “knowledge management” (KM) in the early 2000’s, with some hardware vendors claiming their scanners were “knowledge management ready”. This gave rise to (BI) Business Intelligence software with graphical dashboards to extract and present key data to assist executives’ decision-making. Their lack of capability to extract intelligent data at that time, created a fair bit of cynicism in the marketplace that slowed the growth of such solutions for many years.

However, the advancement in the component technologies needed in BI and KM solutions such as AI technologies, can now further automate many previously manual tasks. Consequently, the solutions are now able to provide a more comprehensive and contextual presentment of corporate information.

Advancements in technology have also added increased complexity, to the detriment of the end user. If end users are given tools which are so complex, they require the assistance of data scientists and IT experts, then the uptake and general acceptance is going to be diminished. It is vital that user interfaces facilitate autonomous use and direct access to results, without the need for interpretation or assistance. This is an issue that needs to be addressed. A study by Vanson Borne* stated: “Only 25 percent of respondents said analytics intelligence sufficiently makes its way into the hands of business decision makers without the need for data scientists. And nearly two thirds (63 percent) say it’s hard for non-analytics workers to consume analytics within their organisation.”

The report continues: “53 percent say their organization is “overburdened” by the complexity of its analytics.”

Challenges to Achieving Pervasive Data Intelligence
Company leaders claim there is too much complexity in their analytics technology.

- 74% said that their organisation’s analytics technology is complex

- 42% say one of the main issues driving complexity is that the technology isn’t easy for all employees to understand.

Getting to the Data:

Users have limited access to the data they need to make effective decisions that combat their toughest challenges.

- 79% need more data, available all the time, to do their job effectively

- 28% of the data needed to perform at peak, is missing

Big Tech Disruption

More recently, many of the software giants have entered the fray by deploying AI technologies to extract, analyse & present key information, held in both structured and unstructured documents.

Amazon’s (AWS) 'Textract' solution applies ML to "extract text & data from virtually any document, including tables & forms, by capturing, extracting and analysing contextual data from text, including PDF files". As this is provided as a service, the low cost per page is very attractive.

Google's Document AI offers a scalable, serverless platform to automatically classify, extract and enrich data from scanned documents. Google uses the same ML that powers Google Search, Google Assistant, NLP API to drive its internal insights from unstructured documents.

Microsoft’s new 'Project Cortex' technology acts as a knowledge network within their 365 Apps, which includes existing tools such as M/S Graph, Search & SharePoint. The application of AI changes the way data is processed, presenting it in a way that shortens employee workflows, deepens company-wide knowledge sharing and enhances automation.

While this may be viewed as a threat by other software vendors in this space, the above solutions expand the awareness and growth in the broader Intelligent Document Processing market. Many may recall when Microsoft launched SharePoint, there was a concern by Enterprise Content Management (ECM) software vendors that this would be a threat to their core offerings in this sector. This concern was never realised and in fact, gave rise to many software vendors providing added value components to integrate with and add value to SharePoint and, more latterly, OS365 apps.

As anticipated by Harvey, Microsoft Resellers will likely now become a key channel for Capture 2.0

The ongoing value of such ISV’s still resides in a combination of their deeper understanding of complex business processes and deep domain knowledge of specific enterprise software applications and markets.

For some years now, IBM too, has applied AI via their 'Watson’ technology which processes key insights, patterns & relationships across unstructured images, emails, Social Media and other inputs, to improve decision-making. This enables organizations to predict future outcomes, automate complex processes and optimize employees’ time. Valuable insights from social trends can also be captured & analysed in real time by integrating IBM’s Watson Natural Classifier Service.

Automated Supply Chain Management & Industry 4.0

The deployment of AI technologies and the drive to achieve RPA is having a dramatic effect on manufacturing, and supply chain (& financial) management in an increasingly dispersed and globalised economy.

Industry 4.0 is the ongoing automation of traditional manufacturing and industrial practices, using modern smart technology. Large-scale machine to machine communication (M2M) and the internet of things (IoT)can now be integrated for increased automation, improved communication and self-monitoring, and production of smart machines that can analyse and diagnose issues without the need for human intervention.

As highlighted by Harvey Spencer, “Until now, Document Capture software has primarily been deployed on a departmental basis in areas such as ordering, invoice processing, payment & contracts management. However, with the addition of AI this can now be further automated and integrated with both ERP and CRM systems.

For example, by adding AI semantic and contextual analysis for call centres, valuable data about customer needs, emotions & preferences can allow manufacturers to dynamically adapt to rapid changes in market demand. This results in faster time to market, lower costs, improved cash flow and happier customers”. So, this represents new opportunities for “Cognitive Capture”

AI, RPA & Hyper Automation

A new term, “hyper automation”, is gaining traction in the context of AI and Robotic Process Automation (RPA).

As defined by Gartner, “hyper automation is a disciplined approach to rapidly identify, vet and automate as many business and IT processes as possible, through the orchestrated use of multiple technologies, tools or platforms.”

Relative to Intelligent Document Processing, the challenge is in adding both a semantic and contextual layer of understanding of data, typically held within documents, PDF’s & emails. While advances in machine learning have automated many manual processes, in many cases, some level of human interaction is still required to validate the data captured and analysed.

According to AI Research from AI Multiple, “Though RPA has been the fastest-growing segment of the global enterprise software market and the payback period is shorter than other digital transformation technologies, RPA has limitations such as its reliance on structured data. A traditional RPA software cannot understand the context and learn itself by using unstructured data. It requires humans to code rules-based tasks, interpret data and make decisions about automation.”

Harvey Spencer agrees: “Capture & Workflow adds the rules-based tasks & accurate interpretation to allow automated decision making. It is critical that the captured and interpreted data is accurate as we become reliant on AI. The old adage of “garbage in; garbage out” is magnified in an AI-centric world and becomes a key challenge”.

Capture software have for many years have understood very clearly the need for accuracy.

This is further highlighted by their graph below.

Another related term that is being referred to is "Cognitive Automation". Powered by AI

technology, cognitive automation possesses the capacity to handle complex, unstructured,
and data-laden tasks to create greater understanding and better decision-making.

SUMMARY:

“Despite all the hype and excitement about AI, it's still extremely limited today, relative to human intelligence.”

Andrew Ng

A long-term positive of Artificial intelligence is its enormous potential to relieve people from menial and boring tasks. Despite the fear of job losses, AI creates a shift in society, freeing people to spend their time doing more meaningful work to which they are more suited. In regard IDP and the developing technologies we have been discussing, there is will be a significant change and increase in capability over the next few years. This does not mean that existing technology solutions and components will become redundant - Much of what is currently available “does the job” so why change?

ML & NLP technologies add an important, contextual & semantic layer for data & document processing. However, existing, long-standing, traditional document capture & extraction techniques remain valid & sufficient for many business-critical document & data applications. Many successful software vendors have deep domain expertise in specific sectors such as FinTech, Public Sector and the increasingly important Healthcare sector. A combination of their IP and the ability to improve complex, document intensive business processes, increases their value.

In many cases, this is not always represented in the overall price of any solution.
The key is in the value of the application not just the technology.
The true value of software platforms is often derived from key technology components that enhance the overall capability of the solution.

By way of illustration on that last bullet point, a few weeks ago, Boss completed the sale of AquaForest to ORPALIS – Both strong technical companies. One of the technical components that AquaForest had developed was the ability to search an archive, a whole department or even an entire enterprise, to identify documents that were not accessible to content search.

A Business Intelligence (BI) solution provider unable to access 25% of the content of the enterprise because it’s in PDF format, password-protected or languishing as attachments to emails, seriously reduces the value of such BI enterprise solutions. When you add into the mix audio and video content that is not searchable, you then have a significant percentage of potentially valuable “knowledge bank” data that is being ignored.

So, Where Is IDP Today?

The state of IDP technology is now at a critical point of delivery. Through the recent advancements of AI, ML, NLP and RPA-related component technologies, the solutions/platforms are now able to open new markets. With increased capability addressing real-world practical applications, more organisations will be setting aside budget to invest in IDP to enhance efficiencies within their business processes.

As is the case with all areas of technology disruption, there is always considerable marketing hype - In many cases, this is more myth than reality. Marketing departments wishing to jump on the latest buzzword bandwagon can increase cynicism from the user community.

However, understanding the true capabilities of the component technologies and what is ‘under the hood’ of these new solutions & platforms, can be very revealing. With respect of Capture technologies - as a critical element of IDP - the addition of associated AI technologies has stimulated a pivotal point in future capabilities. Nevertheless, they need to be integrated as part of the whole solution which we believe will show a sharp increase in end-user acceptance and uptake.

International mergers and acquisitions (M&A) company, Boss Equity https://www.bossequity.com/ , anticipate increased appetite for acquisitions, higher valuations and further investments in this sector.

About the Author:

Mark Edwards: https://www.bossequity.com/about-us/mark-edwards

Market Expert Contributors:

Harvey Spencer: (VP Software at Infosource) – https://www.info-source.com
Alan Pelz-Sharp: (Founder & Principal Analyst, Deep Analysis)
https://www.deep-analysis.net/