Barriers to AI Adoption
Behind the AI hype, what barriers are there to AI adoption?
Artificial Intelligence (AI) has gone through a decade of strong growth, in large part due to progress in a subfield of AI called Machine Learning (ML) alongside increases in computational power, connectivity, and data storage capacity. In conventional software systems, programmers craft rules and instructions that computers follow to the letter. In ML systems, computers learn their behaviour not by following rules, but by uncovering patterns in large sets of data. They apply those patterns to make decisions about data they’ve never seen before. ML has been hugely successful in fields like computer vision, natural language processing (NLP) and automatic speech recognition. These are tasks where it’s simply not possible for a programmer to sit down and write a set of rules for the computer to follow. Learning models from data has proven to be remarkably effective. The accuracy of ML models is not necessarily at the level you’d see people performing the task, but often high enough that they can be useful.
Supporting the AI ecosystem is a key government priority in many countries around the world. Here in the UK, it’s no different. The government recently announced its National AI Strategy, outlining a ten year plan to keep the UK at the forefront of AI research and development.
Yet despite rapid progress in the field, it’s not obvious that “AI Adoption” in and of itself is the right goal to aim for, and there are still barriers that companies face when considering AI solutions.
People
There are a range of job roles and titles in AI. In particular, ML Scientists and Engineers are hard to find, highly paid, and globally mobile. These are the people who deeply understand how ML algorithms work, can build and modify them for new applications, and understand the various trade-offs between different ML approaches. Element AI (a Canadian ML company) estimated in 2018 that there were around 22,000 PhD educated ML Scientists globally.
In their 2020 report, Element AI don’t have a comparable figure for the number of PhD educated scientists. They do however estimate that the number of paper authors in the field has consistently been growing by approximately 50% year on year. The growth in paper authors is likely a reasonable proxy for the growth of the field as a whole. They also estimate the total AI workforce in 2020 (not just PhD level scientists but including all related roles such as Software Engineer and Data Engineer) as 478,000 people worldwide.
One side effect of the rapid growth in the workforce is that there are many more newcomers to the field than there are experienced scientists. Senior scientists and experienced tech leads and research managers are even harder to hire than those who have recently graduated. Notably, experienced professors from universities are moving to industry, accepting roles at Big Tech companies (Google, DeepMind, Amazon, Microsoft etc.).
Salary information is hard to summarise because of geographic differences in compensation and a lack of standardisation of job roles. There are, however, several surveys of Software Engineer salaries — e.g. levels.fyi (which is heavily weighted towards the larger Silicon Valley salaries), Glassdoor and Stack Overflow’s developer survey. ML Scientists are typically paid more than Software Engineers. The upshot being that skilled ML scientists with some experience comfortably earn 6 figure salaries. Many PhD researchers move to Big Tech companies, but smaller organisations cannot offer competing benefits.
Data
ML systems learn their behaviour from data, and so it’s important to train them with a large amount of high quality data. The data they are trained and tested on should be representative of the scenario in which they’ll be used. A speech recognition system trained on the dictation data from medical doctors will not perform well in a different scenario like transcription in a courtroom due to the differences in speaking style and language. Even small details like changing the type and location of microphones can have an impact on accuracy of a ML model.
High quality task-specific data is expensive to obtain and annotate. It typically requires human annotation which is a labour intensive task. A speech recognition system may be trained on thousands or tens of thousands of hours of manually transcribed audio to achieve an acceptable accuracy. Manual transcription costs on the order of £1 per minute of audio, and so a large dataset may cost a business tens or even hundreds of thousands of pounds to create. There are large public datasets available, but they tend to be released with non-commercial licenses.
Data is also not just a one-off cost. One built and deployed, a model needs ongoing monitoring and it might also become outdated over time due to “data drift”. That is, the data the model sees starts to look different over time. For speech recognition applications, people use different language over time, new words become popular, events come and go with their own unique vocabulary. These all mean that a deployed model has to be monitored and regularly retrained using high quality data to keep up with a changing world.
Knowledge
There’s a lot of hype about AI and its capabilities, which makes it very difficult for anyone not embedded in the field to know exactly what AI can and can’t do. There are uncertainties about the kinds of problem that AI can be applied to, and also some unrealistic expectations about how well systems can perform.
Business leaders may not know the capabilities and limitations of AI and ML, and so cannot articulate where and how AI can help their business. ML is a technology which has applications in a specific set of problems — those where data exists and people can easily annotate that data with the desired outcome. In reality, AI adoption as an end goal is not directly in line with the aim of most businesses. Finding the problems to which ML can be applied, and making the business case for using it, is the stronger incentive for organisations.
Unrealistic expectations also lead to frustration when first products and prototypes don’t perform as well as anticipated. Then, projects are cancelled prematurely, before the benefit can be realised. Understanding the life cycle and development process behind ML can ease these frustrations and enable companies to make better decisions about whether to pursue projects that initially look unpromising.
Fear
AI is a technology which has made its way into the public consciousness as something to fear, through movies, books and articles. There are very real worries about AI putting people out of jobs, replacing human expertise, or building powerful systems that cannot be controlled. These narratives can dissuade businesses from exploring the use of current ML technology.
Organisational Mindset
Companies who are successful at building and deploying ML technology typically lean on the agile and iterative product building processes from Software Engineering. These ways of working can be at odds with more traditional organisational structures and incentives.
To build a successful ML product, there is a sequence of experimentation and iterative data-driven improvement needed to take a prototype into a fully fledged product. It takes weeks or months to iterate on early versions of a system. For example, despite the existence of speech recognition, NLP and text-to-speech technology in 2011, it took Amazon Alexa 3 years between pitch and launch in 2014, with continuous improvements to Alexa and new features being added regularly since then.
Building up institutional knowledge about the realities of AI and ML over a long period is key for organisations who want to successfully deploy ML products.
Economics
The economics of machine learning are similar in some ways, but also different from software-as-a-service (SaaS) companies.
ML models are not perfect. They make mistakes (often silly mistakes, but sometimes biased), and need human oversight in places where accuracy has to be high. An automated transcription might have an accuracy rate of 90% — i.e. it gets one in every 10 words incorrect — and that might be good enough as the basis for a podcast audio search tool. But it is likely not good enough for medical dictation, where the automated transcript has to be manually edited for a far higher accuracy. This human oversight may be reduced with more accurate models, but is unlikely to disappear any time soon.
Data and people are expensive, but another expense is the high compute cost for training and testing models. As with data, this is not just a one-time cost, but also needed for the ongoing monitoring and rebuilding of ML models. This computing cost for AI companies is typically higher than for SaaS businesses.
Meanwhile, off-the-shelf solutions charge a few cents or less per API call. Google Cloud and Amazon AWS transcription services are priced at under 4 cents per minute of audio, or just under $2.50 per hour of automatically transcribed audio. For small organisations without large budgets, the up-front expense of data, people, and computing power a business needs to build a custom ML system for their task becomes hard to justify without a large enough market to see a return on the investment.
What does the future hold?
These barriers to AI adoption won’t disappear in the short term, but there are some forces at play that will lessen their influence over time.
The AI workforce is growing, as evidenced by Element AI’s report of 50% growth year-on-year in ML publication authors. This is supported by anecdotal stories about the growing popularity of machine learning options in both undergraduate and graduate university courses. Bootcamps and on-the-job learning are also becoming viable paths into machine learning as a career.
Sharing models and data is common in ML to reduce costs. Hugging Face are one company which host shared models and datasets that are free to use by the research community.
For years, the ML community has used a technique called ‘transfer learning’ to quickly adapt an ML model trained for one task to work in a related task, using smaller amounts of data than would be needed for building from scratch. Recently, larger and larger models have been built (pre-trained) as a foundation for transfer learning. Large models, dubbed “foundation models” by Stanford University, are built by organisations who can afford the cost. Examples include Google’s BERT and Open AI’s GPT-3. The latter may have cost in the range of $10 Million — clearly only achievable by established technology companies. These large pre-trained models are shared with the community, and then require far smaller amounts of data to tune to specific tasks. This cuts the cost of data that organisations need to build competitive ML models for their own specific application. However, it places Big Tech companies with deep pockets right in the middle of AI adoption, as they are the only ones with resources to build foundation models.
There is a steady stream of work to improve and standardise tooling, making ML easier to work with and easier to adapt to new applications. For example, two popular ML frameworks Tensorflow and PyTorch were released about 6 years ago and are now used by a majority of ML Scientists.
Finally, the cost of computing is decreasing, making it cheaper and faster to build and use ML models. In the past decade, ML workloads have moved from CPUs to GPUs, spurred by the success of AlexNet — 2012’s GPU-trained neural network which conclusively won the ImageNet competition. More recently, hardware companies are investing in targeted hardware designed specifically for ML.
AI and ML have come a long way in the last decade. Tech Nation reports that the UK has 1300 AI companies, a growth of 600% in the past decade, with a collective turnover of almost $2B. As these companies grow and others start up, the landscape and barriers are likely to look very different again in another decade.
