By Hanno Brink, Machine Learning Engineer at Synthesis
There are a plethora of technologies for companies to choose from that can add value, increase competitiveness and improve business processes. The point is to choose correctly rather than getting lost in the lingo.
When it comes to cutting edge Artificial Intelligence (AI) and Machine Learning (ML) research, there have been many exciting and well publicised breakthroughs. An example of this is the new GPT3 model that has been causing quite a stir with its text generation abilities. It even seems to be capable of generating code snippets from only a brief description of what the functionality should look like. There are also reinforcement learning agents learning to beat masters of many games without the need for any human input. These breakthroughs are very exciting, but when it comes to the type of advances that organisations care about, there is a whole range of new trends that organisations, entrepreneurs and technical personnel can capitalise on.
ML and AI have been proposed as the tools that every organisation needs to embrace to stay competitive. Many organisations started adopting these technologies for specific use cases such as document validation, email routing, sentiment analysis, data pipeline monitoring, and some compliance and fraud detection tasks. AI can also enable businesses to perform tasks and deliver services that would not have been possible or feasible before the advent of AI technologies. Examples of this include the use of Natural Language Processing (NLP) and chatbots for increased customer engagement, customer base segmentation, churn prediction, hyper personalisation, and fraud detection.
A few organisations have learnt hard lessons with unsuccessful data projects and a few data projects have started adding exponential value. The technology is slowly being understood and the hype is slowly dying. This hype is now being replaced by a true understanding of the cost and limitations of these technologies. This is driving the adoption of technologies that are fundamental to building a solid foundation for improved Business Intelligence (BI), ML and AI adoption. One of the most pressing issues that organisations are addressing is the question of data ingestion and storage at scale. This is being solved by the building of data pipelines that transform raw data and feed it into data lakes, used to store data in a suitable place and format for further analysis.
Once the foundations are in place, organisations are addressing the maintainability issues of ML projects by making ML-Ops a key focus in their implementation of advanced algorithms, as a comprehensive ML-Ops capability provides an easy way to collect, prepare and store training data, iteratively train large ML models, detect model drift and version control models and training data. Many companies have created tools to address these issues, and a shortage of skills in this area makes this an exciting area to keep an eye on.
The high cost of development has led to a decline in the eagerness with which organisations are looking to build bespoke deep learning models for custom ML tasks. This brings us to the other interesting trend that is taking place, namely the use of pre-trained models and “AI as a service” offerings for common ML and AI tasks such as image analysis, document understanding, speech-to-text and many more. Since these tasks require an immense amount of data and fine-tuning to get a performant model, it makes sense to use an existing solution rather than having to build your own.
One final trend in the AI/ML space is the rise of ML platforms as a service, where with very little time and infrastructure investment, organisations can provide their own data and requirements to the ML platform and very quickly iterate over model and data versions without the need to build the entire workflow from scratch.
As these technologies mature, the landscape is becoming more complex, and the thinking more nuanced. We can therefore look forward to many innovations in this space.
