What's the status of AI post-deployment management?
AI Observatory 2020/2021
3 March, 2021
Time reading: 4 minutes
In the year of Covid-19, according to the Artificial Intelligence Observatory of the Politecnico di Milano (https://www.osservatori.net/en), the AI market in Italy has withstood the impact of the pandemic, registering +15% compared to the 2019 value and settling around 300 M€.
A clear gap emerged between large organizations, which have started AI projects in the past and which are now working to put them into production, and medium-sized companies, which, due to the budget reduction – especially in some sectors such as manufacturing – have remained stagnant.
In many companies, the lack of budget for innovative projects, but also the low commitment of the top/middle management and the difficulty in identifying the proper business cases, have slowed down the development of new artificial intelligence projects (we share our case studies here: our case histories).
At the same time, the massive availability of data to exploit and the technological readiness for IoT, Big Data, and Cloud Computing have contributed to the market growth.
Moving down to the ongoing projects, the critical issues highlighted by the companies, mainly concern the preparation of datasets for the training and the performance evaluation.
It is exactly on this last point that we have contributed during this year as a partner of the Artificial Intelligence Observatory.
We suggested including a specific question in company surveys, about the governance and monitoring procedures put in place to manage AI after deployment, to understand their level of awareness and get a picture of current initiatives.
The results showed that 1 company in 3 has not yet implemented any procedures for managing automated systems to ensure they can be integrated into business processes in a way that is reliable, monitored and continuously improving.
Few companies have adopted versioning systems for models and data and procedures for their management (A/B testing, continuous deployment, tests on validation datasets). Even fewer companies have fully understood the importance of input data (don’t forget that AI is a software generated from data!) and how fundamental is to keep them under control in real time and correlate them to model performance.
Companies have mainly focused on periodic reports about the performance of the models, while few companies have chosen to implement a real-time monitoring system.
Moreover, we are still far from the implementation of active learning systems through the automatic construction of datasets for the models retraining, in order to continuously improve the performance of the models.
We read these results as a starting point and a confirmation of our vision focused on production deployment, monitoring and governance, with the aim of ensuring compliance and reliability in all business contexts involving artificial intelligence.
Specifically, we are working to offer our customers the ability to fully exploit the potential of artificial intelligence through invariant.ai®, our technology platform that ensures full observability of AI systems and related processes. From model serving (even on devices with low computational resources), to the monitoring of the machines involved, to the successful execution of services and models, and to the improvement of performance over the time. Up to the complete governance of processes, ensuring compliance with internal procedures and international industry standards.
As an AI Service Company (we share our vision here: who we are), we will work even harder to accompany our clients throughout the entire AI lifecycle, end-to-end, from problem set-up to deployment and monitoring in production, so that the future results of the Artificial Intelligence Observatory can report a step change to a greater awareness by those involved in our companies.
For a conscious use of technology that will bring the innovation and progress needed to restart and grow.
Want to learn more about our technologies?
Visit our solutions section.
Interview with Lisa Lozza -Technical Area Leader Data Science