AI Engineering Through the Hype

By James Luke, CTO, IBM Distinguished Engineer

James Luke, CTO, IBM Distinguished Engineer

The last few years have seen a surge of interesting artificial intelligence (AI) and for good reason. AI has the potential to transform every sector, reimagining processes, and extracting valuable insights from data previously unobtainable to us. With less than one percent of the data generated in the world being collected and analysed, much of that still represents ‘deep data’—data that hasn’t yet been dug up, mined, and examined for its insights and value. In this era of exponential growth of data, there are significant opportunities for organisations to learn on an exponential scale and become disruptors within their industries.

However, the successful use of AI is not engineering-free. For optimal performance, beyond a simple initial proof of concept, the AI components need to be correctly implemented and configured. Data, the critical aspect of any AI project, must be carefully defined and cleansed. The AI capability must be integrated into a broader end-to-end solution. The whole solution must be designed in a way that is scalable and sustainable. All of these aspects require the skills of experienced data scientists and engineers.

There is a perception that the current success of AI is due to a major algorithmic breakthrough in the domain of machine learning (ML) and, specifically, deep learning. Whilst there have been some great developments in the underlying algorithms, the real enablers behind our current success in AI are:

• the increased availability of data,
• the increase in processing power and storage, and
• the rising desire to use AI to deliver operational benefits.

The first task of any engineer looking to apply AI in an organisation is to select the right business problems to focus on and understanding the feasibility of applying AI to those problems. First and foremost, we need to understand what success looks like and whether it is possible to measure business impact. Data is critical and it is important to ensure the data exists in the right form and at the right level of quality to support the intelligence. Ensuring that AI performs in the right way requires the attention and support of subject matter experts. Similarly, it is important not to overlook the business change aspects that are a key part of any IT project.

The first task of any engineer looking to apply AI in an organisation is to select the right business problems to focus on and understanding the feasibility of applying AI to those problems

In defining solutions, it is important to recognise there is no single, one-size-fits all AI Algorithm. What is becoming increasingly important, when AI is applied, is that the source of the data, the methods used to create, and train the algorithms are transparent and explainable. Detecting and mitigating any bias in algorithm design is an ongoing effort that we at IBM, and all companies advancing AI, have an obligation to address proactively.

To empower companies to take control of their systems, IBM released AI Fairness360, a comprehensive open-source toolkit of metrics to detect unwanted bias in datasets and ML models, and state-of-the-art algorithms to mitigate such bias. The toolkit, available to the open source community, helps developers and data scientists examine and repair bias in AI models throughout the entire AI application lifestyle.

In addition, IBM recently launched ‘Everyday Ethics for Artificial Intelligence’ a first-of-its-kind resource for AI designers and developers, articulating practical best practices captured from experts in fields such as AI, engineering, ethics, and philosophy. It describes five ethical principles that AI developers must consider throughout the design process.

With the skyrocketing growth of data as a driver for the AI era, responsible AI also extends to being responsible when collecting, storing, managing, or processing data. With around 80 percent of resources available to any AI project focused on data definition, preparations, and cleansing, it is of no surprise that the demand for the role and function of the data scientist has dramatically outpaced the supply with 63 percent of executives’ interviews as part of a recent Institute of business Value study: identifying the lack of in-house talent needed to build and implement AI solutions.

The current developments within AI represent an incredible opportunity. The winners in this domain will be those who understand how to successfully apply AI to real world problems. That means selecting the right problem, building solutions instead of algorithms, focusing on the data, and ensuring its responsible and ethical adoption.

One example of this is start up Hello Tractor. Working with IBM scientists in Africa, engineers are building a blockchain-enabled and AI-based decision support platform enabling farmers in sub-Saharan Africa to access tractor services on demand. With more than 60 percent of crops managed manually and food demand increasing due to population growth averaging 11 million per year, it is an unsustainable model. Using ML and IoT, Hello Tractor, will allow tractor fleet owners to view and manage fleet utilisation and predictive maintenance as well as forecast future tractor utilizations based on history, real-time weather, and remote sensing satellite data. This concept of an agricultural digital wallet will enable capturing, tracking, and instant sharing of data, while creating end-to-end trust and transparency for all the parties involved across the agribusiness value chain.

AI requires new skills for new kinds of work. To realise the opportunities this new era presents, we must look to build new partnerships to modernise skills, education for students, invest in re-skilling, and in developing the next generation of engineers with the abilities to fulfil our AI ambitions.