Pulling some data into an ML model is only the start, with governance, processes, metric tracking, and continuous delivery mechanisms all required to keep an MLOps pipeline running smoothly.  

If you’re new to Machine Learning Operations (MLOps) then this article is for you. We’ll look at what MLOps means, the principles that underpin it, and the benefits it can bring to your business. Then to finish, we’ll out four areas to think about to help you get started on your own MLOps journey. 

What is MLOps? A Machine Learning Operations definition

MLOps is the repeatable process of planning, building, automating, deploying, and monitoring new Machine Learning models within your production environment.  

Like other continuous delivery workflows, such as DevOps, MLOps encompasses a range of different disciplines, such as new model design, feature engineering, testing, and new model deployment, as well as the governance controls that sit around the process such as data analysis protocols, security, and cataloging.  

For all of these disciplines to come together, data science teams need to work to a set of principles and standards to ensure model performance and reliability is guaranteed to enable business operations.  

To achieve this performance and stability, most MLOps teams look to achieve the following goals: 

• Achieve the highest level of data quality through stable model architecture and testing 
• Accelerate the model training process to speed up the enhancement of capabilities 
• Automate and streamline the deployment process through continuous integration 
• Enable collaboration between data scientists, ML engineers, and IT professionals 
• Ensure regulatory compliance and responsible AI practices 
• Always striving to improve reliability, reproducibility, auditability, and governance 

Ultimately, MLOps works to leverage the benefits of Machine Learning by putting in place a set of processes and workflows that balance speed, quality, performance, and stability.  

Also read: 

• Fighting Tech Debt with AI 
• Data-driven decision making 

MLOps Principles – pipeline & model development

Before you begin building any ML pipeline components, you need to set the principles you’ll work from. These principles underpin each aspect of MLOps, providing stability on how you’ll go from an initial idea to getting your first ML model in production. 

Best practice MLOps principles include: 

• Continuous flow. Machine Learning isn’t a one and done deployment. Instead, teams should establish a way of working that’s continuous, whether that be continuous design, continuous training, continuous integration, continuous deployment, or continuous monitoring (ideally, all of the above!). 
• Automation by design. Building on from continuous flow, ML teams should strive to use automation to achieve repeatability, consistency, and scalability. Not only will this help improve the effectiveness of ML projects, it also reduces risk across the entire production pipeline.  
• Version Control & Reproducibility: All code, data, models, and configurations should be version controlled for full transparency and reproducibility of experiments. This allows for easy rollbacks and prevents regressions.
• Modular, Reusable Components: The ML codebase should be designed in a modular way with clean separation of assets. This improves code reuse, testability, and enables faster iterations to improve the pace of the ML training pipeline.  
• Strong Data & Model Governance: MLOps platforms help manage the full data pipeline, including data identification, data preparation and transformation, data experimentation, and data testing. The effectiveness of any MI model depends on the data quality, so it needs to be strictly controlled and held to a high standard. 

The benefits of MLOps – best practices, automation & more

Adopting MLOps principles and tooling can unlock significant value for Machine Learning teams and the wider business. Here are just some of the biggest benefits on offer for those who get MLOps right: 

• Faster Delivery. By automating manual ML pipeline processes and hand-offs, MLOps accelerates the entire delivery pipeline from model experimentation to production deployment and monitoring. This means new MI models make it into production faster, enhancing the capabilities available to the business operations teams. 
• Improved Quality & Reliability. End-to-end testing, monitoring, and artifact management ensure models are robust and stable before they enter the production environment. This is especially important in the model training and learning phase, where errors in testing can lead to negative business consequences, especially where an ML model is used to make business-critical decisions. 
• Greater model performance. When combining the speed, automation, consistency, and reliability of ML Ops, ultimately, you get a better ML model at the end of it. This enhances the value of Machine Learning capabilities, making them more effective and increasing their ability to drive better business outcomes.  
• Cost Optimization. Rapid learning, faster deployment, and stronger performance all lead to efficiencies that optimize the overall running costs of MLOps practices. MLOps automation also reduces technical debt, while strategies like multimodel deployment and automated retraining help optimize cloud costs.
• Easier Collaboration. Much like DevOps and other agile-based ways of working, MLOps fosters collaboration between the diverse roles involved in building Machine Learning capabilities. This includes data engineers, data scientists, ML developers, DevOps engineers, testers, business analysts and more, all brought together through common processes and tooling. 
• Scale & Governance. As Machine Learning gets more and more popular, your business use case and production pipeline is only going to get busier. MLOps practices help maintain oversight, traceability, and control over your sprawling landscape, enabling you to scale while still maintaining control.  

MLOps maturity – MLOps Level 0, MLOps Level 1, and MLOps Level 2 explained

As you begin on your MLOps journey, many people will refer to the three levels of MLOps – levels 0, 1, and 2. This MLOps maturity metric helps organizations understand where they are on the journey and the key aspects of MLOps practice they can improve on in the future.  

Let’s look at each level of the MLOps maturity journey, and what they mean.  

MLOps Level 0 

MLOps Level 0 is the starting point for most organizations, where: 

• Machine Learning model development and deployment processes are largely manual, ad-hoc, and lacking standardization at every step. 
• There is minimal to no automation, with data scientists and engineers performing manual tasks. 
• Model monitoring and training are often reactive and/or neglected entirely. 

MLOps Level 1 

At MLOps Level 1, organizations begin introducing basic automation by: 

• Scripting and scheduled activities at certain stages of the ML lifecycle, such as data ingestion, model training, or testing. 
• Deployment and monitoring processes typically remain manual. 
• It’s likely the ML pipeline still lacks the end-to-end governance, control, and standardization to make it fully robust. 

MLOps Level 2 

Finally, MLOps Level 2 represents a mature and comprehensive implementation, including:  

• Fully automating the ML pipeline, including data preparation, model training, evaluation, deployment, monitoring, and retraining.  
• Governance, standards, and control are implemented systematically across the pipeline. 
• Manual intervention is minimal, with data science teams focusing on Deep Learning strategies and analyzing monitoring outputs.  
• Level 2 enables reliable, repeatable, and scalable ML deployment processes across the organization. 

How to implement MLOps – build, deploy, monitor, and automate your own ML pipeline

As we’ve seen, MLOps brings together many practices, platforms, and processes to make it a success. While there are many MLOps solutions out there to choose from, there are four, high-level, MLOps areas you need to create to get started. These are: 

1. Build – A phase where you pull together data and ML models that meet your pipeline needs. 
2. Deploy – Taking your newly build ML capability and putting it into production. 
3. Monitor – Tracking key MLOps metrics and monitoring performance to make improvements. 
4. Governance & Automation – With a pipeline in place, automating it to drive maturity, stability, and efficiency.  

Let’s take a look at some points to think of in each area to help you use MLOps in your business right away. 

#1 – Starting building your ML capability 

The build stage of the best MLOps frameworks helps get the foundations in place by pulling together data, selecting a model, and testing that it works for your business use case. Specifically, think about these key things: 

• Data Management. Start by identifying, curating, cleansing, and aligning the training data you’ll use to build models. This will depend on your ML application and may even include data that’s specific to your business. 
• Choosing a Model. Next is model selection, where you’ll select the optimal MLOps architecture and algorithm for your use case. Whether it’s linear regression, decision tree, or k-means, there are many out there to pick from, so take the time to research and make the best decision.  
• Testing & Evaluation. Before deploying, comprehensive testing is critical to avoid any mistakes. This includes data integrity checks, unit/integration tests, model validation, regression testing, and model evaluation. Inadequate testing is where Machine Learning models crash and burn, so take the time to thoroughly test your pipeline.  

#2 – Deploy your first ML application 

Once you have your ML application ready to go, it’s time to deploy it out into the world. To nail the execution of the ML pipeline, you need to package and deploy it safely. This includes: 

• Model Packaging. Winning models must be packaged in a secure and scalable way that enables them to be rolled back if any issues occur. Choose a packaging approach that enables this and fits in with your wider deployment practices. 
• Serving Infrastructure. Serving infrastructure like Kubernetes or a serverless platform needs to be ready to host and scale your package. Before you deploy a whole training pipeline, your infrastructure should support capabilities like automated scaling, rolling updates, canary deployments, and monitoring/logging.  

#3 – Monitor your deployment closely 

Now that your application is deployed in the real world, you must keep a close eye on it to ensure it performs as expected. This includes looking at each model in isolation and across your portfolio of models, if applicable. Specifically, put the following in place: 

• Model Monitoring. Once deployed, it’s critical to monitor models for drift, staleness, bias, and performance degradation. Not only does this help ensure quality and stability, but it also enables you to capture feedback for retraining if required. 
• Multi-Model Management. MLOps platforms can also help manage a portfolio of multiple models, comparing performance baselines across the models while also enabling techniques like canary rollouts and A/B tests. 

#4 – Build in governance and automation to drive best practice 

To help overcome many of the day-to-day MLOps challenges, work on progressing to automated MLOps processes over time. This includes CI/CD pipelines, leveraging MLOps tools, and embedding a strong governance culture.  

• CI/CD Pipelines. As your maturity grows, automated CI/CD pipelines can build, test, package and deploy models triggered by any numbers of events. This enables speed while reducing the chance of human error. 
• MLOps Tools. To maximize the key benefits of MLOps, utilize purpose-built platforms and tools to automate workflows, centralize artifacts and metadata, and enforce governance policies. AzureMl, Amazon SageMaker, and DataBricks are three of the most popular MLOps tools on the market. 
• MLOps Culture. Success from MLOps comes when everyone is fully bought into the process. Promoting a culture of collaboration, breaking down cross-functional silos, creating Data Science empowerment, and a production mindset ultimately help drive the best MLOps results.  

It’s time to get started with MLOps 

DevOps and MLOps share many similarities, and as Machine Learning technologies continue to grow, it’s time to put as much energy into your Machine Learning as you do the rest of your development.  

MLOps is the best way to get the most from your Machine Learning technologies, providing structure and control that turn your use cases into reality. Get your MLOps framework right, and you’ll reap the benefits of faster execution, less risk, and higher performing data models.  

While you can start your MLOps journey alone, like many things in IT, working with an expert partner often drives the best results. At Inetum, our value proposition is built upon helping you work alongside the best 3rd party suppliers and combining their capabilities with expert technical advice.  

Elevate Your Application Development

Our tailored Application Development services meet your unique business needs. Consult with Marek Czachorowski, Head of Data and AI Solutions, for expert guidance.

Schedule a meeting

– In the United Kingdom, two-thirds of banks use Machine Learning solutions, and solutions based on Artificial Intelligence plot new courses for the development of the banking sector around the world. According to Garner’s report, Machine Learning is one of the main technological trends which are worth paying close attention to in 2020. Like any other technology, Machine Learning brings enormous opportunities, but also challenges. How does ML differ from traditional programming, what threats can it help to eliminate, and how does this technology build new standards in banking? We discussed this with Machine Learning developer Piotr Kubica, who is developing a solution to be used by ING Bank Śląski.

Let’s start with what probably interests financial sector representatives the most. What in your view are the main benefits of using Machine Learning in banking?

– I think that surely the basic goals that all companies strive for are maximizing profits and reducing costs. Machine Learning can help in both cases. The most popular use of ML at the moment is personalization in advertising: the goal is to maximize profits by increasing sales, suggesting products tailored to specific user needs. In addition, Machine Learning helps to minimize costs by automating repeatable processes.

In the context of automation – we often come across the opinion that someday everything will be automated… Will it be possible to automate all processes in the future?

– I think that until we have this general Artificial Intelligence that will be able to do everything (in the way that humanoid robots are depicted in all Science Fiction films) – Machine Learning will simply be based on optimization, mathematics, figures and on how these programs and algorithms are written. At the moment, people still determine the quality – the technology is only as good as the people who have created it. However, this area is constantly evolving and over time we will be able to solve more complex problems. But will we ever be able to solve all of them? I don’t think so.

What are the expectations of the banking and financial sector in terms of Machine Learning? You have mentioned how important cost optimization is.

– I think that the banking sector expects a lot from ML. Currently, all organizations strive to be “data-driven”. For the banking and financial sector you have mentioned, the regulations are also a significant aspect. There is the Polish Financial Supervision Authority, which right now does not say yes to everything. Machine Learning is so new that sometimes the algorithms used are not easily understandable.

In which situations will Machine Learning work?

–For example, ML is great for decisions on granting loans. However, the regulator must first agree to use Machine Learning. Currently, traditional credit risk modeling, traditional assessment, statistics and rule models are used for this purpose. However, Machine Learning is also used in many other areas of banking and finance – for example in the personalization of offers, which we discussed earlier on. It is very important to provide customers with a suitable offer.

So it is mainly the marketing aspect?

– Not only that. I think that currently it is the most developed part, but right now solutions based on text processing, e.g. document handling automation, are also developing strongly. I think that the automation of customer service processes through the use of chatbots and voicebots will also be the future.

How does Artificial Intelligence support banking consultants? Here I mainly mean customer service.

– The bank uses a chatbot solution that allows customers to solve problems by themselves without having to contact a human consultant. It is possible to get answers to some simple questions that often come up. There is no need to read the regulations or the Frequently Asked Questions section. I think it can also work the other way around. Consultants can use this kind of knowledge base when the customer calls the helpline. They do not need to have particularly detailed knowledge as they can find information in a faster way during the conversation with the client, and thus the service becomes more effective.

And when it comes to customers’ User Experience?

– The bot cannot answer all the questions, certainly not non-standard ones. Some people, however, do not realize that they are talking to a bot, because the chatbots’ answers are prepared in such a way and are so good that it’s impossible to tell the difference by asking simple questions.

This is interesting in the context of Turing tests. Many companies have struggled to pass them successfully.

– A year or two ago, the Google AI conference took place. There was quite a popular film on the Internet, in which the bot was making an appointment at the hairdresser’s for a client, a smartphone user. We could see the Google assistant calling and arranging such a visit, knowing the specific plans, and having an insight into the calendar. It turned out great, everyone at the conference applauded. However, now information is emerging that 70 percent of calls are reportedly being tracked or taken over by people anyway. So it’s all controlled or taught during the testing phase. It’s not as perfect as everyone would like.

How does ML support security processes and all processes aimed at preventing data theft or identifying users?

– There are many possibilities when it comes to using Machine Learning for user identification. Each user has their own unique behavior on the site. It’s a behavioral approach that allows others to identify the user. I do not necessarily see the potential in data protection alone, but I do see the potential in user identification – creating a model for each user and checking how a person behaves in different environments using a telephone or computer. Machine Learning would be able to distinguish user data. The question is, how well would such a thing work?

The behavioral biometrics you are talking about is an interesting issue. Biometric banking is also often used. The customer calls the bank to unlock an account and the system can recognize their voice. How does it work?

– This is an interesting issue. We each have a unique fingerprint, iris or voice. Machine Learning is also used to recognize them, to the best of my knowledge. In the event that we want to unlock our account using voice recognition, we must first teach the system the model of our voice, so that later it will be able to ascertain with a high degree of probability that we are the person we claim to be.

We have talked about the benefits and possibilities of ML. What are the biggest requirements and threats in this area?

– There are also some threats, as Machine Learning also has some weaknesses that can be exploited. But here the war on reinforcement will begin again – or even just begin. Just as there are IT safeguards, so too will safeguards against using model gaps exist.

What model gaps do you mean?

– What can prevent you from training a model based on the voice of the customer, which will be able to impersonate him at the bank and recreate his voice based on a few minutes of conversation? IBM has already published research stating that they are able to replicate someone’s voice based on a mere five minutes of voice material. In fact, something similar probably took place in September 2019 – I mean social hacking by means of Artificial Intelligence, where millions were extorted.

How has it come to this?

– Artificial Intelligence learned and copied the voice of a company’s CEO and persuaded the vice-president of another company to transfer some money to another account. This is a kind of arms race, a fight for better and better models. So there is a model that learns to recognize your voice, and then we create a network that learns your voice and impersonates your voice. Subsequently, a model is created that will later distinguish your artificial voice from your real voice.

Machine Learning and data collection also raises questions about whether our data is secure.

– It depends. Machine Learning itself uses data that must be stored somewhere. ML models do not store data, but store certain ways, rules, weights or divisions of decision making. Also, the moment of data storage is important here – this is not my area of expertise, you would have to ask data storage security specialists. In the learning process, however, we use available data that is secured, and later all application development processes already have to comply with these security requirements.

What are the real threats and how do we counteract them?

– There are interesting examples of how you can cheat on the neural network by changing one pixel in the image. There are many possibilities, many things that you need to pay attention to while modeling. But surely a lot of things will arise as this field develops. Because, as you mentioned, this industry is quite new, so there is a lot to discover – not everything is precisely defined, e.g. how Machine Learning processes should be created. There is no single, rigid pattern – and that’s why it’s interesting. As part of the work of a Machine Learning developer, it is very often necessary to sit down and think about how to do a given thing well, how to prevent the leakage of relevant information.

When it comes to the use of Machine Learning in banking – what would you personally consider to be the greatest achievement in this area?

– Theft prevention. This issue often comes up. Some users have negative experiences with this though. They try to pay for something by card and it turns out that their card is blocked. Then there is a phone call from the bank: “Is that really you making a transaction?”. And this is an example of an unsuccessful use of the system.

Bad User Experience. However, in good faith.

– It all happens in good faith to prevent someone from using your card for a nefarious purpose. Previously, rule models were used here. It is about such cases when someone is shopping in Katowice at about 4pm and in Moscow around 6pm. It is impossible to cover such a distance and get around so quickly by plane.

And now these methods are no longer used?

– It seems to me that hybrid models are coming in – which also represent a highly interesting solution. The combination of ML with rule models, where Machine Learning allows us to draw upon some additional cases that people could not define or lacked some vital knowledge.

What does Machine Learning offer in such situations?

– In these cases, Machine Learning can detect additional scenarios that were not taken into account. But it can also limit just the kind of cases that we discussed earlier on – meaning that these rule models can have too many so-called false positives. That is, cases in which the model thought that a theft had occurred, when in practice it had not. It also means a reduction in the number of cases that are incomprehensible to a user who is trying to make a transaction. The purpose of blocking a bank card is to prevent theft – which is certainly not pleasant. By way of comparison, it is certainly “less pleasant” than unlocking the card later.

Thank you for the interview.

DATA ANALYTICS SERVICES

Use data to your advantage

Discover our Data Management End-to-End offer! Find out more!