I’m Kesha Williams, an AWS Machine Learning Hero and Alexa Champion, and this is Kesha's Korner, where we learn about artificial intelligence and machine learning on AWS. Learn with me as we transform your engineering skills and future-proof your career!
In the last video, I introduced machine learning and the concept of "the model". As a reminder, the mathematical model is nothing more than a collection of trends and patterns found in data. This model can be queried (or consulted) by giving it data it hasn't seen before and the model then provides a prediction. So today, we are going to talk about the process to create this model.
Building The Model
In the machine learning space, most of us didn’t start out as data scientists or machine learning engineers, we come from other disciplines. For example, I've done software engineering for the last 25 years. So maybe you're a developer or a software engineer that’s used to building apps, well the process to "build" this model is a bit different from traditional software development.
Let’s take the case of a footwear classification system, where we take shoes and classify them in groups like boots, sandals, or slippers. When we approach solving this problem from a traditional software development standpoint, we'll end up with something like this. You'll have a user, using a web application that you've developed that allows them to classify the data. For example, there'd be a slick front-end user interface that allows them to drag and drop shoes into the different categories and behind the scenes on the back-end, you'd have an API that stores and/or updates the data in a database.
You would probably use the Agile methodology to deliver functionality to your users in sprints. Now, solving the problem in this manner is a bit inefficient, and a machine can do this easier, faster, and better. When you transition this footwear classification problem to machine learning, how you build a "system" is different.
Kesha also covers a range of topics, like Alexa Skill Builder, to help you
Let's look at the machine learning lifecycle.
The first step is getting the data from a reputable source (and you need lots of data).
The second step is putting the data in a format that a machine can learn from. This step takes the most time and is the most important part. Just remember, "garbage in, garbage out".
The third step is to feed the data to the machine so that it can find trends and patterns to produce the model.
The fourth step is evaluating the model's performance. How well does the model perform? How accurate is it when answering questions?
The fifth step is "productionizing" the model when we place it in a production environment so that other systems can use it to generate predictions.
The process to produce this model, unlike traditional software development, comes with a lot of experimentation. For example, during training and evaluation, you're producing multiple models and testing them. During this experimentation, you're changing settings and configuration parameters and testing the model with different data sets until you finally settle on the best performing model out of the bunch.
Historically, you've had to use different systems for different phases of the machine learning lifecycle; but AWS simplifies this process by allowing you to stay in one system for the entire lifecycle. This makes the whole process easier to manage. I'll go more in-depth on that topic in the future.
Today, we looked at the process to produce the model. I hope you'll join me in our next edition of Kesha's Korner where we will dive deep on data.
Ready to learn more? Dig deeper into Machine Learning with A Cloud Guru’s AWS Machine Learning Specialty Certification course.