An introduction to four methods that combine mathematical optimization and machine learning

Mathematical Optimization and Machine Learning

Mathematical optimization (or mathematical programming) is a powerful decision-making tool. By setting goals and specifying constraints and variables, mathematical optimization can help make the best decisions under current real-world circumstances. It has proven its worth in many different industries including aviation, logistics, power and finance.

Machine learning is a branch of artificial intelligence. Computers can recognize patterns in data and learn to predict the future. They can perform clustering, detect anomalies, or generate new music or images. The three types of machine learning (supervised, unsupervised, and reinforcement learning) can be applied to all industries, such as healthcare and even art. Machine learning models are all about probability, and predicting the probability of something happening

Both approaches have advantages and disadvantages. When the data changes too much, the machine learning model becomes useless and the model needs to be retrained or rebuilt from scratch. Mathematical optimization requires good mathematical description, and it cannot handle unstructured data like machine learning. Additionally, if the problem becomes too large, you may need a commercial fix to fix the problem, which can be quite expensive.

Some problems are better suited to machine learning, while other problems are better suited to mathematical optimization. Machine learning should be used when you want to discover patterns in data, find similar data samples, or predict the weather. If you want to create a schedule, find the optimal location of a facility or minimize the cost of a problem, mathematical optimization is a better choice.

How to combine mathematical optimization and machine learning?

It is useful to combine mathematical optimization and machine learning. They have different advantages and disadvantages, and some problems are too complex to use just one of the two. They can complement each other. Here are four methods and practical examples of how to combine them.

1. Using Machine Learning Predictions as Constraints in Optimization Models

First, you use machine learning to make predictions that are used as input to the optimization problem. You can use the output of a machine learning model to set constraints.

Example: Use machine learning to predict viewership, using them as input to create an optimized optimal plan

Suppose you are a data scientist for a platform and you sell ad space to other companies. Advertisers buy airtime, and the platform predicts based on experience how many people will see the advertiser's ads. As a data scientist, you want to use your advertising space in the best possible way. First, use machine learning to predict viewing data based on historical data. Then, create an optimization model that uses viewing data as input. You optimize your plan by watching the data. By doing this, you can maximize the platform’s profits.

2. Use optimization decisions as training features in machine learning models

Compared to method 1, this is the opposite way: first the optimization model makes decisions, and the decisions are used as machine learning features in the model. In practice, this approach is less common since most decisions (MO) follow predictions (ML). This approach may be useful in certain projects.

Example: Using transportation decisions in machine learning models

Mathematical optimization has a wide range of applications in logistics. If you use optimization to decide how much supply needs to be shipped from the production plant to the market, it can save a lot of time, money, and resources. After you obtain these results, you can use them in machine learning problems, such as predicting how many employees each factory will need on a specific day.

3. Utilize machine learning outputs to scope mathematical optimization models

In addition to using machine learning outputs directly in optimization problems, you can also choose to treat them independently combined. You can use them in the same project, but not in the same process. You can use machine learning output to simplify mathematical optimization problems: You can use machine learning to determine the scope of an optimization model. An added benefit here is that the optimized model can be solved in less time.

Example: Using Predictive Maintenance and Clustering to Determine the Scope of Routing Issues

In this example, we look at a company that repairs power boxes. They have a limited number of repairmen and want to use them in the best possible way. First, you can use predictive maintenance (machine learning) to decide which power boxes have the highest risk of failure. Then, clustering (machine learning) is used to cluster high-risk power boxes. Clustering occurs because you want a group of power boxes to be close to each other. You can choose a number of clusters equal to the number of repairmen available. Finally, through mathematical optimization, you can create the best route between the power boxes for each cluster, one route for each repairman.

4. Use optimization to solve machine learning problems

You can use optimization to find an optimal set of parameters for a machine learning problem. Machine learning and mathematical optimization are closely coupled here, since optimization is used in machine learning problems. The following example shows that Mixed Integer Programming (MIP) has proven its worth in solving a classic research problem.

Example: Solving the optimal subset selection problem in linear regression

When building a regression model, removing irrelevant features will make the model easier to interpret and less likely to overfit the data. It is difficult to find the optimal subset of features, which is called the optimal subset selection problem. Mixed integer programming (mathematical optimization) has seen significant improvements in speed over the past few decades, which makes it useful to test it on existing problems.

Finally

I hope this article inspired you to try an interesting combination of mathematical optimization and machine learning! They are suitable for different types of problems and can complement each other. Besides the obvious way (using the output of one as the input of the other), there are other ways to combine them. You can loosely combine them, as in the third approach, using machine learning to scope the optimization problem. Or you can integrate them tightly to solve a research problem, as in the last example.

The above is the detailed content of An introduction to four methods that combine mathematical optimization and machine learning. For more information, please follow other related articles on the PHP Chinese website!