23/09/2021
What Is Argmax in Machine Learning?
Argmax is a mathematical function that you may encounter in applied machine learning.
What Is Argmax?
Argmax is a mathematical function.
It is typically applied to another function that takes an argument. For example, given a function g() that takes the argument x, the argmax operation of that function would be described as follows:
result = argmax(g(x))
The argmax function returns the argument or arguments (arg) for the target function that returns the maximum (max) value from the target function.
Consider the example where g(x) is calculated as the square of the x value and the domain or extent of input values (x) is limited to integers from 1 to 5:
g(1) = 1^2 = 1
g(2) = 2^2 = 4
g(3) = 3^2 = 9
g(4) = 4^2 = 16
g(5) = 5^2 = 25
We can intuitively see that the argmax for the function g(x) is 5.
That is, the argument (x) to the target function g() that results in the largest value from the target function (25) is 5. Argmax provides a shorthand for specifying this argument in an abstract way without knowing what the value might be in a specific case.
argmax(g(x)) = 5
Note that this is not the max() of the values returned from function. This would be 25.
It is also not the max() of the arguments, although in this case the argmax and max of the arguments is the same, e.g. 5. The argmax() is 5 because g returns the largest value (25) when 5 is provided, not because 5 is the largest argument.
Typically, “argmax” is written as two separate words, e.g. “arg max“. For example:
result = arg max(g(x))
It is also common to use the arg max function as an operation without brackets surrounding the target function. This is often how you will see the operation written and used in a research paper or textbook. For example:
result = arg max g(x)
You can also use a similar operation to find the arguments to the target function that result in the minimum value from the target function, called argmin or “arg min.”
How Is Argmax Used in Machine Learning?
The argmax function is used throughout the field of mathematics and machine learning.
Nevertheless, there are specific situations where you will see argmax used in applied machine learning and may need to implement it yourself.
The most common situation for using argmax that you will encounter in applied machine learning is in finding the index of an array that results in the largest value.
Recall that an array is a list or vector of numbers.
It is common for multi-class classification models to predict a vector of probabilities (or probability-like values), with one probability for each class label. The probabilities represent the likelihood that a sample belongs to each of the class labels.
The predicted probabilities are ordered such that the predicted probability at index 0 belongs to the first class, the predicted probability at index 1 belongs to the second class, and so on.
Often, a single class label prediction is required from a set of predicted probabilities for a multi-class classification problem.
This conversion from a vector of predicted probabilities to a class label is most often described using the argmax operation and most often implemented using the argmax function.
Let’s make this concrete with an example.
Consider a multi-class classification problem with three classes: “red“, “blue,” and “green.” The class labels are mapped to integer values for modeling, as follows:
red = 0
blue = 1
green = 2
Each class label integer values maps to an index of a 3-element vector that may be predicted by a model specifying the likelihood that an example belongs to each class.
Consider a model has made one prediction for an input sample and predicted the following vector of probabilities:
yhat = [0.4, 0.5, 0.1]
We can see that the example has a 40 percent probability of belonging to red, a 50 percent probability of belonging to blue, and a 10 percent probability of belonging to green.
We can apply the argmax function to the vector of probabilities. The vector is the function, the output of the function is the probabilities, and the input to the function is a vector element index or an array index.
arg max yhat
We can intuitively see that in this case, the argmax of the vector of predicted probabilities (yhat) is 1, as the probability at array index 1 is the largest value.
Note that this is not the max() of the probabilities, which would be 0.5. Also note that this is not the max of the arguments, which would be 2. Instead it is the argument that results in the maximum value, e.g. 1 that results in 0.5.
arg max yhat = 1
We can then map this integer value back to a class label, which would be “blue.”
arg max yhat = “blue”
How to Implement Argmax in Python
The argmax function can be implemented in Python for a given vector of numbers.
Argmax from Scratch
First, we can define a function called argmax() that enumerates a provided vector and returns the index with the largest value.
