Why Does Scikit-learn's F1-Score Produce an 'UndefinedMetricWarning'?

UndefinedMetricWarning: F-Score Error

When calculating F-scores with scikit-learn's metrics.f1_score, users may encounter the warning:

"UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples."

Understanding the Warning

This warning arises when some labels in the true labels (y_test) do not appear in the predicted labels (y_pred). In such cases, the F-score for these unpredicted labels cannot be calculated and is assumed to be 0.0.

Example

Consider the following example where label '2' is absent in the predictions:

y_test = [1, 10, 35, 9, 7, 29, 26, 3, 8, 23, 39, 11, 20, 2, 5, 23, 28,
       30, 32, 18, 5, 34, 4, 25, 12, 24, 13, 21, 38, 19, 33, 33, 16, 20,
       18, 27, 39, 20, 37, 17, 31, 29, 36, 7, 6, 24, 37, 22, 30, 0, 22,
       11, 35, 30, 31, 14, 32, 21, 34, 38, 5, 11, 10, 6, 1, 14, 12, 36,
       25, 8, 30, 3, 12, 7, 4, 10, 15, 12, 34, 25, 26, 29, 14, 37, 23,
       12, 19, 19, 3, 2, 31, 30, 11, 2, 24, 19, 27, 22, 13, 6, 18, 20,
        6, 34, 33, 2, 37, 17, 30, 24, 2, 36, 9, 36, 19, 33, 35, 0, 4,
        1]

y_pred = [1, 10, 35, 7, 7, 29, 26, 3, 8, 23, 39, 11, 20, 4, 5, 23, 28,
       30, 32, 18, 5, 39, 4, 25, 0, 24, 13, 21, 38, 19, 33, 33, 16, 20,
       18, 27, 39, 20, 37, 17, 31, 29, 36, 7, 6, 24, 37, 22, 30, 0, 22,
       11, 35, 30, 31, 14, 32, 21, 34, 38, 5, 11, 10, 6, 1, 14, 30, 36,
       25, 8, 30, 3, 12, 7, 4, 10, 15, 12, 4, 22, 26, 29, 14, 37, 23,
       12, 19, 19, 3, 25, 31, 30, 11, 25, 24, 19, 27, 22, 13, 6, 18, 20,
        6, 39, 33, 9, 37, 17, 30, 24, 9, 36, 39, 36, 19, 33, 35, 0, 4,
        1]

print(metrics.f1_score(y_test, y_pred, average='weighted'))

This code will produce the warning.

Why Only Sometimes?

The warning appears only the first time F-score is calculated because most Python environments show specific warnings only once. However, this behavior can be changed using warnings.filterwarnings('always').

How to Avoid the Warning

To avoid seeing the warning, you can either set warnings.filterwarnings('ignore') before importing scikit-learn or explicitly specify the labels you are interested in when calculating F-score, as follows:

# Ignore warnings
warnings.filterwarnings('ignore')
metrics.f1_score(y_test, y_pred, average='weighted')

# Explicitly specify labels
unique_labels = np.unique(y_pred)
metrics.f1_score(y_test, y_pred, average='weighted', labels=unique_labels)

The above is the detailed content of Why Does Scikit-learn's F1-Score Produce an 'UndefinedMetricWarning'?. For more information, please follow other related articles on the PHP Chinese website!