Multi-class Classification vs Multi-label Classification vs Multi-output Classification

Let's consider the popular dataset MNIST Digits. It has pictures of the same size of hand-written numbers from 0 to 9. We will use it to illustrate those terms.

Multi-class classification is the problem of classifying instances into one of three or more classes.

Considering the dataset we have, if we want to assign each picture to its correct label (The number written in the picture) then this is considered as Multi-class classification as we have a an instance of data to be assigned to one of 10 classes.

Multi-label Classification Such a classification system outputs multiple binary tags on two or more classes.

Consider a face-recognition classifier: You need to know which of Alice, Bob, and Charlie is in the picture. It should attach one tag per person it recognizes. Say the classifier has been trained to recognize their three faces. Then when the classifier is shown a picture of Alice and Charlie, it should output [True, False, True] (meaning “Alice yes, Bob no, Charlie yes”). 

In our MNIST dataset, what if you want to know two things about each picture:

1. Does the picture has number 7 or more?
2. Does the picture has an odd number?

So, you would get a boolean output for each class.

Note: Sometimes you may need to consider the output of the first class while predicting the second class. In other words, notice that the first class outputs  (7, 8, or 9) which is twice more likely to be odd than even. So, the second classifier might need to consider that pattern while predicting its label, but how?

The models can be organized in a chain: when a model makes a prediction, it uses the input features plus all the predictions of the models that come before it in the chain. The "ChainClassifier" by Scikit-Learn does that!

Multi-output Classification It is a generalization of multilabel classification where each label can be multiclass.

To illustrate this, let’s consider building a system that removes noise from images. It will take as input a noisy digit image, and it will output a clean digit image, represented as an array of pixel intensities, just like the MNIST images. Notice that the classifier’s output is multilabel (one label per pixel) and each label can have multiple values (pixel intensity ranges from 0 to 255). It is thus an example of a multioutput classification system.