Types of tasks in machine learning

Introduction

Machine learning is a broad field with a variety of approaches to addressing a gamut of tasks. In this article, we will describe some of the commonly addressed tasks using machine learning. We will also comment and point to suitable approaches for handling such tasks.

Classification

Classification is the task of assigning categories (or classes) to given instances automatically. The machine learning model that has been trained to achieve such a goal is known as a classifier. Classification falls in the realm of supervised learning — the sub-field of machine learning that enables models to be trained by observing labeled or supervised examples. For example, to learn a classifier to identify spam emails, each supervised example will be a tuple consisting of the email information (text, subject, from, to) and its category (spam or no spam).

Depending on the number of categories and their relationships, classification problems fall into several types.

• Binary classification: An instance must belong to exactly one among two categories. The classifier itself is known as a binary classifier.
• Multi-class classification: An instance must belong to exactly one among many (more than two) categories. In a multi-class scenario, the categories are mutually exclusive.
• Multi-labeled classification: An instance may simultaneously belong to more than one category from among several categories. Thus, in a multi-labeled set up, the categories are not mutually exclusive.

Mathematically, we can express the classification problem as follows: If \( \vx \) denotes an \( \ndim\)-dimensional input instance, then the goal of classification is to assign \( \vx \) to the appropriate category (or categories, in the case of multi-labeled) from among \( \nclass \) categories \( \set{C_1, \ldots, C_\nclass} \), where \( \nclass \ge 2 \).

The classifier is trained over a collection of labeled observations provided as tuples \( (\vx_i,y_i) \) containing the instance vector \( \vx_i \) and the true target variable \( y_i \in \set{C_1,\ldots,C_\nclass}\). This collection of \( \nlabeled \) labeled observations is known as the labeled training set, or simply the training set, \( \labeledset = \set{(\vx_1,y_1), \ldots (\vx_\nlabeled,y_\nlabeled)} \).

Below, we list some of the most popular approaches to classification. Follow the corresponding links to study the classifiers in further detail.

• Logistic regression (binary)
• Perceptron (binary)
• Support vector machine (binary)
• Decision trees
• Linear discriminant analysis
• Naive Bayes
• Nearest neighbors classifier
• Random forest
• Tree-boosting
• AdaBoost

The binary classifiers in the above list can be adapted to support multi-class or multi-labeled classification scenarios through the one-vs-one or one-vs-rest strategies.

Clustering

Clustering involves the assignment of input instances into groups or clusters of similar instances. For example, we may wish to automatically group news items coming from disparate sources into clusters of related news to be summarized by a single headline. Clustering is a form of unsupervised learning scenario, one that does not involve the use of prior supervision or labels about the assignment of individual instances to various groups.

Mathematically, we can express the clustering problem as follows: Consider observations represented as vectors, for example \( \vx \in \real^\ndim \) — vectors consisting of \( \ndim \) features, \(\vx = [x_1, x_2, \ldots, x_\ndim] \). A collection of such observations is provided in the unlabeled set \( \unlabeledset = \set{\vx_1,\ldots,\vx_\nunlabeled} \). The goal of clustering is to automatically infer the groups of examples \( G_1, \ldots, G_\nclass \) such that the examples belong to a single group, say \( G_i \), are similar in some sense. The groups \( G_1, \ldots, G_\nclass \) are not pre-defined. They are automatically discovered as part of the clustering process.

Below, we list some of the most popular approaches to clustering. Follow the corresponding links to study the specifics of a clustering algorithm in further detail.

• Gaussian mixture models
• K-means

Dimensionality reduction

Dimensionality reduction, as the name implies, involves transforming an multivariate input instance to an output instance with fewer dimensions than the input, while retaining task-dependent relevant information in the instance. For example, we may wish to reduce a 10-dimensional dataset to a 2-dimensional dataset for easy visualization as a scatter plot, while maybe retaining the natural groupings among the input instances, even in the two-dimensional space. Dimensionality reduction may involve an unsupervised or a supervised learning strategy, depending on whether the reduced dimensions are arrived at by being informed with some categorical labels.

Mathematically, we can express the dimensionality reduction problem as follows: Consider observations represented as vectors, for example \( \vx \in \real^\ndim \) — vectors consisting of \( \ndim \) features, \(\vx = [x_1, x_2, \ldots, x_\ndim] \). In dimensionality reduction, we wish to transform these into output vectors with \( \nclass \) dimensions, \( \vy \in \real^\nclass \), such that \( \nclass \ll \ndim \).

Below, we list some of the most popular approaches to dimensionality reduction. Follow the corresponding links to study the specifics of a dimensionality reduction algorithm in further detail.

• Principal component analysis
• Gaussian mixture models
• Autoencoder
• t-SNE

In addition to these, the approaches to clustering, that we saw earlier, can also be considered as dimensionality reduction approaches where the examples are reduced to a single dimension!