An interactive explanation of linear combination of vectors

This learning module has many interactive demos. It is easier to work with them on a larger screen. Bookmark and revisit if you are currently on a small screen device.

\(\DeclareMathOperator*{\argmax}{arg\,max} \DeclareMathOperator*{\argmin}{arg\,min} \DeclareMathOperator*{\asterisk}{\ast} \newcommand{\sup}{\text{sup}} \newcommand{\inf}{\text{inf}} \newcommand{\min}{\text{min}\;} \newcommand{\max}{\text{max}\;} \newcommand{\maxunder}[1]{\underset{#1}{\max}} \newcommand{\minunder}[1]{\underset{#1}{\min}} \newcommand{\real}{\mathbb{R}} \newcommand{\natural}{\mathbb{N}} \newcommand{\integer}{\mathbb{Z}} \newcommand{\rational}{\mathbb{Q}} \newcommand{\irrational}{\mathbb{I}} \newcommand{\complex}{\mathbb{C}} \newcommand{\cardinality}[1]{|#1|} \newcommand{\vec}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\star}[1]{#1^*} \newcommand{\inv}[1]{#1^{-1}} \newcommand{\indicator}[1]{\mathcal{I}(#1)} \renewcommand{\BigO}[1]{\mathcal{O}(#1)} \renewcommand{\BigOsymbol}{\mathcal{O}} \renewcommand{\smallo}[1]{\mathcal{o}(#1)} \renewcommand{\smallosymbol}[1]{\mathcal{o}} \newcommand{\set}[1]{\mathbb{#1}} \newcommand{\complement}[1]{#1^c} \newcommand{\powerset}[1]{\mathcal{P}(#1)} \newcommand{\setdiff}{\setminus} \newcommand{\setsymmdiff}{\oplus} \newcommand{\dash}[1]{#1^{'}} \newcommand{\permutation}[2]{{}_{#1} \mathrm{ P }_{#2}} \newcommand{\combination}[2]{{}_{#1} \mathrm{ C }_{#2}} \newcommand{\prob}[1]{P(#1)} \newcommand{\pmf}[1]{P(#1)} \newcommand{\pdf}[1]{p(#1)} \newcommand{\cdf}[1]{F(#1)} \newcommand{\expect}[2]{E_{#1}\left[#2\right]} \newcommand{\entropy}[1]{\mathcal{H}\left[#1\right]} \newcommand{\expe}[1]{\mathrm{e}^{#1}} \newcommand{\textexp}[1]{\text{exp}\left(#1\right)} \def\independent{\perp\!\!\!\perp} \def\notindependent{\not\!\independent} \newcommand{\yhat}{\hat{y}} \newcommand{\vs}{\vec{s}} \newcommand{\vt}{\vec{t}} \newcommand{\vu}{\vec{u}} \newcommand{\vv}{\vec{v}} \newcommand{\vw}{\vec{w}} \newcommand{\vx}{\vec{x}} \newcommand{\vy}{\vec{y}} \newcommand{\vz}{\vec{z}} \newcommand{\va}{\vec{a}} \newcommand{\vb}{\vec{b}} \newcommand{\vc}{\vec{c}} \newcommand{\vd}{\vec{d}} \newcommand{\ve}{\vec{e}} \newcommand{\vg}{\vec{g}} \newcommand{\vh}{\vec{h}} \newcommand{\vi}{\vec{i}} \newcommand{\vk}{\vec{k}} \newcommand{\vo}{\vec{o}} \newcommand{\vp}{\vec{p}} \newcommand{\vq}{\vec{q}} \newcommand{\vr}{\vec{r}} \newcommand{\vs}{\vec{s}} \newcommand{\vmu}{\vec{\mu}} \newcommand{\vsigma}{\vec{\sigma}} \newcommand{\vphi}{\vec{\phi}} \newcommand{\vtau}{\vec{\tau}} \newcommand{\vtheta}{\vec{\theta}} \newcommand{\mA}{\mat{A}} \newcommand{\mB}{\mat{B}} \newcommand{\mC}{\mat{C}} \newcommand{\mD}{\mat{D}} \newcommand{\mE}{\mat{E}} \newcommand{\mH}{\mat{H}} \newcommand{\mK}{\mat{K}} \newcommand{\mP}{\mat{P}} \newcommand{\mQ}{\mat{Q}} \newcommand{\mR}{\mat{R}} \newcommand{\mS}{\mat{S}} \newcommand{\mU}{\mat{U}} \newcommand{\mV}{\mat{V}} \newcommand{\mW}{\mat{W}} \newcommand{\mX}{\mat{X}} \newcommand{\mY}{\mat{Y}} \newcommand{\mZ}{\mat{Z}} \newcommand{\mI}{\mat{I}} \newcommand{\mLambda}{\mat{\Lambda}} \newcommand{\mSigma}{\mat{\Sigma}} \newcommand{\mTheta}{\mat{\theta}} \newcommand{\setsymb}[1]{#1} \newcommand{\sA}{\setsymb{A}} \newcommand{\sB}{\setsymb{B}} \newcommand{\sC}{\setsymb{C}} \newcommand{\sO}{\setsymb{O}} \newcommand{\sP}{\setsymb{P}} \newcommand{\sQ}{\setsymb{Q}} \newcommand{\sH}{\setsymb{H}} \newcommand{\sX}{\setsymb{X}} \newcommand{\sY}{\setsymb{Y}} \newcommand{\norm}[2]{||{#1}||_{#2}} \newcommand{\infnorm}[1]{\norm{#1}{\infty}} \newcommand{\fillinblank}{\text{ }\underline{\text{ ? }}\text{ }} \newcommand{\lbrace}{\left\{} \newcommand{\rbrace}{\right\}} \newcommand{\set}[1]{\lbrace #1 \rbrace} \newcommand{\seq}[1]{\left( #1 \right)} \newcommand{\ndim}{N} \newcommand{\ndimsmall}{n} \newcommand{\dataset}{\mathbb{D}} \newcommand{\ndata}{D} \newcommand{\ndatasmall}{d} \newcommand{\labeledset}{\mathbb{L}} \newcommand{\nlabeled}{L} \newcommand{\nlabeledsmall}{l} \newcommand{\unlabeledset}{\mathbb{U}} \newcommand{\nunlabeled}{U} \newcommand{\nunlabeledsmall}{u} \newcommand{\nclass}{M} \newcommand{\nclasssmall}{m} \newcommand{\loss}{\mathcal{L}} \newcommand{\sign}{\text{sign}} \newcommand{\Gauss}{\mathcal{N}} \newcommand{\hadamard}{\circ} \newcommand{\doh}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\dox}[1]{\doh{#1}{x}} \newcommand{\doy}[1]{\doh{#1}{y}} \newcommand{\doxx}[1]{\doh{#1}{x^2}} \newcommand{\doyy}[1]{\doh{#1}{y^2}} \newcommand{\doxy}[1]{\frac{\partial #1}{\partial x \partial y}} \newcommand{\doyx}[1]{\frac{\partial #1}{\partial y \partial x}} \newcommand{\qed}{\tag*{$\blacksquare$}}\)

        Linear combination of vectors
        Linear Algebra
      

Introduction

In this article, we will study vectors, their properties, and the operations on them. In the process, we will build an intuition to the various operations from a geometric perspective.

Prerequisites

To understand linear combination of vectors, we recommend familiarity with the concepts in

Follow the above links to first get acquainted with the corresponding concepts.

Rotating a vector

If a vector can be stretched or shrunk, so can it be rotated.

Here's a thought experiment.

Suppose an object $ \va $ is moving in a certain direction. Another object $ \vb $ strikes it, moving in another direction, not necessarily head-on.

What happens? The object changes its original trajectory.

The new trajectory and the new speed is a result of two conditions prior to the strike.

The speed of the object $ \va $ relative to that of the object $ \vb $.
The direction of the object $ \va $ with respect to that of the object $ \vb $.

So is the case with deflecting or rotating a vector. A vector $ \va $ can be rotated by introducing the effects of another vector $ \vb $.

$$ \vc = \alpha \va + \beta \vb $$

This operation of scaling each vector and summing them up is known as a linear combination. A linear combination of $ m $ vectors, $ \va_i \in \real^n, \forall i \in \{1,\ldots,m\} $ is defined as

$$ \vc = \sum_{i=1}^m \alpha_i \va_i $$

So, the result of a linear combination of a set of vectors is a new vector with the same number of entries, but different direction and magnitude.

If the $ \alpha_i $ are constrained to be positive and sum to one, $ \sum_i \alpha_i = 1, \forall \alpha_i \ge 0 $, then, the linear combination is more specifically known as a convex combination.

Vector linear combination: demo

Build intuition about the linear combination operation with our interactive demo, presented next. Use the sliders to modify the scalars $ c $ and $ d $ in the operation. Drag the arrowheads to change the vectors $ \va $ and $ \vb $. Note the impact on the result of the linear combination operation $ c\va + d\vb $.

Drag the cricle to change the vectors

Span, basis, and null-space

Now that you understand linear-combination, it is time to understand the span of a set of vectors.

The set of all vectors obtainable by a linear combination of a given set of vectors is known as the span of those vectors.

$$ \text{span}(\{\va_1,\ldots,\va_m\}) = \left\{ \vc \in \real^n : \vc = \sum_{i=1}^m \alpha_i \va_i, \forall \alpha_i \in \real \right\} $$

We saw in an earlier interactive demo that multiplication by a scalar stretches, shrinks, or flips a vector. So, it is easy to imagine that a single vector is unlikely to span a 2-dimensional space such as this screen.

From the linear-combination demo, you must have noticed that by just varying the scalars it is possible to move the output vector a whole 360 degrees. Try it if you didn't already.

Thus, you need at least 2 vectors to span the whole of a 2-dimensional space. Such 2 vectors are known as the basis set of that space.

From the same interactive demo, you will note that any two input vectors can be a basis for the 2-dimensional space. Unless the two input vectors are merely scaled versions of each other. Try it in the linear combination demo. Crucial point.

What if we introduced a third input vector in a 2-dimensional space. Well, no harm, but not benefit either. We are not covering any new ground. So, the basis of a space is defined as the minimal set of vectors that spans that space.

In the interactive demos that we have been using so far, the span of the two axes is the entire 2-dimensional plane of the screen. So, the two axes are known as the basis of this span or space.

Where to next?

Now that you understand linear combination of vectors, it is time to build expertise in other topics in linear algebra.

Please support us

Help us create more engaging and effective content and keep it free of paywalls and advertisements!

Please donate

Subscribe for article updates

Stay up to date with new material for free.