An interactive overview of vectors from a geometric perspective

This learning module has many interactive demos. It is easier to work with them on a larger screen. Bookmark and revisit if you are currently on a small screen device.

\(\DeclareMathOperator*{\argmax}{arg\,max} \DeclareMathOperator*{\argmin}{arg\,min} \DeclareMathOperator*{\asterisk}{\ast} \newcommand{\sup}{\text{sup}} \newcommand{\inf}{\text{inf}} \newcommand{\min}{\text{min}\;} \newcommand{\max}{\text{max}\;} \newcommand{\maxunder}[1]{\underset{#1}{\max}} \newcommand{\minunder}[1]{\underset{#1}{\min}} \newcommand{\real}{\mathbb{R}} \newcommand{\natural}{\mathbb{N}} \newcommand{\integer}{\mathbb{Z}} \newcommand{\rational}{\mathbb{Q}} \newcommand{\irrational}{\mathbb{I}} \newcommand{\complex}{\mathbb{C}} \newcommand{\cardinality}[1]{|#1|} \newcommand{\vec}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\star}[1]{#1^*} \newcommand{\inv}[1]{#1^{-1}} \newcommand{\indicator}[1]{\mathcal{I}(#1)} \renewcommand{\BigO}[1]{\mathcal{O}(#1)} \renewcommand{\BigOsymbol}{\mathcal{O}} \renewcommand{\smallo}[1]{\mathcal{o}(#1)} \renewcommand{\smallosymbol}[1]{\mathcal{o}} \newcommand{\set}[1]{\mathbb{#1}} \newcommand{\complement}[1]{#1^c} \newcommand{\powerset}[1]{\mathcal{P}(#1)} \newcommand{\setdiff}{\setminus} \newcommand{\setsymmdiff}{\oplus} \newcommand{\dash}[1]{#1^{'}} \newcommand{\permutation}[2]{{}_{#1} \mathrm{ P }_{#2}} \newcommand{\combination}[2]{{}_{#1} \mathrm{ C }_{#2}} \newcommand{\prob}[1]{P(#1)} \newcommand{\pmf}[1]{P(#1)} \newcommand{\pdf}[1]{p(#1)} \newcommand{\cdf}[1]{F(#1)} \newcommand{\expect}[2]{E_{#1}\left[#2\right]} \newcommand{\entropy}[1]{\mathcal{H}\left[#1\right]} \newcommand{\expe}[1]{\mathrm{e}^{#1}} \newcommand{\textexp}[1]{\text{exp}\left(#1\right)} \def\independent{\perp\!\!\!\perp} \def\notindependent{\not\!\independent} \newcommand{\yhat}{\hat{y}} \newcommand{\vs}{\vec{s}} \newcommand{\vt}{\vec{t}} \newcommand{\vu}{\vec{u}} \newcommand{\vv}{\vec{v}} \newcommand{\vw}{\vec{w}} \newcommand{\vx}{\vec{x}} \newcommand{\vy}{\vec{y}} \newcommand{\vz}{\vec{z}} \newcommand{\va}{\vec{a}} \newcommand{\vb}{\vec{b}} \newcommand{\vc}{\vec{c}} \newcommand{\vd}{\vec{d}} \newcommand{\ve}{\vec{e}} \newcommand{\vg}{\vec{g}} \newcommand{\vh}{\vec{h}} \newcommand{\vi}{\vec{i}} \newcommand{\vk}{\vec{k}} \newcommand{\vo}{\vec{o}} \newcommand{\vp}{\vec{p}} \newcommand{\vq}{\vec{q}} \newcommand{\vr}{\vec{r}} \newcommand{\vs}{\vec{s}} \newcommand{\vmu}{\vec{\mu}} \newcommand{\vsigma}{\vec{\sigma}} \newcommand{\vphi}{\vec{\phi}} \newcommand{\vtau}{\vec{\tau}} \newcommand{\vtheta}{\vec{\theta}} \newcommand{\mA}{\mat{A}} \newcommand{\mB}{\mat{B}} \newcommand{\mC}{\mat{C}} \newcommand{\mD}{\mat{D}} \newcommand{\mE}{\mat{E}} \newcommand{\mH}{\mat{H}} \newcommand{\mK}{\mat{K}} \newcommand{\mP}{\mat{P}} \newcommand{\mQ}{\mat{Q}} \newcommand{\mR}{\mat{R}} \newcommand{\mS}{\mat{S}} \newcommand{\mU}{\mat{U}} \newcommand{\mV}{\mat{V}} \newcommand{\mW}{\mat{W}} \newcommand{\mX}{\mat{X}} \newcommand{\mY}{\mat{Y}} \newcommand{\mZ}{\mat{Z}} \newcommand{\mI}{\mat{I}} \newcommand{\mLambda}{\mat{\Lambda}} \newcommand{\mSigma}{\mat{\Sigma}} \newcommand{\mTheta}{\mat{\theta}} \newcommand{\setsymb}[1]{#1} \newcommand{\sA}{\setsymb{A}} \newcommand{\sB}{\setsymb{B}} \newcommand{\sC}{\setsymb{C}} \newcommand{\sO}{\setsymb{O}} \newcommand{\sP}{\setsymb{P}} \newcommand{\sQ}{\setsymb{Q}} \newcommand{\sH}{\setsymb{H}} \newcommand{\sX}{\setsymb{X}} \newcommand{\sY}{\setsymb{Y}} \newcommand{\norm}[2]{||{#1}||_{#2}} \newcommand{\infnorm}[1]{\norm{#1}{\infty}} \newcommand{\fillinblank}{\text{ }\underline{\text{ ? }}\text{ }} \newcommand{\lbrace}{\left\{} \newcommand{\rbrace}{\right\}} \newcommand{\set}[1]{\lbrace #1 \rbrace} \newcommand{\seq}[1]{\left( #1 \right)} \newcommand{\ndim}{N} \newcommand{\ndimsmall}{n} \newcommand{\dataset}{\mathbb{D}} \newcommand{\ndata}{D} \newcommand{\ndatasmall}{d} \newcommand{\labeledset}{\mathbb{L}} \newcommand{\nlabeled}{L} \newcommand{\nlabeledsmall}{l} \newcommand{\unlabeledset}{\mathbb{U}} \newcommand{\nunlabeled}{U} \newcommand{\nunlabeledsmall}{u} \newcommand{\nclass}{M} \newcommand{\nclasssmall}{m} \newcommand{\loss}{\mathcal{L}} \newcommand{\sign}{\text{sign}} \newcommand{\Gauss}{\mathcal{N}} \newcommand{\hadamard}{\circ} \newcommand{\doh}[2]{\frac{\partial #1}{\partial #2}} \newcommand{\dox}[1]{\doh{#1}{x}} \newcommand{\doy}[1]{\doh{#1}{y}} \newcommand{\doxx}[1]{\doh{#1}{x^2}} \newcommand{\doyy}[1]{\doh{#1}{y^2}} \newcommand{\doxy}[1]{\frac{\partial #1}{\partial x \partial y}} \newcommand{\doyx}[1]{\frac{\partial #1}{\partial y \partial x}} \newcommand{\qed}{\tag*{$\blacksquare$}}\)

        Vector geometry
        Linear Algebra
      

In this article, we will build some intuition about the geometry of vectors, their properties, and the operations on them.

Prerequisites

To understand vectors, we recommend familiarity with the concepts in

Follow the above links to first get acquainted with the corresponding concepts.

Vector: a geometric perspective

In the following interactive, we show a $2$-dimensional vector. The vector is an arrow, with the tail at the origin, and the head at the value of the vector. In this sense, the vector is an ordered tuple, such that the X-coordinate is the first coordinate of the vector and the Y-coordinate is the second.

A similar geometric interpretation can be imagined for an $ n $-dimensional vector, in an $n$-dimensional space.

Drag the cricle to change the vector

Vector magnitude and direction

A vector has magnitude and direction. This should be intuitive from the geometric perspective you saw earlier.

The magnitude of a vector is measured in terms of its norm. The $ p $-norm, for $ p \ge 1 $, provides a generalized way of computing any norm of a vector.

$$ ||\va||_p = \left( \sum_{i=1}^n |a_i|^p \right)^\frac{1}{p} $$

Three norms, derived from $p$-norm, are popular in machine learning.

$L_1$ or Manhattan or Taxi-cab norm

$$ ||\va||_1 = \sum_{i=1}^n |a_i| $$

$L_2$ or Euclidean norm

$$ ||\va||_2 = \left( \sum_{i=1}^n a_i^2 \right)^\frac{1}{2} $$

$L_\infty$ or max norm

$$ ||\va||_\infty = \max_{i = 1}^n |a_i| $$

A vector with unit Euclidean norm, $ || \va ||_2 = 1 $, is known as a unit vector.

Any vector can be normalized to a unit vector by dividing it with its norm, for example, the Euclidean norm.

$$ \vec{\va} = \frac{\va}{||\va||_2} $$

$\vec{\va} $ has the same direction as that of the original vector, but has unit-magnitude.

Vector norm: demo

Next, we present an interactive demonstration of vector norms.

Stretch and rotate the vector $ \va $. Note the change in the various norms to build an intuition about vector magnitudes and direction. Also, use the slider to change the norm type to get a shape of the unit ball, the region that is traversed by the unit vector of a given norm type.

Understanding the shapes of these regions is crucial in choosing appropriate norm types in machine learning.

Drag the cricle on the arrowhead to change the vector

Stretching or shrinking a vector

The magnitude of the vector can be stretched or shrunk. This is achieved by multiplication with a scalar.

$$ \vb = \alpha \va $$

The result, $ \vb $, is a vector with the same number of entries as $ \va $. It is easy to prove that the magnitude of $ \vb $ and that of $ \va $ are related.

$$ || \vb ||_2 = |\alpha| || \va ||_2 $$

Note that the scalar multiplication can also change direction.

If $ 0 < \alpha < 1 $, then $ || \vb ||_2 < || \va ||_2 $. No change in direction.
If $ \alpha > 1 $, then $ || \vb ||_2 > || \va ||_2 $. No change in direction.
If $ -1 < \alpha < 0 $, then $ || \vb ||_2 < || \va ||_2 $. Also, $ \vec{\vb} $ points in the opposite direction of $ \vec{\va} $
If $ \alpha < -1 $, then $ || \vb ||_2 > || \va ||_2 $. Also, $ \vec{\vb} $ points in the opposite direction of $ \vec{\va} $

To build intuition about scalar multiplication, try out our interactive demo presented next. Vary the value of the scalar multiplier $ c $ by adjusting the slider. Note the difference in the vector $ \va $ while you do that.

Vector stretching and shrinking: demo

Next, we present an interactive demonstration of scalar multiplication with a vector. Stretch and rotate the vector $ \va $ by moving the arrowhead or change the scalar multiplier $ c $ using the slider. Note the change in their product $ c\va $.

Understanding these results intuitively is crucial to building a geometric perspective scalar multiplication of vectors.

Drag the cricle on the arrowhead to change the vector

Rotating a vector

We saw that vectors have direction and that they can be shrunk, stretched, or flipped. In linear combination of vectors we will study how vectors can be rotated!

Where to next?

Now that you understand vector geometry, it is time to build expertise in other topics in linear algebra.

Please support us

Help us create more engaging and effective content and keep it free of paywalls and advertisements!

Please donate

Subscribe for article updates

Stay up to date with new material for free.