The importance of optimization cannot be overstated in the context of machine learning and deep learning. Most, if not all, of machine learning involves fitting a model to data, a process that involves optimization.

Optimization is a broad research area with many open challenges. Through interactive tutorials, we intend to cover most of the important concepts in constrained and unconstrained optimization.

Before venturing into this article, it is strongly recommended that you are well familiar with the concepts introduced in our previous modules on Linear Algebra and Calculus for machine Learning.


To understand optimization, we recommend familiarity with the concepts in

Follow the above links to first get acquainted with the corresponding concepts.


To build a strong foundation for advanced material such as machine learning, deep learning, and reinforcement learning, we recommend familiarity with these concepts in optimization. Follow the links to our deep-dive articles to become an expert.

Approaches to unconstrained optimization

Improvements to gradient descent


Optimization is a broad field, much older than machine learning. There are numerous good resources, we list the important ones.

Where to next?

Expand your knowledge of optimization approaches with our detailed interactive articles on this subject.

To brush up on calculus, explore our articles on topics in calculus and multivariate calculus. Or start with strong mathematical foundations.

Already an optimization expert? Check out comprehensive courses on machine learning or deep learning.

Please support us

Help us create more engaging and effective content and keep it free of paywalls and advertisements!

Subscribe for article updates

Stay up to date with new material for free.