Awesome Optimization Courses

Welcome to the "Awesome Optimization" repository! This repository contains a curated list of (mostly) free and open educational resources for mathematical optimization.

This list tries to cover vast topics in math. opt. i.e. discrete and combinatorial optimization, operations research, linear and nonlinear programming, integer programming, constraint programming, convex optimization, continuous optimization, or unconstrained optimization. You'll find valuable resources here to enhance your understanding of these subjects.

Notion version

Table of Contents

1. Video Lectures and Courses
   1. Convex Optimization
   2. Discrete, Combinatorial, and Integer Optimization
   3. Operations Research
   4. Meta-heuristics
   5. Dynamic Programming and Reinforcement Learning
   6. Constraint Programming
   7. Continuous Optimization
   8. Stochastic Optimization and Control
2. Books and Lecture Notes
   1. Convex Optimization
   2. Bayesian Optimization
   3. Operations Research
   4. Meta-heuristics
   5. Dynamic Programming and Reinforcement Learning
   6. Constraint Programming
   7. Combinatorial Optimization
   8. Stochastic Optimization
3. Solvers, Tools, and Libraries
   1. Licensed/Commercial Solvers
   2. Free Solvers
   3. Tools and Libraries
4. Other Resources
   1. Communities and Forums
   2. Events, Societies, and Conferences
   3. Code Repositories
5. Contribution Guidelines

Video Lectures and Courses

Convex Optimization

• Stanford EE364A: Convex optimization I by Stephen Boyd: [2023 version on YouTube] | [Course Website]
  Concentrates on recognizing and solving convex optimization problems that arise in engineering. Convex sets, functions, and optimization problems. Basics of convex analysis. Least-squares, linear and quadratic programs, semidefinite programming, minimax, extremal volume, and other problems. Optimality conditions, duality theory, theorems of alternative, and applications. Interiorpoint methods. Applications to signal processing, control, digital and analog circuit design, computational geometry, statistics, and mechanical engineering. Old version (2008) of the course on [YouTube] | [Stanfored website] | [edX]

• Stanford EE364b: Convex optimization II by Stephen Boyd: [YouTube] | [Course Website] | [edX]
  Continuation of Convex Optimization I. Subgradient, cutting-plane, and ellipsoid methods. Decentralized convex optimization via primal and dual decomposition. Alternating projections. Exploiting problem structure in implementation. Convex relaxations of hard problems, and global optimization via branch & bound. Robust optimization. Selected applications in areas such as control, circuit design, signal processing, and communications. Course requirements include a substantial project.

• CMU: Convex Optimization (Fall 2018): (YouTube) by Ryan Tibshirani

  • Fall 2019
  • Fall 2016 (YouTube)

• Convex Optimization by Ahmad Bazzi (YouTube Playlist) + CVXPy tutorials

• Convex Optimization - NPTEL by Joydeep Dutta

Discrete, Combinatorial, and Integer Optimization

• U Wisconsin-Madison: Integer Optimization - ISyE/Math/CS 728 by Alberto Del Pia

  • Spring 2023: Syllabus | Slides | YouTube | Website
  • Spring 2020: YouTube

• A Second Course in Algorithms (Stanford CS261, Winter 2016) by Tim Roughgarden

• Combinatorial Optimization Course by Constantine Caramanis at UT Austin

• CSE 550: Combinatorial Optimization and Intractability by Joshua J. Daymude from Arizona State University

• U Warwick: MA252 Combinatorial Optimization by Jonathan Noel

• Coursera: Discrete optimization (series) - The University of Melbourne

  • Discrete Optimization (YouTube version)
  • Basic Modeling for Discrete Optimization
  • Advanced modeling for discrete optimization
  • Solving algorithms for discrete optimization

• Discrete Optimization, MSRI Berkeley by Shmuel Onn

• Texas A&M: ISEN 668: Integer Programming (partial) by Sergiy Butenko

• U Illinois - Urbana Champaign: IE511 - Integer Programming by James Davis

• Overview of Linear and Mixed Integer Programming YouTube Series by Mike Wagner

Operations Research

• Penn State: Math484 Linear Programming - Summer 2020 by Wen Shen

  • 2016 version: YouTube

• U Wisconsin-Madison: Linear Optimization - ISyE/Math/CS/Stat 525 (Fall 2021) by Alberto Del Pia

• Texas A&M: ISEN 622: Linear Programming by Sergiy Butenko

• Colorado State U: Math 510 - Linear Programming and Network Flows (Fall 2020) by Henry Adams

• Linear Programming YouTube Series by Shokoufeh Mirzaei

• Advanced Operations Research - NPTEL by G. Srinivasan

• Coursera: Operations Research (series) - National Taiwan University

  • Operations Research (1): Models and applications
  • Operations Research (2): Optimization algorithms
  • Operations Research (3): Theory

• Optimization Techniques/Operation Research Playlist by MathPod Channel

• Operations Research I & II by Dedy Suryadi

  • OR I (YouTube)
  • OR II (YouTube)

Meta-heuristics

• Metaheuristics Graduate Course - by Helena Ramalhinho Lourenço - Universitat Pompeu Fabra

• Evolutionary Computation for Single and Multi-Objective Optimization - by Deepak Sharma - NPTEL IIT Guwahati

• Introduction to Metaheuristics - by Luis R. Izquierdo

• CPAIOR 2020 Master Class: Metaheuristics

Dynamic Programming and Reinforcement Learning

• Dynamic Programming Lectures by Dimitri Bertsekas

• UC Berkeley CS 285: Deep Reinforcement Learning - Fall 2022

  • Course website
  • Fall 2021 version

• Stanford CS234: Reinforcement Learning — Winter 2019 - by Emma Brunskill

  • Course website

• Reinforcement/Deep Learning Lecture Series 2021 - by DeepMind x UCL

  • 2018 version

Constraint Programming

• An Introduction To Constraint Programming - Jacob Allen

• CPAIOR 2020 Master Class: Constraint Programming

• EdX: Constraint Programming course - UCLouvain (LouvainX):
  Understand the constraint programming paradigm. Design and implement a modern constraint programming library. Model using the constraint programming. Extend the solver with new global constraints. Design custom and black-box searches. Approach Scheduling and Vehicle Routing problems with constraint programming.

Optimization Algorithms

• Optimization Algorithms by Constantine Caramanis at UT Austin

  • additional lectures

• Optimization Methods for Machine Learning and Engineering (KIT Winter Term 20/21) by Julius Pfrommer

  • slides and recordings

• Dimitri Bertsekas's Videos on Dynamic Programming, Reinforcement Learning, etc.

Continuous Optimization

• Mathified YouTube Channle

  1. Unconstrained Optimization Lecture
  2. Constrained Optimization Lecture I
  3. Constrained Optimization Lecture II

• Arizona Math Camp: Optimization
  Local and global optimization. Unconstrained and constrained optimization. Solution function and value function. Implicit Function Theorem. Envelope Theorem. KKT conditions. Kuhn-Tucker Theorem.

• Optimization - NPTEL by A. Goswami & Debjani Chakraborty

Stochastic Optimization and Control

• Basic Course on Stochastic Programming from Instituto de Matemática Pura e Aplicada

• Stochastic Programming by Anthony Papavasiliou

• CTU Prague: Optimal and robust control

• CMU Robotic Exploration Lab: Optimal Control 2022

Books and Lecture notes

Convex Optimization

• Convex Optimization by Boyd and Vandenberghe - PDF
• Convex Optimization Theory by Dimitri P. Bertsekas - PDF

Bayesian Optimization

• Bayesian Optimization In Action by Quan Nguyen - Amazon
• Experimentation for Engineers by David Sweet - Amazon

Operations Research

• Operations Research An Introduction by Hamdy A. Taha - Pearson
• Introduction to Operations Research by Frederick Hillier and Gerald Lieberman - McGraw Hill
• Julia Programming for Operations Research by Changhyun Kwon - PDF - code
• Mathematical Programming and Operations Research: Modeling, Algorithms, and Complexity. Examples in Python and Julia. Edited by Robert Hildebrand - PDF
• A First Course in Linear Optimization by Jon Lee - PDF
• Decomposition Techniques in Mathematical Programming by Conejo , Castillo , Mínguez , and García-Bertrand - Springer
• Algorithms for Optimization by Mykel J. Kochenderfer and Tim A. Wheeler - PDF
• Model Building in Mathematical Programming - Introductory modeling book by H. Paul Williams - Wiley

Meta-heuristics

• Metaheuristics by Patrick Siarry - Springer (open access)

• Essentials of Metaheuristics by Sean Luke - link

• Handbook of Metaheuristics by Michel Gendreau and Jean-Yves Potvin - Springer (open access)

• An Introduction to Metaheuristics for Optimization by Bastien Chopard , Marco Tomassini - Springer (open access)

• Metaheuristic and Evolutionary Computation: Algorithms and Applications by Hasmat Malik, Atif Iqbal, Puneet Joshi, Sanjay Agrawal, and Farhad Ilahi Bakhsh - Springer (open access)

• Clever Algorithms: Nature-Inspired Programming Recipes by Jason Brownlee - GitHub

• Metaheuristics: from design to implementation by El-Ghazali Talbi - Wiley

Dynamic Programming and Reinforcement Learning

• Various tiltes on Dynamic Programming, Optimal Control and Reinforcement Learning by Dimitri Bertsekas. - List

• Reinforcement Learning: An Introduction (2nd Edition) by Richard Sutton and Andrew Barto - PDF

• Decision Making Under Uncertainty: Theory and Application by Mykel J. Kochenderfer - PDF

• Algorithms for Decision Making by Mykel J. Kochenderfer, Tim A. Wheeler, and Kyle H. Wray - PDF

Constraint Programming

• Handbook of Constraint Programming by Francesca Rossi, Peter van Beek and Toby Walsh - Amazon

• A Tutorial on Constraint Programming by Barbara M. Smith (University of Leeds) - PDF

Combinatorial Optimization

• Combinatorial Optimization: Algorithms and Complexity by by Christos H. Papadimitriou and Kenneth Steiglitz - Amazon
• Combinatorial Optimization: Theory and Algorithms by Bernhard Korte and Jens Vygen - Springer
• A First Course in Combinatorial Optimization by Jon Lee - Amazon

Stochastic Optimization

• Lectures on Stochastic Programming Modeling and Theory (SIAM) - by Shapiro, Dentcheva, and Ruszczynski - PDF
• Introductory Lectures on Stochastic Optimization by John C. Duchi - PDF

Check out More of Prof. Bertsekas's Books

Solvers, Tools, and Libraries

Solvers

Licensed/Commercial Solvers

• BARON - The Optimization Firm
• CPLEX - IBM
• Gurobi - Gurobi
• FICO Xpress - FICO
• LocalSolver - LocalSolver
• Knitro - Artelys
• Mosek - Mosek
• Concorde - UWaterloo

Free Solvers

• CBC - COIN-OR Branch and Cut
• GLPK - GNU Linear Programming Kit
• HiGHS - High Performance Software for Linear Optimization
• lp_solve - (Mixed Integer) Linear Programming solver
• SCIP - Solving Constraint Integer Programs
• DIDP - Domain-Independent Dyanmic Programming

Benchmarks

• Hans Mittelmann's Benchmarks

Tools and Libraries

• Google OR-Tools | examples | repository
• COIN-OR | repository
• MiniZinc - Constraint modeling language.
• GAMS - High-level modeling system.
• AMPL - Algebraic modeling language, both linear/integer and constraint programming.
• JuMP - Open source modeling language for Julia.
• Pyomo - Open source modeling library for Python.
• PuLP - Open source modeling library for Python.
• PyOptInterface - Efficient modeling interface for mathematical optimization in Python.
• Scipy Optimize

Other Resources

Communities and Forums

• OR Stack exchange
  • Related question to this repo
• Subreddit on OR (r/Operations Research)
• OR Discord server
• Discrete Optimization Talks (DOTs) Discord server
• Google OR tools Discord and Discussion forum

Events, Societies, and Conferences

• Mathematical Optimization Society - international organization dedicated to the promotion and the maintenance of high professional standards in the subject of mathematical optimization.
• Mixed Integer Programming Society
• INFORMS - Institute for Operations Research and the Management Sciences.
• Discrete Optimization Talks (DOTs) - virtual seminar series.
• CPAIOR - Integration of Constraint Programming, Artificial Intelligence, and Operations Research.
• CP Conference - Principles and Practice of Constraint Programming.
• CORS - Canadian Operational Research Society
• CO@Work - Optimization course with a focus on computational aspects, industrial applications and networking (2024, 2020, 2015, 2009, 2005)

Code Repositories

• Open Optimization:
  a platform for sharing and collaborating on open educational resources for optimization related courses.
• Introduction to Linear Programming with Python:
  Introduction to Linear Programming with Python
• cvxpy:
  A Python-embedded modeling language for convex optimization problems.
• MIPLearn:
  Framework for solving discrete optimization problems using a combination of Mixed-Integer Linear Programming (MIP) and Machine Learning (ML)
• or-gym:
  nvironments for OR and RL Research
• ecole:
  Extensible Combinatorial Optimization Learning Environments
• pyconcorde:
  Python wrapper around the Concorde TSP solver
• BOTorch:
  Bayesian optimization in PyTorch
• EvoTorch:
  Advanced evolutionary computation library built directly on top of PyTorch, created at NNAISENSE.
• TorchOpt:
  efficient library for differentiable optimization built upon PyTorch.

Contribution Guidelines

We welcome contributions to this repository. If you have a course or resource that you'd like to add, please follow these guidelines:

1. Make a pull request, adding the course to the relevant section with a brief description and a link to the resource.
2. Use a consistent format for entries to maintain readability.

Thank you for your contributions to making this repository a valuable resource for optimization enthusiasts in the academic community!

TODO

1. add missing topics
   • numerical optimization/methods
   • bayesian optimization
2. add github repos
3. complete solver/software list
4. add short description
5. better fomatting
   • tables instead of bullets