Resource Aggregator for Online Learning

A collection of resources on Online Learning, Multi-Armed Bandits, and related areas.

Books

Online Learning

• Introduction to Online Convex Optimization by Elad Hazan

• Introduction to Online Optimization by Sebastien Bubeck

• A Modern Introduction to Online Learning by Francesco Orabona

• Online Learning and Online Convex Optimization by Shai Shalev-Shwartz

• Prediction, Learning, and Games by Nicolo Cesa-Bianchi and Gabor Lugosi

• Statistical Learning and Sequential Prediction by Alexander Rakhlin and Karthik Sridharan

• Online learning Lecture Notes by Gábr Bartók, Dávid Pál, Csaba Szepesvári and István Szita

Bandit

• Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems by Sebastien Bubeck and Nicolo Cesa-Bianchi

• Introduction to Multi-Armed Bandits by Aleksandrs Slivkins

• Bandit Algorithms by Tor Lattimore and Csaba Szepesvári

• Bandit Convex Optimisation by Tor Lattimore

Online Algorithms

• Online Computation and Competitive Analysis by Allan Borodin and Ran El-Yaniv

• The Design of Competitive Online Algorithms via a Primal-Dual Approach by Niv Buchbinder and Joseph (Seffi) Naor

• An Introduction to Online Computation by Dennis Komm

• Prophets and Secretaries by Anupam Gupta

Reinforcement Learning Theory

• Reinforcement Learning: Theory and Algorithms by Alekh Agarwal, Nan Jiang, Sham M. Kakade and Wen Sun

• Algorithms for Reinforcement Learning by Csaba Szepesvári

• Statistical Reinforcement Learning and Decision Making: Course Notes by Dylan J. Foster and Alexander Rakhlin

• Reinforcement Learning: Foundations by Shie Mannor, Yishay Mansour, and Aviv Tamar

Control Theory

• Online Nonstochastic Control by Elad Hazan and Karan Singh

Surveys

• Online Learning Algorithms by Nicolo Cesa-Bianchi and Francesco Orabona

• The Multiplicative Weights Update Method: a Meta-Algorithm and Applications by Sanjeev Arora, Elad Hazan, and Satyen Kale

• Potential-Function Proofs for Gradient Methods by Nikhil Bansal and Anupam Gupta

• Handbook of Convergence Theorems for (Stochastic) Gradient Methods by Guillaume Garrigos and Robert M. Gower

• No-Regret Dynamics in the Fenchel Game: A Unified Framework for Algorithmic Convex Optimization by Jun-Kun Wang, Jacob Abernethy and Kfir Y. Levy

Blogs

• Parameter-free Learning and Optimization Algorithms by Francesco Orabona

• Bandit Algorithms by Tor Lattimore and Csaba Szepesvári

• I’m a bandit by Sebastien Bubeck

• StephenTu’s blog by Stephen Tu - great content on control theory related topics

• Tim van Erven’s blog by Tim van Erven

• Adversarial Intelligence by Wouter Koolen

Courses

• Haipeng Luo
  • Introduction to Online Optimization/Learning, Fall 2022. [link]
  • Introduction to Online Learning, Fall 2017. [link]
• Akshay Krishnamurthy
  • Spring 2022: COMS 6998-11: Bandits and Reinforcement Learning
  • Fall 2017: Machine Learning Theory
• Kevin Jamieson
  • Winter 2022: CSE 541 Interactive Machine Learning
  • Spring 2021: CSE 599 Interactive Machine Learning in Non-stochastic Environments
  • Winter 2021: CSE 599 Interactive Machine Learning in Stochastic Environments
  • Winter 2020: CSE 599 Interactive Machine Learning
  • Winter 2018: CSE 599 Online and Adaptive Methods for Machine Learning
• Chi Jin
  • Spring 2022: ECE524: Foundations of Reinforcement Learning
  • Spring 2023: ELE539/COS512: Optimization for Machine Learning
  • Fall 2021: ECE434/COS434: Machine Learning Theory

Video Lectures

• Nicolo Cesa Bianchi at Summer Graduate School on Mathematics of Machine Learning 2022

• Kevin Jamieson at Summer Graduate School on Mathematics of Machine Learning 2019

• Five Miracles of Mirror Descent by Sebastien Bubeck

• Bandit Convex Optimization by Sebastien Bubeck

• Bandit Algorithm (Online Machine Learning) by Prof. Manjesh Hanawal

Simons Programs

• Data-Driven Decision Processes

• Learning and Games

• Theory of Reinforcement Learning

• Algorithms and Uncertainty

• Interactive Learning within the Foundations of Machine Learning program

• Optimization, Statistics and Uncertainty within the Bridging Continuous and Discrete Optimization program

Written on November 16, 2020