Date: 26/11/2024
There are some hard-core maths background for convex optimization which I don’t want to rehash off of these posts
Mirror descent is a method for constrained optimization
Re-writing the Projected Gradient Descent method, we arrive at two terms inside the minimizer function. One term is \(\frac{1}{\alpha_k}\frac{\|x-x_k\|^2_2}{2}\), where k is the iteration step and \(\alpha_k\) is the learning step (I guess). If we generalize the Euclidean distance to any notion of distance, then we obtain a term which is like \(\frac{1}{\alpha_k}\frac{d(x,x_k)}{2}\), where \(d : \mathbb{R}^n\times\mathbb{R}^n \rightarrow \mathbb{R}^{+}\).
WHY THOUGH?: That’s coz if we know the shape of the convex set, we can utilize that to better guide our minimizer
https://tlienart.github.io/posts/2018/10/27-mirror-descent-algorithm/