Recent Papers

Quickly discover relevant content by filtering publications.

We present two new remarkably simple stochastic second-order methods for minimizing the average of a very large number of sufficiently smooth and strongly convex functions. The first is a stochastic variant of Newton’s method (SN), and the second is a stochastic variant of cubically regularized Newton’s method (SCN). We establish local …
PDF arXiv NeurIPS

We present a strikingly simple proof that two rules are sufficient to automate gradient descent: 1) don’t increase the stepsize too fast and 2) don’t overstep the local curvature. No need for functional values, no line search, no information about the function except for the gradients. By following these rules, you get a method adaptive to the …
PDF Code arXiv

We present a new perspective on the celebrated Sinkhorn algorithm by showing that is a special case of incremental/stochastic mirror descent. In order to see this, one should simply plug Kullback-Leibler divergence in both mirror map and the objective function. Since the problem has unbounded domain, the objective function is neither smooth nor it has …
PDF Slides arXiv NeurIPS

We revisit the local Stochastic Gradient Descent (local SGD) method and prove new convergence rates. We close the gap in the theory by showing that it works under unbounded gradients and extend its convergence to weakly convex functions. Furthermore, by changing the assumptions, we manage to get new bounds that explain in what regimes local SGD is faster …
PDF Slides arXiv NeurIPS

We provide the first convergence analysis of local gradient descent for minimizing the average of smooth and convex but otherwise arbitrary functions. Problems of this form and local gradient descent as a solution method are of importance in federated learning, where each function is based on private data stored by a user on a mobile device, and the data of …
PDF Slides arXiv NeurIPS
See all publications

Recent Posts

In addition to my free NeurIPS registration, which I received as on of the top reviewers, I will also receive $1400 from the NeurIPS Foundation to sponsor my travel to the conference.

I was invited by Boris Polyak to present my work on Sinkhorn algorithm at his seminar. The talk took place on Tuesday, 22 October, at the Institute of Control Sciences. It was a great pleasure to hear that Boris liked my work for its simplicity. The slides of my talk are now attached to the corresponding publication ( …

We have submitted 5 papers to 4 different workshops hosted by NeurIPS and all of them were accepted, including one work for oral presentation. The list of papers:

  1. Stochastic Newton and Cubic Newton Methods with Simple Local Linear-Quadratic Rates (oral, with D. Kovalev and P. Richtárik)

  2. Sinkhorn Algorithm as a Special Case of Stochastic Mirror Descent …

We just got a notification that our paper (by D. Kovalev, P. Richtárik and me) was accepted to the NeurIPS workshop “Beyond First-Order Optimization Methods in Machine Learning” for a spotlight (8 minute talk) and poster presentation. Together with the free registration that I got as one of the top reviewers, this gives more than enough reason to …

My new paper (for the first time I wrote a single authored work!) is now available online, see the publication section. It turned out the famous Sinkhorn algorithm is nothing but an instance of stochastic mirror descent. Very exciting to see the notion of relative smoothness appear as the only explanation of convergence from the mirror descent perspective.

As I’ve done some research in the field of minmax optimization and deep learning, I was invited to be a reviewer for this year instance of the Smooth Games Optimization and Machine Learning Series of workshops.

We just uploaded two papers on federated learning to arxiv. The links are above on my website (“Recent publications”).

I received free NeurIPS registration for providing high quality reviews. It is awarded to the top 400 reviewers, and some people call it “Best Reviewer Award”.

This year I am also serving as a reviewer for the AAAI conference, which will take place in February in New-York. See the official website for more details https://aaai.org/Conferences/AAAI-20/#.

I was at the ICCOPT conference in Berlin from 5 to 8 August as the chair of 3 sessions: 2 on variational inequality/minimax/GANs and 1 on non-smooth optimization.
See all posts

Contact