Recent Papers

Quickly discover relevant content by filtering publications.

We present two new remarkably simple stochastic second-order methods for minimizing the average of a very large number of sufficiently smooth and strongly convex functions. The first is a stochastic variant of Newton’s method (SN), and the second is a stochastic variant of cubically regularized Newton’s method (SCN). We establish local …
PDF Poster arXiv NeurIPS

We present a strikingly simple proof that two rules are sufficient to automate gradient descent: 1) don’t increase the stepsize too fast and 2) don’t overstep the local curvature. No need for functional values, no line search, no information about the function except for the gradients. By following these rules, you get a method adaptive to the …
PDF Code arXiv

We present a new perspective on the celebrated Sinkhorn algorithm by showing that is a special case of incremental/stochastic mirror descent. In order to see this, one should simply plug Kullback-Leibler divergence in both mirror map and the objective function. Since the problem has unbounded domain, the objective function is neither smooth nor it has …
PDF Poster Slides arXiv NeurIPS

We revisit the local Stochastic Gradient Descent (local SGD) method and prove new convergence rates. We close the gap in the theory by showing that it works under unbounded gradients and extend its convergence to weakly convex functions. Furthermore, by changing the assumptions, we manage to get new bounds that explain in what regimes local SGD is faster …
PDF Slides arXiv NeurIPS

We provide the first convergence analysis of local gradient descent for minimizing the average of smooth and convex but otherwise arbitrary functions. Problems of this form and local gradient descent as a solution method are of importance in federated learning, where each function is based on private data stored by a user on a mobile device, and the data of …
PDF Slides arXiv NeurIPS
See all publications

Recent Posts

I was selected as one of 12 Outstanding Program Committee members of AAAI, out of (my guess) about 5000 reviewers in total. There were about 7737 submissions, which is 10% more than that of NeurIPS, for which there were about 4500 reviewers. The award will be officially given to me and the other outstanding reviewers on 11 February at 8am. I’m looking …

Today we received the final decisions for our papers submitted to the AISTATS conference (https://www.aistats.org/). Although one work was rejected, this constitutes a good acceptance rate. Unfortunately, we also had to withdraw one of our submissions. Below is the list of papers that we will present:

  1. Revisiting Stochastic Extragradient (K. Mishchenko, D. …

One of the first papers that I wrote got accepted to the SIAM journal on optimization (SIOPT). The review process was quite long and included several revisions, but I’m happy I got it accepted before my graduation. This work is a result of my collaboration with J. Malick and F. Iutzeler, from whose experience I learned a lot about optimization. The …

In addition to my free NeurIPS registration, which I received as on of the top reviewers, I will also receive $1400 from the NeurIPS Foundation to sponsor my travel to the conference.

I was invited by Boris Polyak to present my work on Sinkhorn algorithm at his seminar. The talk took place on Tuesday, 22 October, at the Institute of Control Sciences. It was a great pleasure to hear that Boris liked my work for its simplicity. The slides of my talk are now attached to the corresponding publication ( …

We have submitted 5 papers to 4 different workshops hosted by NeurIPS and all of them were accepted, including one work for oral presentation. The list of papers:

  1. Stochastic Newton and Cubic Newton Methods with Simple Local Linear-Quadratic Rates (oral, with D. Kovalev and P. Richtárik)

  2. Sinkhorn Algorithm as a Special Case of Stochastic Mirror Descent …

We just got a notification that our paper (by D. Kovalev, P. Richtárik and me) was accepted to the NeurIPS workshop “Beyond First-Order Optimization Methods in Machine Learning” for a spotlight (8 minute talk) and poster presentation. Together with the free registration that I got as one of the top reviewers, this gives more than enough reason to …

As I’ve done some research in the field of minmax optimization and deep learning, I was invited to be a reviewer for this year instance of the Smooth Games Optimization and Machine Learning Series of workshops.

My new paper (for the first time I wrote a single authored work!) is now available online, see the publication section. It turned out the famous Sinkhorn algorithm is nothing but an instance of stochastic mirror descent. Very exciting to see the notion of relative smoothness appear as the only explanation of convergence from the mirror descent perspective.

We just uploaded two papers on federated learning to arxiv. The links are above on my website (“Recent publications”).
See all posts

Contact