Distributed stochastic gradient descent (SGD) has attracted considerable recent attention due to its potential for scaling computational resources, reducing training time, and helping protect user ...

The conventional mini-batch gradient descent algorithms are usually trapped in the local batch-level distribution information, resulting in the “zig-zag” effect in the learning process. To ...