While a 10 million GPU synchronous pre-training run may not look like current methods, there will likely be some form of training run at that scale in the future, and I would call the learning process semi-synchronous, as it involves many GPUs working together on an AI system, but not all parts will necessarily communicate with each other.