Will supercomputers develop better supercomputers?

Microsoft builds supercomputers for OpenAI

At Build 2020, Microsoft announced further details on the collaboration with the research institute OpenAI. The software company is working with OpenAI to develop a supercomputer to teach large distributed AI models. One of the goals is to improve the artificial intelligence capabilities of your own software and services.

According to the company, the fifth fastest supercomputer in the world will be created according to the TOP500 ranking. The two partners are aligning the system to Massive Distributed AI Models, as researchers assume that they offer better performance than the small and separate AI models that were previously in use.

Microsoft already has its own large AI models called Microsoft Turing. They are primarily used to improve natural language understanding in Bing, Office, and Dynamics. The Turing Natural Language Model is considered to be the largest publicly available AI language model in the world.

In addition, Redmond announced that it would place the Turing model under an open source license. The same applies to the recipes for training Turing using Azure Machine Learning. In addition, the ONNX Runtime, an open source library for the transferability of AI models, is to receive support for distributed training.

The OpenAI supercomputer is designed as a single system with 285,000 CPU cores, 10,000 GPUs and a network connection of 400 GBit / s for each GPU server. The supercomputer is hosted in Azure with access to Azure Services.

Turing is part of a broader Microsoft initiative called AI at Scale. It should make it possible to train huge neural networks once on a high-performance infrastructure in order to then reuse the same model in many other scenarios. This should help to improve the artificial intelligence in various products. For example, Microsoft has trained a model for language understanding that is known as the Turing NLR. The model is also used for Bing, Word, SharePoint and Outlook.

Microsoft customers will not have direct access to the OpenAI supercomputer. However, you can use the updated Azure compute infrastructure. Its DeepSeed software, which is also available as open source, can train large models. According to Microsoft, the ONNX runtime should also accelerate the setting up of models. However, Microsoft also offers the option of accessing Microsoft's Turing model instead of learning your own models.