Creating GPU-based systems for the clouds

GPU developers also release hardware solutions that increase the performance of vGPU clusters. For example, last year Nvidia introduced a new graphics processor for Tesla T4 data centers based on the Turing architecture. Compared to the previous generation GPU, T4 performance for 32-bit floating-point operations increased from 5.5 to 8.1 teraflops (p. 60 of the document).

The new architecture has increased the performance of the accelerator due to the separation of work with integer operations and floating-point operations. Now they run on separate cores. Also, the developer combined the shared memory and L1 cache into one module. This approach increased the cache throughput and its volume. T4 cards are already used by large cloud providers.

In the near future, experts expect an increase in demand for cloud graphics accelerators. This will be facilitated by the development of hybrid technologies that combine GPUs and CPUs in one device. In such integrated solutions, two types of cores use a common cache, which speeds up the transfer of data between graphics and traditional processors. Special load balancers are already being developed for such chips, which increase the vGPU performance in the cloud by 95% (slide 16 of the presentation).

But some analysts believe that virtual GPUs will be replaced by a new technology - optical chips in which data is encoded by photons. On such devices, machine learning algorithms are already running. For example, the LightOn startup optical chip completed the task of transfer learning several times faster than a regular GPU - in 3.5 minutes instead of 20.

Most likely, large cloud providers will be the first to introduce new technology. Optical chips are expected to accelerate the training of recurrent neural networks with LSTM architecture and direct distribution neural networks. The creators of optical chips hope that such devices go on sale in a year. A number of major providers are already planning to test a new kind of processor in the cloud.

It is hoped that such solutions will increase the popularity of cloud high-performance technologies and the number of companies that process big data in the cloud will increase.