Recently, market research firm LightCounting (hereinafter referred to as LC) released a new report on the theme of "Optics for Artificial Intelligence". The report points out that artificial intelligence has emerged in the blink of an eye. LC's first "Optics for Artificial Intelligence" report emphasizes how artificial intelligence is changing computer architecture and networks, and optics plays a key role in this. LC's artificial intelligence predictions focus on optics. However, the agency said that if there is one more prediction, it is: not only will optics play an important role in the development of artificial intelligence systems, but artificial intelligence will increasingly contribute to the design of these systems at the transistor, chip and system levels.
The pace of innovation varies across industries. New applications are developed at a rapid pace. Most of them will fail, but some will succeed and seemingly change the world overnight. Software and AI algorithms are being innovated faster than we can imagine. At least that’s what it seems to be to outside observers, but experts may not see it that way.
Innovation in the hardware space is a more gradual but relentless process. Optical connectivity is no exception, and there is solid data to support this. The adoption of silicon photonics has taken a decade, waiting for the technology to bring truly disruptive solutions, such as reliable co-packaged optics. This will undoubtedly happen by the end of this decade, but the forecasts in this report focus on pluggable optical modules deployed in AI clusters - the dominant solution for optical connectivity today and over the next five years.
Currently, more than 90% of optical modules deployed in AI clusters are used for InfiniBand and Ethernet connections. As shown in the figure below, the bandwidth required for NVLink to connect to GPUs is 4 times that of Ethernet and InfiniBand. Another bottleneck in AI cluster design is the limited high-bandwidth memory (HBM) available to GPUs, which is another factor in the 3x bandwidth increase.
It turns out that optical switches can be used to scale and reconfigure its AI clusters to improve cluster performance while minimizing cost and power consumption. The research firm expects more companies to adopt this technology in the next 3-5 years.
The scale of optical demand for applications in AI clusters is a pleasant surprise in 2023, and the timing of ChatGTP making headlines in late 2022 is particularly significant. Fears of an impending recession and initial signs of declining revenue growth have forced all leading cloud companies to cut spending, including investments in data centers and purchases of optical modules. Strong growth in 2024-2025 is unquestionable.
The growth of business from large network equipment manufacturers is the main factor affecting the sales volume of optical modules in 2023-2025. New designs for AI clusters require more optical modules. All previous systems used only InfiniBand networks for optical connections, and most of these systems were AOCs. The latest systems based on NDR (400G) InfiniBand use pluggable 400/800G SR4/SR8 and DR4/DR8 optical modules instead of AOCs. The company also released an NVLink chassis switch designed for 800G optical connections in March 2022. The company is currently testing optical fiber NVLink internally, but these solutions should be available to end users by the end of 2024. If it takes longer, LC said it will have to lower its forecast for 2025-2029.
This report provides the first forecast for optical modules provided by a large network equipment manufacturer and compares them with other optical modules used in AI clusters. The manufacturer designs optical modules with stricter bit error rate specifications to minimize transmission errors. This does not prevent customers from using third-party optics, but system performance cannot be guaranteed. LC expects that end users will eventually transition to using third-party optics to save costs, but this will be a gradual transition process.
