400ZR
DISTRIBUTED DATA CENTER INTERCONNECT NEEDS A LOT OF BANDWIDTH
As data center operators shifted to distributed data center architectures around 2019, existing optical interconnect solutions proved inadequate. Distributed data centers offer great benefits, such as allowing locations to be much closer to the end user, increasing service availability, reducing latency and eliminating the need for expensive, large and centralized real estate. However, distributed data centers require massive data center interconnect (DCI) bandwidth. In order to make the distributed sites operate as a single data center, many hundreds of gigabits per second of bandwidth are required between the sites, at distances longer than traditional data center optics can reach.
Traditional embedded coherent optics can provide that level of bandwidth, but they come with costs that make them unsuitable for massive DCI: they are large, require hardware elements in separate racks along with additional power to operate those elements, and they are expensive. Building a distributed DCI network with embedded optics would be cost-prohibitive. Initial pluggable solutions that were designed for use in telecom networks were in a large CFP2 format that was incompatible with data center routers, did not have line-side interoperability and had no common management interface for control.
400ZR: THE IDEAL SOLUTION AT THE PERFECT TIME
Through operator and vendor member negotiations and discussions, OIF developed an agreement for a pluggable, data center-oriented, 400G coherent solution designed to work over the distances required for distributed data centers (i.e. 40-80km). In late 2016, OIF started work on a solution with approximately 10 contributing companies, and it published its first agreement in early 2020. That agreement – 400ZR – led to the development of the most successful commercial coherent interface of all time.
400G pluggables built on the OIF 400ZR base and its extensions are now in their fourth year of commercial availability, and they have sold 3X the amount of any other coherent type at the same level of maturity. 400ZR modules are available from over 10 different vendors using DSPs from four different sources, and the number of suppliers continues to grow every year.
400ZR, as defined by OIF, not only provided high-speed bandwidth, it also provided it in a data center module format. 400ZR could be provided in QSFP-DD and OSFP formats that fit directly into the switches and routers that data center operators were already rolling out – so no additional optical hardware was required. The 400G speed of 400ZR lined up exactly with the latest Ethernet speed, 400GbE. Both hardware and optics arrived at almost exactly the same moment – so the timing was perfect. And the interoperability enabled by both the optical interface and OIF-defined management protocols meant that coherent optics, for the first time, became a commodity.
Thanks to 400ZR, a standard, interoperable coherent format became available in a cost-effective package. Operators found ways to use the new optics in creative ways as distributed data center configurations quickly grew in popularity. 400ZR opened network operators’ eyes to what was possible with pluggable coherent solutions.
BUILDING ON SUCCESS
The new 400ZR standard proved so popular, it catalyzed several derivative standards and use cases to address adjacent applications leveraging the 400ZR specification. Organizations such as OpenROADM and OpenZR+ authored expansions to the original agreement, growing the market for 400ZR DSPs and optics beyond 80km DCI. These new applications included longer distances and higher powers for metro, long-haul, and traditional telco ROADM networks.
More than any other technology in recent years, OIF’s 400ZR agreement has enabled the explosion of data centers through its low-cost, standardized, high-speed solution.
Authored by: Scott Wilkinson, Cignal AI
