Data Center

Layer 0: What to Expect for Fiber Connectivity

Qing Xu

The footprints of cloud data centers continue to grow substantially to accommodate massive amounts of servers and switches. To support sustainable business growth, many Web 2.0 companies, such as Google, Facebook and Microsoft, have decided to deploy 100G Ethernet using singlemode optics-based infrastructure in their new data centers.

 

According to LightCounting and Dell’Oro, 100G transceiver module and switch port shipments this year will outpace last year’s shipments, with 10 times as many being shipped in 2017 vs. 2016. Shipment for 200G/400G switch ports will begin in 2018.

 

Data Center Architecture and Interconnects

Most intra-rack fiber connectivity has been implemented with DAC (direct-attach cables). As we discussed in our fiber infrastructure deployment blog series, system interconnects with a reach longer than 5 m must use more fiber connectivity to achieve the desired bandwidth.

 

100G, 200G, and 400G transceivers for data center applications have already been showcased by various vendors; massive deployment is expected to start in 2018. Based on reach requirements, different multimode and signal optical transceivers are being developed with optimized balance between performance and cost. Examples include:

  • In-room or in-row interconnects with multimode optics or active optical cables (AOCs), with a reach of up to 100 m. (New multimode transceivers, such as 100G-eSR4, paired with OM4/OM5 multimode fiber, can support a maximum reach of up to 300 m for 100G connectivity, which is suitable for most intra-rack interconnects.)
  • On-campus interconnects (inside the data center facility), with transceiver types such as PSM4 (parallel singlemode four-channel fiber) or CWDM4/CLR4 (coarse wavelength division multiplexing over duplex singlemode fiber pair) for 500 m reach.
  • On-campus interconnects (between data center buildings), with transceiver types such as PSM4 and CWDM4/CLR4 for a reach of 2 km.
  • Regional data center cluster interconnects, also referred as data center interconnects (DCIs), using coherent optics (CFP2-ACO and CFP2-DCO) for a reach of over 100 km, or direct modulation modules, such as QSFP28 DWDM ColorZ, for reach of up to 80 km.

Multimode Fiber Roadmap to 400G and Beyond

Multimode optics use low-cost VCSELs as the light source. When compared to singlemode transceivers, which utilize silicon photonics, VCSELs have some native performance disadvantages:

  • Fewer available wavelengths for wavelength division multiplexing
  • Speed is limited by the singlemode laser
  • Less advanced modulation options
  • High fiber counts needed to deliver required bandwidth
  • Shorter reach in multimode fiber (limited by fiber loss and dispersion) compared to singlemode fiber
MMF-Road-map 

On the other hand, compared to singlemode 500 m and 2 km transceiver alternatives, VCSEL-based multimode optics shine in short-reach applications, offering advantages such as:

  • Shortwave wavelength division multiplexing (SWDM), which allows up to eight wavelengths in one single multimode fiber (MMF)
  • PAM4 modulation, which can enable faster data rates at 50G and 100G
  • Extending the reach of 100G in OM4/OM5 from 100 m to 300 m with DSP and FEC
  • Lower power consumption
  • Lower manufacturing, testing and packaging costs
  • Efficient fiber coupling, lower mating insertion loss and higher return loss, allowing more flexible patching and cabling with more connection points

Moving toward 400G, IEEE 802.3bs has adopted the single wavelength solution (16x 25G) with an MPO-32 optical interface and CFP-8 form factor; however, this fiber connectivity solution is not sustainable. It’s not backward compatible with the established MPO-12 system, and will make the cabling system more complex with comprised fiber connectivity performance.

 

To retain the appeal of MMF in next-generation data centers, SWDM and PAM4 modulation in OM5 MMF will considerably reduce fiber counts, providing a much smoother migration path from 100G to 400G and beyond.

 

IEEE 802.3 has a study group working actively on this new development. Although still in a consensus-building phase for 800G development, 800G multimode PMD can be implemented with four wavelengths with 50G-PAM4 using an MPO-12 interface in the next few years.

 

Singlemode Fiber Roadmap to 400G and Beyond

The future of singlemode optics is very bright when it comes to supporting 400G and beyond. With virtually unlimited bandwidth in singlemode fiber, WDM (wavelength division multiplexing) is the way to go to achieve higher aggregated bandwidth. In addition, many ultra-high-speed transmission technologies have already been developed for long-haul optical transport networks (OTNs) that can benefit the data center industry.

 

SMF-Road-map 

Silicon photonics have already become a game changer, not only in 500 m and 2 km applications (mega data center campus) with tight electronics and optics integration, but also to connect regional data center clusters in DCIs up to 80 km.

 

DC-Architecture 

Although technical feasibility isn’t a deal breaker, cost and product availability (production capacity) can be a barrier since singlemode optics require more complexed packaging and testing procedures than multimode transceivers. For this reason, eight-fiber single-λ PSM4 (parallel singlemode) and two-fiber four-λ CWDM4 (coarse wavelength division multiplexing) transceivers are the most popular in cloud data centers to support breakout and direct connection.

 

To achieve faster time to market, new four-fiber CS connectors have been introduced for 200G/400G QSFP-DD and OSFP transceivers. CS connectors allow the reuse of 100G technology and components to double the bandwidth (to 200G); it also allows 400G products to be available earlier.

 

New singlemode transceiver variants designed specifically for data center applications, including 2x100G-PSM4 (500 m), 2x100G-CWDM4 (2 km), 200G-DR4 (500 m), 2x200G-CWDM4 (2 km), 400G-PSM4 (500 m) and 400G-FR4 (2 km) have already been demonstrated at tradeshows, and will be available for 200G and 400G deployment soon.

 

CSPs (cloud service providers) and system/component vendors have already seen the imminent needs and opportunities for 800G and 1.6T Ethernet deployment. Development of 800G has already been called into the consensus-building phase in conjunction with the OIF-CEI-112G VSR project.

 

Make sure to subscribe to our blog to follow this series on the road to 800G Ethernet. We’ve already covered:

We still plan to cover:

  • The ubiquity of 100G Ethernet
  • Optical fiber cabling migration toward 100G, 200G, 400G and 800G Ethernet

Belden’s complete line of copper and fiber cabling and connectivity solutions will help you prepare for 800G Ethernet – and every other speed along the way. Learn more here.