PCIe7.0 specification: version 0.3 released

Blog / By /

In June 2023, the PCI-SIG released version 0.3 of the PCI Express 7.0 specification, a significant milestone poised to elevate PCIe data transfer rates to 128 GT/s. This release signifies consensus among members on crucial features and architectures for the forthcoming technology. PCIe 7.0 will surpass the 64 GT/s of PCIe 6.0 and the 32 GT/s of PCIe 5.0, enabling a single x16 connection to deliver a staggering 512 GB/s of bidirectional bandwidth before accounting for encoding overhead.

PCIe Rate Table
PCIe Rate Table

To achieve higher data rates and bandwidth, PCIe Gen7 will adopt four levels of Pulse Amplitude Modulation (PAM4) signaling, 1b/1b flit-mode encoding, and Forward Error Correction (FEC), while inheriting standards from PCIe Gen6. While version 0.3 provides a broad overview of goals and methodologies, specifics regarding PCIe Gen7 implementation are eagerly anticipated. As seen in past transitions to PCIe 4.0 and 5.0, PCIe 7.0 will necessitate shorter PCIe lines due to higher signaling speeds, reducing the distances between root and end devices such as CPUs and expansion cards.

Implementing PCIe Gen5 requires thicker PCBs and higher-quality materials, increasing costs. However, cost considerations for PCIe Gen7 remain undisclosed. The PCI-SIG underscores that PCIe 7.0 is tailored for bandwidth-intensive applications such as 800G Ethernet, AI/ML, cloud and quantum computing, hyperscale data centers, HPC, edge, and aerospace/military sectors. While PCIe 7.0 will eventually penetrate client PCs, the PCI-SIG refrains from explicitly mentioning desktops or laptops.

PCI-SIG Chairman Al Yanes asserts, “PCI Express technology is leading the industry as a fundamental I/O interconnect found in everything from automobiles to data center servers.” With escalating PCIe speeds, we are poised to fortify existing verticals and explore novel domains to satisfy the burgeoning demand for high-bandwidth, low-latency interconnects.

PCIe Standard Development History

PCIE3.0

The first generation of PCIe 1.0 standard was launched in 2003, supporting a transfer rate of 2.5 GT/S per lane and a data rate of 250 MB/S per lane. With the advancement of technology, the second generation of PCIe was launched in early 2007, with a transfer rate of 5 GT/s per lane and a doubling of the throughput (bandwidth) to 500 MB/s per lane. Still, due to the 20% overhead for the 8b/10b encoding scheme, the transfer bandwidth of a single lane is 4 Gb/s. PCIe 3.0 and later versions now support 4 Gb/s per lane. However, due to the 20% overhead of the 8b/10b encoding scheme, the transfer bandwidth is 4 Gb/s for a single lane.

PCIe 3.0 and later versions use the more efficient 128b/130b encoding, reducing the overhead to 1.5%. By reducing the overhead, PCIe 3.0 doubles the single-lane transfer bandwidth to 8 Gb/s compared to PCIe 2.0 while maintaining compatibility with PCIe version 2.0 software and mechanical interfaces. With full backward compatibility, PCIe 3.0 provides the same topology for client and server configurations as PCIe 2.0. PCIe 1.x and 2.x cards can be seamlessly inserted into PCIe 3.0-enabled slots, and vice versa, enabling these configurations to operate at the highest negotiated performance levels.

The PCIe 3.0 specification includes the Base and CEM (Card Electromechanical) specifications. The electrical portion of the Base specification defines integrated circuit (IC) level electrical performance and supports 8 GT/s signaling.

Eye Diagram is a time-domain analysis tool commonly used in the communications field to evaluate the signal integrity and transmission quality of digital communication systems (so-called because the graph displayed by an oscilloscope resembles a human eye). As the PCIe transmission rate increases with each standard iteration, signal quality is also affected.

As shown by the eye diagram closure in the figure below, the longer the channel length, the lower the signal quality, and as the speed and channel distance increase, the physical layer verification tests become more challenging. 8 GT/s in PCIe 3.0 severely degrades the signal at the receiver, which will appear on the oscilloscope as an eye diagram closure (unbalanced). For accurate communication, the sender and receiver must reach an equilibrium at the levels that represent ones and zeros and use techniques such as equalization and de-emphasis to make the data visible to the receiver.

Eye Diagram
Eye Diagram

The PCIe 3.0 standard focuses on receiver equalization and transmitter de-emphasis, which are critical to achieving 8 GT/s and higher rates. Equalization can be at the transmitter, receiver, or both. PCIe 1.x and PCIe 2.x specify a simple form of equalization called transmitter de-emphasis. De-emphasis reduces the amount of low-frequency energy the receiver receives, and equalization reduces the effects of more significant channel loss at high frequencies. Various algorithms are used to implement receiver equalization; the two most common are linear feedback equalization and decision-feedback equalization (DFE). Transmitter de-emphasis equalization occurs at the transmitter, while DFE pre-emphasis occurs at the receiver. Receiver equalization may also include continuous-time linear equalization (CTLE) in combination with DFE.

To improve the transmission distance between the transmitter and receiver, PCIe 3.0 introduces an active equalization adaptation process whereby the receiver can adjust the transmitter pre-tip signal and de-emphasize to achieve the equalization performance best suited to its particular transmission line. This performance requires a new physical-layer test: the link equalization test for both the receiver and the transmitter. The purpose of the Link Equalization Receiver test is to check that the receiver can adjust the transmitter equalization of its link under worst-case stress conditions, while the purpose of the Link Equalization Transmitter test is to check that the transmitter performs the changes physically and logically as requested by the receiver of the link.

PCIE 4.0

The PCIe 4.0 standard was introduced in 2017, seven years after PCIe 3.0 went live. Compared to its predecessor, PCIe 4.0 increases transfer rates from 8 Gb/s to 16 Gb/s and is fully compatible with previous generations of the technology, from software to clocking architecture to mechanical interfaces.

From a protocol and coding perspective, PCIe 4.0 has much in common with PCIe 3.0, including 128- or 130-bit encoding. At first glance, PCIe 4.0 has more in common with PCIe 3.0 than PCIe 3.0 has with PCIe 2.0. However, increasing the device’s speed automatically transmits at a higher frequency over the same channel. Resistance in the link during electrical signal transmission causes insertion loss or attenuation that increases with frequency. At 16 GT/s, PCIe 4.0 signals attenuate significantly in a typical FR4 channel (the most common printed circuit board material). Therefore, additional testing is required to ensure the signal integrity of PCIe 4.0 designs, as signal loss at 16 GT/s (PCIe 4.0) is much greater than at 8 GT/s (PCIe 3.0).

PCIe 4.0 added a timer section to the specification to extend the channel range and specifically added complexity testing for the system. Even with increased test complexity, the number of PCIe 3.0 tests at 8 GT/s exceeds that of PCIe 4.0 tests at 16 GT/s. This is because PCIe 3.0 requires three different lane scenarios to be tested: short, medium, and long, whereas PCIe 4.0 is sufficient to test only the long lane scenario.

Like PCIe 3.0, PCIe 4.0 is sometimes called a “closed-eye” specification. This means that even if you have a perfect transmitter with essentially zero jitter, inter-symbol interference will force the “eyes” to close when you connect the transmitter to a channel. Successful transmission of PCIe 4.0 signals depends on the receiver’s equalization strategy, opening the “eyes”. When a PCIe 4.0 device supporting 16 GT/s is linked to another PCIe 4.0 device supporting 16 GT/s, there are two steps to link equalization. First, the link is established at 8 GT/s, and if successful, the link equalization process is repeated one more time to achieve the 16 GT/s rate.

For PCIe 4.0, designers should evaluate their system’s tolerance for performance variations. Understanding performance variations is essential because signaling performance varies from card to card. These variations can increase channel loss, crosstalk, and channel incoherence, resulting in greater system noise, degraded jitter performance, and signal eye closure.

PCIE5.0

Since PCIe 4.0, PCIe iterations have significantly accelerated. The PCI-SIG released the PCIe 5.0 specification in May 2019, doubling the transfer rate to 32 GT/s while maintaining low power consumption and backward compatibility with previous generations. PCIe 5.0 promises up to 128 GB/s throughput with x16 configurations, enabling 400GE speeds in the data center. PCIe 5.0 and 400GE speeds support applications such as artificial intelligence (AI), machine learning, gaming, visual computing, storage, and networking. These advancements foster innovation in 5G, cloud computing, and hyperscale data centers.

The PCIe 5.0 standard is a relatively straightforward extension of 4.0. It uses the same Tx and Rx test methodology as PCIe 4.0 and an “eye” width-and-height methodology similar to that used to calibrate receiver stress jitter tests. The new standard further reduces latency and is compatible with signal loss in long-haul applications. PCIe 5.0 uses the 128b/130b encoding scheme, which debuted in PCIe 3.0 and is compatible with CEM connectors.

New in PCIe 5.0 is an equalization bypass mode that directly enables training from 2.5 GT/s to 32 GT/s, speeding up link initialization. This helps reduce link startup time in transmitter, channel, and receiver-conditioned systems, such as embedded systems, and provides a new training path for link equalization testing at 32 GT/s. In general, specification changes are minimal, except for the need to implement speed increases or electrical modifications to improve the connectors’ signal integrity and mechanical strength.

PCIE 6.0

The PCI-SIG released the PCIe 6.0 specification in January 2022. PCIe 6.0 technology is the first PCI Express standard to use Pulse Amplitude Modulation Level 4 (PAM4) signaling encoding, enabling PCIe 6.0 devices to achieve twice the throughput of PCIe 5.0 devices while maintaining the same channel bandwidth. PCIe 6.0 technology can reach up to 64 GT/s while maintaining low power consumption and backward compatibility.

PCIe 6.0 promises 256GB/s throughput in x16 configurations, enabling 800GE speeds in the data center. These speeds support applications such as AI, machine learning, gaming, visual computing, storage, and networking, powering 5G, cloud computing, hyperscale data centers, and more.

PCIe 6.0 uses the higher-order modulation format, PAM4 signaling, and significantly upgrades PCIe 5.0 technology. However, it uses the same high-level methodology for Tx and Rx testing while adding new emitter-measurement codes specific to PAM4.

Like previous generations, PCIe 6.0 devices use transmitter and receiver equalization for 64 GT/s operation and require forward error correction (FEC).

In addition to these electrical changes, PCIe 6.0 introduces the Flow Control Unit (FLIT) encoding. Unlike PAM4 at the physical layer, FLIT encoding is used at the logical layer to break data into fixed-size packets. PCIe 6.0 transacts in FLITs, with 256 B of data per FLIT (1 FLIT = 236B TLP + 6B DLP + 8B CRC + 6B FEC = 256B), and each B of data occupies 4 UIs. In addition, FLIT encoding also eliminates the 128B/130B encoding and DLLP (Data Link Layer Packet) overhead of previous PCIe specifications, resulting in significant TLP (Transaction Layer Packet) efficiencies.

Although PCIe 6.0 is more advantageous and has been proposed for over a year, PCIe 5.0 has not yet been widely adopted. PCIe 6.0 is needed for high-performance and throughput applications, such as graphics processing units for AI workloads, high-throughput web applications, and Compute Express Link (CXL) technology, which serves as a highway for data interactions in heterogeneous computing architectures. While maintaining backward compatibility with previous generations, the PCIe 6.0 interface doubles the transfer rate to 64 GT/s, providing 256 GB/s of throughput for the same maximum of 16 lanes.

PCIE 7.0

After AMD debuted PCIe 4.0, Intel began popularizing PCIe 5.0 last year. Although the PCIe 6.0 specification was only released to the public earlier this year, PCI-SIG, the standards organization, formally announced the development of PCIe 7.0 today and outlined the core parameters. Like the changes in these generations, PCIe 7.0 is based on PCIe 6.0 again to achieve a bandwidth flip to 128 GT/s, with x16 bidirectional channels reaching 512 GB/s. SSDs often use x2/x4 channels, increasing the theoretical peak speed to 64GB/s and 128GB/s, respectively, allowing unlimited possibilities.

Details-wise, PCIe 7.0 and 6.0 are the same with the new PAM4 modulation and 1b/1b encoding. It’s worth noting that PCIe 7.0 remains backward compatible. The PCI-SIG organization says the next draft will focus on optimizing channel parameters and improving energy efficiency. According to the plan, the original PCIe 7.0 standard will be finalized in 2025, and the outside world expects full adoption around 2028.

It is worth mentioning that the update interval from PCIe 3.0 to 4.0 was up to six years, and from 4.0 onwards, the PCI-SIG organization maintains an iterative speed of every two years.

Related Posts