Addressing compliance test specific challenges requiring robust PCIe 5.0 transmitter verification

“As 5G and IoT connected devices and the associated high bandwidth requirements are expected to increase dramatically, data center operators will need to migrate to higher bandwidth networks beyond the 100 Gigabit Ethernet (100GE) commonly used today. Migrating to next-generation 400GE networks requires faster memory and higher-speed serial bus communications. In addition to upgrading the Ethernet interface to 400GE, the server also needs to use a higher-speed serial expansion bus interface and memory.

“

Author: Tektronix

As 5G and IoT connected devices and the associated high bandwidth requirements are expected to increase dramatically, data center operators will need to migrate to higher bandwidth networks beyond the 100 Gigabit Ethernet (100GE) commonly used today. Migrating to next-generation 400GE networks requires faster memory and higher-speed serial bus communications. In addition to upgrading the Ethernet interface to 400GE, the server also needs to use a higher-speed serial expansion bus interface and memory.

The PCIe (PCI Express) expansion bus is now migrating to the latest standardized PCIe 5.0, also known as PCIe Gen 5. At the same time, DDR (Double Data Rate) memory is also migrating from DDR 4.0 to DDR ≈ 5.0. The PCIe Gen 5 specification is a fast-rising enhancement to the PCIe 4.0 standard developed by the PCI-SIG. PCI-SIG is a standards body that defines all PCIe specifications. With the finalization of the PCIe 5.0 Plug-In Electromechanical (CEM) specification, the PCIe 5.0 standard was recently completed and released in June 2021, a companion piece to the existing PCIe 5.0 base (silicon) specification released in 2019.

The evolution of the PCIe standard doubles the transmission speed

The original parallel PCI bus was introduced in 1992 to expand the capabilities of personal computers, allowing the addition of graphics and network cards and many other peripherals. PCIe is a high-speed serial bus designed to replace PCI and other existing legacy interfaces such as PCI-X (PCI eXtended) and AGP (Accelerated Graphics Port). PCIe not only has high throughput, but also has a small size, and the link width can be expanded between ×1, ×2, ×4, ×8, and ×16. PCIe is based on a point-to-point bus topology between the root complex (system/host) and endpoints (plug-ins), supporting full-duplex packet-based communication.

PCIe Duplex Link Communication

The PCIe 1.0 standard was introduced in 2003 and provided a rate of 2.5 G transfers per second (2.5 GT/s). PCIe currently offers rates from 2.5 GT/s to 32 GT/s. PCIe 5.0 doubled PCIe 4.0 transfer rates from 16 GT/s to 32 GT/s, but didn’t offer any new additions, as the goal at the time was to provide additional speed in the shortest amount of time.

All PCIe standards released today use non-return-to-zero (NRZ) signaling. However, the PCI-SIG is currently developing the PCIe Gen 6 specification, which will again double the transfer rate to 64 GT/s, migrating away from NRZ signaling. The Gen 6 sixth-generation specification will use PAM-4 signaling, as well as low-latency FEC (forward error correction) technology to improve data integrity.

All PCIe standards must be backward compatible, that is, PCIe 5.0 (32 GT/s maximum data rate) must also support 2.5 GT/s, 5 GT/s, 8GT/s, 16 GT/s, and 32 GT/s .

PCIe Specification Timeline

PCIe lanes and link speeds

PCIe compliance testing with specific challenges

PCI-SIG is the developer of the non-proprietary PCI technology standard and related specifications, and PCIe is now the de facto standard for servers. The PCI-SIG defines the PCI specification to support the required I/O functionality while being backward compatible with previous specifications. To enable industry-wide adoption of PCI technology, the PCI-SIG supports both interoperability and compliance testing, including the tests that must be performed and passed to achieve compliance.

PCI-SIG allows members to conduct interoperability testing against other member products and test suites, and products that participate in the test either pass or fail the test. To pass formal conformance testing, products must pass at least 80% of interoperability tests and pass all standard conformance tests.

PCIe 5.0 presents specific challenges. With a maximum data rate of 16 GT/s, PCIe 4.0 is a speed-enhancing specification for the previous generation of PCIe and has proven to be more difficult to implement than previous standards. In PCIe 5.0, both computer PCIe lanes and motherboards face significant challenges as they deal with 32 GT/s data rates. In addition to the challenges encountered at lower data rates, PCIe 5.0 devices are expected to experience significant signal integrity challenges. Tektronix has PCI-SIG approved test suites for all data rates (Tx, Rx and PLL bandwidth).

Tektronix PCIe Gen 5 Tx Compliance Test Solutions

Tektronix is ​​a major contributor to the PCI-SIG, has made significant contributions to the PCIe 4.0 and 5.0 physical layer test specifications, and has done extensive road-seeking experiments to define PCIe 6.0 Tx/Rx measurement methods. Tektronix also played a key role in conformance and interoperability testing during the development and implementation of the PCIe standard.

PCIe 5.0 transmitter testing, proper test equipment and automation software are essential

When developing a PCIe Gen 5 transmitter device, either at the base (chip) level or at the CEM (system and plug-in) level, chip-level verification (usually performed by a PHY IP company) and pre-compliance testing will be required before the Submit the device to the PCI-SIG for formal compliance testing. Therefore, obtaining appropriate test equipment and associated automation software is critical.

PCIe compliance testing includes:

• Electrical Testing – Evaluate Platform, Plug-In Transmitter (Tx) and Receiver (Rx) Features
• Configuration Testing – Evaluate configuration space in PCIe devices
• Link Protocol Test – Evaluate the device’s link-level protocol characteristics
• Transaction Protocol Test – Evaluate the device’s transaction-level protocol features
• Platform BIOS Test – Evaluates the ability of the BIOS to recognize and configure PCIe devices

In terms of electrical testing, it is divided into two sets of measurements, one at the basic level and one at the CEM level. These tests are further divided into standard tests and reference tests:

PCIe Basic and CEM Compliance Measurements

Eye

Both types of measurements require a high-bandwidth real-time oscilloscope capable of capturing data waveforms. Post-processing techniques are then used to make the appropriate voltage and timing measurements required in the base specification and the CEM specification. Uncorrelated jitter examines the jitter inherent in the system after removing packet and channel intersymbol interference (ISI). In addition to jitter, the oscilloscope makes eye height and eye width measurements. A number of “conformance test patterns” are specified in the base specification. A waveform record containing multiple occurrences of the entire compliance test pattern is recommended to construct a representative eye diagram.

In basic Tx testing of the device, the specification states that measurements are made directly at the pins of the transmitter. If direct access is not possible, test points should be as close as possible to the device pins. If the user has a good understanding of the S-parameters, any splice channel loss can be de-embedded by physically reproducing the channel or emulating it. Starting with the 4.0 specification, another de-embedding technique is described, applying CTLE (Continuous Time Linear Equalization) to uncorrelated jitter measurements during waveform post-processing, which effectively eliminates ISI down to the pin.

Tx EQ presets

Any PCIe 5.0 product submitted for PCI-SIG certification must successfully pass compliance testing using the specified Tx equalizer settings preset, supporting speeds from 2.5 GT/s up to 32 GT/s. These presets are used to equalize intersymbol interference caused by frequency-dependent attenuation differences within the code stream, improving signal integrity. Each preset is a specific combination of undershoot (before the cursor) and de-emphasis (after the cursor).

There are various specific implementations that allow the DUT transmitter to scan through various data rates and Tx EQ presets. However, the base specification specifies a common method in which a 100 MHz clock burst is delivered to lane 0 of the receiver. This can be done automatically using an arbitrary function generator (AFG).

For PCIe links with a maximum rate of 32 GT/s, there are new verification challenges for the base clocks (Refclks). The base specification has expanded the jitter limit proportional to the data rate, but Gen5 has lowered the limit disproportionately to 150 fs. This high-frequency jitter measurement requires proper application of the common clock transfer function and consideration of worst-case transfer delays. This latest version of the specification also pushes measurements from a basic level specification (chip level) to a CEM specification requirement (appearance level), which must meet compliance testing.

CEM plug-in PCIe 5.0 conformance test and automatic preset switching

Tektronix PCIe Solutions for Conformance Testing with Confidence

Oscilloscope bandwidth and sample rate requirements. For the basic Tx test, each PCIe 5.0 lane runs at 16 GHz (since two bits can be sent in one cycle), and the third harmonic reaches 48 GHz. Since there is not much valid signal information above the third harmonic, PCIe 5.0 basic Tx testing only requires a real-time oscilloscope with 50 GHz bandwidth. For the CEM Tx test, the measurement is made near the end of the worst-case channel, reducing high frequency content and requiring a bandwidth of 33 GHz. To ensure adequate waveform post-processing (SigTest), a minimum of 4 points per unit interval is required, and CEM allows up to 2x sinx/x interpolation, so a minimum sampling rate of 128 GS/s is required.

Automatic conformance testing. In conformance testing, performing analysis manually is time-consuming and error-prone. To save time, it is best to use automated software, which not only reduces the workload but also speeds up compliance testing. For electrical verification, PCI-SIG offers SigTest offline analysis software, which performs analysis using data acquired by an oscilloscope. The automation software also controls the device under test (DUT), using an arbitrary function generator as the pattern source, allowing the DUT to automatically pass the various speeds, de-emphasis, and presets required for compliance testing.

A complete round of compliance testing requires the acquisition of multiple waveforms per channel at different DUT settings. This set of waveforms will be increased by the number of channels (up to 16) that need to be analyzed. The ability of software to manage and store data for analysis and future reference requirements is an important metric for any compliance test solution. The automation software can also adjust the oscilloscope horizontal and vertical settings and acquisition. In addition to configuration and analysis, multiple acquired waveforms can be managed using automation software.

The automation software can select the data rate, voltage swing, presets, and tests to perform. It can also provide options to embed package parametric models, de-embed cables, test fixtures, or other elements needed to reach the target test points specified by the specification. Analysis results from the software can often be compiled into a report in PDF or HTML format, which can include pass/fail test summaries, eye diagrams, setup configurations, and user notes.

Using Tektronix DPO70000SX Series oscilloscopes and AFG31252 Arbitrary Function Generator, PCI Express Gen1/2/3/4/5 solution automates transmitter verification and compliance testing at basic (chip) and CEM (system and plug-in) levels .

TekExpressPCIe 5.0 Tx Automatic Software Features:

• Auto-step the DUT through different tempo, pattern and Tx EQ presets
• Verify correct signal at transmitter before taking measurements
• Perform channel and packet embedding and de-embedding
• Support for SigTest and SigTest Phoenix software and template files
• 100 MHz reference clock jitter and signal integrity measurements using Silicon Labs. “PCIe Clock Jitter Tool” and Tektronix DPOJET software

Historically, when a new generation of PCIe devices entered compliance testing, a significant portion of the devices would fail the first interoperability workshop when they were tested for PHY and link training compliance. Before the PCI-SIG workshop, it is critical to ensure that a complete oscilloscope, AFG, BERT (for Rx test) and automation software solutions are in place. Tektronix PCIe Test and Debug Tx, Refclk and Rx solutions guide you through compliance testing and debugging prior to interoperability testing, ensuring your designs meet PCI-SIG® PCIe standard requirements with confidence.