Skip navigation
All Places > Keysight Blogs > Oscilloscopes Blog > Blog > 2017 > October
2017

Written by Rick Eads

The Need for a 16GT/s PCIe Interconnect

PCI Express represents one of the most successful computer interconnects yet devised and helped to enable high-speed connections between external devices such as displays and storage adapters to the internal CPU of the computer.  As networking speeds have increased (datacenter), as display resolution has increased (4K streaming) and as disk drive capacity and speed has increased (cloud computing), the need for an improvement in the speed of a host adapter to the CPU has also driven the development of the next generation of the PCI Express 4.0 Standard.  PCie 4.0 technology doubles the bandwidth of the previous generation of PCie 3.0 devices as PCIe 4.0 is able to achieve throughput of nearly 32 GBytes/s for 16 lanes.

 

PCie 4.0 technology doubles the bandwidth of the previous generation of PCie 3.0 devices as PCIe 4.0 is able to achieve throughput of nearly 32 GBytes/s for 16 lanes.

What’s New about the PCIe 4.0 standard versus PCie 3.0?

The biggest improvement brought by the PCIe 4.0 standard is a doubling of the speed to 16GT/s per lane.  Nevertheless, there are additional changes to the PCIe 4.0 specification versus the Gen3 Spec that is worth noting.  Due to the higher data rate, the maximum channel loss accommodated by the PCIe 4.0 standard is approximately -28dB, which implies a maximum channel length of about 12” (or 25cm) with a single CEM class connector.  To accommodate channels longer than 12”, the PCie 4.0 specification provides protocol and electrical requirements for a retimer device which can be used to extend the PCIe 4.0 channel and can help to accommodate multiple connector topologies.  Another new feature of the PCIe 4.0 specification has been the addition of a lane margining capability.  Lane margining allows for in-band (L0) based adjustment of the receiver sampling point which allows for an estimation of eye width, and an optional mode also allows for voltage margining which may provide information regarding eye height.  This feature is intended to help system integrators determine product readiness for shipment.

 

Testing PCIe 4.0 Card Electromechanical (CEM) Devices

While the PCI Express 4.0 BASE specification is nearing 1.0 status, it primarily is written to accommodate new silicon or ASIC devices operating at 16GT/s.  PCI Express also accommodates a system specification referred to as the Card Electromechanical (CEM) specification.  This is the specification that is used at PCISIG compliance workshops to determine if motherboards and add-in cards are compliant to the PCIe 4.0 application.  To accomplish this, the PCISIG has commissioned the development of both new CEM test fixtures and new test software.  As of this writing, both are still in development but are being used in US-based compliance workshops since April 2017.  One new aspect of testing CEM devices under the PCIe 4.0 specification is the addition of an external physical ISI channel that is chosen/calibrated to ensure the maximum CEM channel loss is achieved.

 

                      

Fig 1: Prototype CEM Test Fixtures (CBB4 and CLB4)

Oscilloscope bandwidth requirement

The minimum oscilloscope bandwidth required for PCI Express 4.0 is 25GHz.  As the minimum eye height is 15mV, it is important to utilize real-time oscilloscopes with the lowest noise floor to minimize error and to maximize the measured margins.

 

The minimum oscilloscope bandwidth required for PCI Express 4.0 is 25GHz.  As the minimum eye height is 15mV, it is important to utilize real-time oscilloscopes with the lowest noise floor to minimize error and to maximize the measured margins.

Making PCI Express BASE spec transmitter measurements

Keysight’s new N5393F PCI Express transmitter test application provides step-by-step instructions to guide you through the process of configuring the test setup, selecting the tests and connecting the signals to the oscilloscope.   The N5393F supports PCI Express 4.0 testing at 16GT/s for BASE spec tests and also supports legacy testing under the PCIe 3.0, 2.0, and 1.0a/1.1 standards.  The N5393F test provides visual connection aids to facilitate the connection of your DUT to the oscilloscope and with optional software also integrates the ability to de-embed test fixtures, provide DUT automation for test mode selection (2.5G, 5G, 8G).

 

                 

Figure 2: PCI Express 4.0 BASE spec connection diagram including replica channel for de-embedding

 

                  

Figure 3: Keysight N5393F PCI Express automated test application for the oscilloscope showing the PCIe 4.0 BASE specification tests for 16GT/s validation

Summary

Keysight has been a consistent and valued contributor to the development and authorship of the PCI Express specification since the initial 1.0a draft and continues to provide valuable guidance to the PCISIG for transmitter testing, receiver testing, and channel testing and characterization.  Other optional tools such as the N5465A InfiniiSim Waveform Transformation toolset and N5461A equalization application can provide deep analysis and debug capability. In addition, Keysight has other comprehensive test solutions from design simulation to physical layer testing that includes transmitter, receiver and channel for the PCI Express 4.0 standard as well as previous generations of PCIe.

Written by Doug Beck

What is mask testing?

A fundamental aspect of electrical engineering is finding problems with signal integrity. In the real world, systems don’t always work as simulation says they are supposed to. There is the concept of a “golden trace” where the waveforms are precisely where they need to be. But there are a great number of things that can cause a waveform to vary from the “golden trace.” Unfortunately, these problems can also be quite rare, which makes them difficult to find.

 

The idea of a mask on an oscilloscope is to provide a region around the golden trace where the waveform can vary without causing any problems. An example is shown in Figure 1. The gray area is the mask. This can be thought of as “out of bounds” because if the waveform strays into the gray areas, it is considered an error. However, if the waveform remains within the black areas, the waveform is considered fine.

 

Figure 1. An example of a mask on a waveform

 

The most common alternative to using a mask is to use triggers such as glitch, run, slow edge, fast edge, etc. However, a mask combines all of these triggers into a single test. In general, oscilloscope hardware does not allow all of these errors to be searched for at the same time. The bottom line is that using a mask test can save quite a bit of time.

 

Standards-based mask testing

Because mask tests can be so effective, it is very common for masks to be provided for common standards. Examples include Ethernet and FlexRay. The good news is that if your signal happens to be one where an industry standard mask is available, the job is very easy. Just load the mask into an oscilloscope that supports mask testing and you will quickly find errors.

 

However, it is very common for industry standard masks to be unavailable. An obvious case is proprietary buses. In my experience, not having an industry standard mask is the rule rather than the exception. Fortunately, there are alternatives which allow users to create a mask on their own.

 

Creating your own mask with Auto Mask

A very simple method to create a mask is known as Auto Mask. The idea behind an Auto Mask is to take a single waveform and add a tolerance of the desired size around it both vertically and horizontally. An example of an Auto Mask setup dialog from a Keysight Infiniium oscilloscope is shown in Figure 2. It should be noted that the selection of the source channel is made in the main Mask Test dialog.

 

Figure 2. Auto-mask setup dialog

 

In one single step, a mask is created, such as the one shown previously in Figure 1. It is easy to think that this solves the problem completely. Regrettably, it does not work for most situations where masks are needed.

 

Auto Mask only works where the waveform only follows a specific path and only that path. This means if there is an entire unit interval (or more) on screen, the mask will continually fail. Consider the example in Figure 3. We have what seems to be a perfectly good auto mask drawn on the screen. However, in this case, we only have a single run’s worth of waveforms.

 

 

Figure 3. An auto mask of multiple unit intervals

 

Figure 4 shows what happens when we do multiple runs. Suddenly, we get a whole bunch of errors. What happened? The key point is: unless we are zoomed in on a very small part of the waveform, there are many points where either a rising edge or falling edge are valid. This means that the Auto Mask finds a great number of errors when there is actually none at all, in this case.

 

Figure 4. Showing the same auto mask as Figure 3 with multiple runs

 

This problem is just as bad with a single unit interval, which is known as an eye diagram. Eye diagrams are a very powerful method to determine signal integrity but auto mask will never work. An example of an eye diagram is shown in Figure 5. The middle opening is known as the “eye”. The size of this opening is one key measure of signal integrity.

 

Figure 5. Eye diagrams show a single unit interval

 

Creating your own mask with Draw Mask

For a long time, users who needed to create their own mask needed to use Excel to create a mask file. This is very error prone and tedious. However, starting in Version 6.0, Keysight Infiniium oscilloscopes provide a Draw Mask dialog. An example is shown in Figure 6. There is the option for manual creation of polygons which are used to indicate the areas where the waveform should not go. Much of the focus is typically on the eye area itself, although users are free to put the polygons anywhere.

 

 

Figure 6. Manually creating a mask using the Draw Mask dialog

 

Up to eight polygons are allowed, and each polygon can have up to fifteen points. Each point can be editable simply by clicking it and moving it to a new location. Moving the entire polygon at once is done by clicking anywhere inside it and dragging it to a new location. The bottom line is that users can create a new mask visually which is far less error-prone because they can see the existing waveforms in the editor. This is a massive improvement over creating mask files by hand in Excel!

 

The readout at the bottom shows the numeric values for each of the points in the selected polygon. These can be manually edited as shown in Figure 7. This is useful because sometimes users have specific values in mind to match a specification.

 

Figure 7. Entering a specific value for a point in a polygon

 

Each polygon can consist of as few as three points or as many as fifteen points. The reason to allow this flexibility is to give users the option to be as accurate as they desire. Fifteen points are the most accurate, but it also takes the most time to create.

 

An example of a complex multi-polygon mask is shown Figure 8. Notice the use of a different numbers of points. Creating this example took me only a couple of minutes. With Excel, it could have taken hours to get right.

 

Figure 8. Creating a complex mask in the Draw Mask dialog

 

The final result in the oscilloscope is shown in Figure 9. In this case, the waveform is well behaved and we have no errors.


Figure 9a. An eye diagram with a complex mask

 

Figure 10 shows the same mask with some errors. Notice the edges of the mask are now in a bright red. These are waveforms which violate the mask. Optionally, users can stop as soon as a failure occurs to allow them to see what led up to the problem.

 

Figure 9b. An eye diagram with a complex mask

 

It gets better: Automatic mask creation

While creating polygons is pretty fast and definitely a big improvement over manually creating mask files, users asked for even more efficiency. Can’t the oscilloscope automatically create shapes based upon an eye diagram? An example of doing this is shown in Figure 10. To use “Auto Eye,” users need to specify a tolerance and the maximum number of points and then click in the region where they want the shape created.  These shapes are still editable so that users can tweak them if desired. But most of the time the points are fairly close to the desired location so the amount of editing isn’t large. The key point of Auto Eye is not that it always gets the points exactly right but rather that it reduces the amount of editing by putting the points in approximately the correct location. That way, users might only have to adjust a few points of the polygon instead of moving all of them.

 

Figure 10. Automatic creation of polygons for an eye diagram

 

Mask testing transformed

Mask testing has come a long way! Because of enhancements such as Keysight’s Draw Mask dialog, it is now an analysis that can be used on any waveform. This means mask testing has moved from a relatively small niche to a fast and effective tool that should be used early in the testing process to quickly detect errors. Debugging electronics is challenging and can often be time-consuming, but mask testing is now a huge asset to oscilloscope users.

 

 

About the Author

Doug Beck is an Expert Usability Engineer with Keysight Technologies focused on oscilloscopes. He holds a PhD in Industrial & Operations Engineering from the University of Michigan and has 12 patents.

Pushing the limits of Ethernet speeds and NRZ

The increased demand for more data from consumers and businesses has required faster and faster Ethernet technologies. To that end, a shift from NRZ(Non-Return to Zero or PAM2) to PAM4 (Pulse Amplitude Modulation 4) has now become the answer to increasing data throughput. PAM has never been used at the high speeds we are seeing today and this is where the challenge begins. Margins have shrunk, edges are more critical and error rates increase when working with PAM4.

 

PAM has never been used at the high speeds we are seeing today and this is where the challenge begins. 

 

Pulse Amplitude Modulation (PAM4) vs. Non-Return to Zero (NRZ/PAM2)

First let’s look at the differences between NRZ (PAM2) and PAM4. NRZ(PAM2) (Fig 1) has two amplitude levels with 1 bit of information per symbol. The real-time eye shows one distinct eye opening with one distinct rise and fall time.

 

            

Figure 1: NRZ signal and Eye Diagram

 

The PAM4 (Fig 2) by contrast has four distinct amplitude levels and two bits of information per symbol. The real-time eye shows three distinct eye openings with six rise and fall times.

 

             

Figure 2: PAM4 signal and Eye Diagram

 

Data Throughput

PAM4 has four amplitude levels with two bits of information per symbol, while NRZ(PAM2) has one bit of information per symbol. Symbols are expressed in terms of baud (Bd). So PAM4 has twice the throughput for the same baud rate of NRZ.

 

 

Figure 3: Same data expressed as NRZ vs. PAM4

 

With standard (linear) PAM4 we have the potential for two transitions at the same time. These transitions can cause two bit errors per symbol. If we convert standard PAM4 to gray code, we can cut our bit error down to one bit error per symbol. This reduces our overall bit error in half.

 

 

Figure 4: Standard (Linear) PAM4 converted to Gray code

 

Clock Skew and Eye Vertical alignment

Clock skew can have a significant impact on the vertical alignment of PAM4 eyes. When the upper and lower eyes are skewed to the left relative to the middle eye as shown below, this indicates that the most significant bit (MSB) is early with respect to the least significant bit (LSB). We can imagine a mask (shown in green in Fig 5) that sets a margin for what skew is acceptable. In this case, a quarter unit interval (UI) mask might be a good starting point. As the eyes drift, further past center alignment of the middle eye, symbol errors (SER) will increase and data recovery suffers.

 

 

 

Figure 5: Eye Skew – skew between top, bottom eyes relative to the middle eye

 

Non-linearity and Amplitude compression

Non-linearity and amplitude compression are also an issue when rise/fall times differ between the upper (MSB) and lower (LSB) eyes or voltage amplitude for various levels are too high or low. In Fig 6 the lower eye is compressed with the upper eye dilated.

 

 

Figure 6: Non-linearity and amplitude compression can also effect SER.

 

To measure the transmitter linearity of the PAM4 signal, we measure the mean signal level transmitted for each PAM4 symbol. Symbol levels are derived from the voltage levels V0, V1, V2 and V3 as shown in Fig 7. Vmid is the halfway point between V0 and V3. To determine how far off we are from the ideal symbol level, we can calculate the effective symbol level (ES) as shown below. Ideally, we would like both ES1 and ES2 to be 1/3 so that the eyes are perfectly symmetrical and our voltage levels are aligned. This all brings us to level separation mismatch ratio (RLM). Ideally, we would want our level separation mismatch ratio to equal 1. This would imply that all our PAM4 eyes are symmetrical and open.

 

Figure 7: Transmitter linearity measurement and level separation mismatch ratio RLM

 

Forward Error Correction (FEC)

 With all that can go wrong with the signal from the transmitter though the channel and to the receiver, how can we correct for errors along the way? This is where error correction can help to correct at least some of the errors. With error correction, we have the advantage that we don’t need to retransmit the data again. The Reed-Solomon error correction scheme has properties that are well suited for PAM4, as it can correct for burst errors shown below. Reed Solomon error correction treats symbols the same no matter how many bits are contained in the symbol. So, with PAM4 having two bits per symbol compared to NRZ only having one bit per symbol doesn’t penalize the efficiency of the Reed Solomon correction scheme. Of course, with any error correction scheme, parity symbols are sent with the data and this will add to the overhead in our data stream.

 

 

Figure 8: Burst Errors showing corrupted bits

 

Conclusion

What PAM4 brings to the table is an opportunity to overcome NRZ (PAM2) speed challenges while doubling the data throughput at the same baud rate. Keeping PAM4 eyes open, symmetrical and un-skewed when transmitting data brings new challenges to designers. Understanding how these characteristics play together is important for successful implementation of PAM4 solutions. This introduction to PAM4 shows just a taste for what PAM4 has to offer and some of the challenges that must be overcome.

 

Understanding how these characteristics play together is important for successful implementation of PAM4 solutions. This introduction to PAM4 shows just a taste for what PAM4 has to offer and some of the challenges that must be overcome.

 

Keysight has been a key contributor to IEEE802.3bs and other Ethernet standards that use PAM4 and understands the test requirements. New test challenges with PAM4 can be overcome, and Keysight has the solutions to overcome these challenges. Keysight has other comprehensive test solutions from design simulation to physical layer testing that includes transmitter, receiver and channel for PAM4.

 

For more information go to https://www.keysight.com/find/PAM4

 

Request Free Master 400G poster

 

PAM4 Applications

N8827A PAM-4 Analysis Software for Infiniium Real-time Oscilloscopes

N8836A PAM-4 Measurement Application for Ethernet and OIF-CEI for Real-Time Oscilloscopes

86100D-9FP PAM-N Analysis Software for 86100D DCA-X Oscilloscopes

N1085A PAM-4 Measurement Application for Ethernet and OIF-CEI

Hacking the specs

Everyone loves a bargain. And who doesn’t love a hacked oscilloscope? Well, it would be pretty odd for an oscilloscope company to teach you how to hack your own hardware. Besides, that’s already been done (Fig. 1). So, I’m coining a new term: “spec hack.”

 

Webster’s dictionary will one day define it like this:

 

Spec hack  (/spek’hak/), n. When an engineer uses expert-level knowledge of their test equipment to achieve performance above and beyond typical expectations of said equipment. <Thanks to a spec hack of the flux capacitor, Doc Brown discovered he only needed to go 77 miles per hour to travel through time.>

 

Today’s spec hack will look at the built-in frequency counter on an InfiniiVision 1000 X-Series oscilloscope.

 

You may think that a 100 MHz oscilloscope will only let you see signals up to 100 MHz – but that’s not actually true. Why? Oscilloscope bandwidth isn’t as straightforward as the labeled spec.

 

Figure 1: just a few days after its release, the EEVBlog YouTube channel posted an oscilloscope hack to double the bandwidth of the 

1000 X-Series.

 

You may think that a 100 MHz oscilloscope will only let you see signals up to 100 MHz – but that’s not actually true. Why? Oscilloscope bandwidth isn’t as straightforward as the labeled spec.

 

Oscilloscope Bandwidth brush up

To fully understand how far you can push your frequency counter, you must first understand how your oscilloscope’s bandwidth works. If you are confident that you know all about bandwidth, feel free to skip this next little section. If not, get ready to have your mind blown (or at least maybe learn something new).

 

Bandwidth

Essentially, if you have a 100 MHz oscilloscope bandwidth, it means you can view a sine wave (or frequency components of a non-sine wave) of 100 MHz with ≤ 3 dB of attenuation.

 

But, here’s the main take away – bandwidth is all about signal attenuation, not just about the frequencies you can or can’t see (Fig. 2).

 

Figure 2: A Keysight 6000 X-Series Oscilloscope demonstrates what a 2.5 Gbps waveform looks like at varying bandwidths.

Even at 200 MHz, there is still a visible signal.

 

Usually this won’t matter for your day-to-day oscilloscope usage. You may see round corners on what should be a crisp square wave, but it probably won’t change how you use your scope. But, when you’re using a built-in frequency counter, it can be used to your advantage.

 

But, when you’re using a built-in frequency counter, it can be used to your advantage.

 

How a frequency counter works

To understand why this effect can be advantageous, you need to understand how a frequency counter works. It’s called a frequency counter because it literally counts. It counts the number of edges found over a specific amount of time, called the gate time. The frequency is calculated like this:

 

Frequency = Number of pulses/Gate time

 

From a circuitry perspective, the counter is simply a comparator (to identify signal edges) and a microcontroller to count the output and display the results (Fig. 3) As it turns out, oscilloscopes already have this infrastructure inside their trigger systems.

 

Figure 3: An old-school HP frequency counter’s nixie tube display

 

Why oscilloscopes make great frequency counters

As luck would have it, the trigger circuitry of a scope often has comparators built into the signal path (think “edge trigger”). With some planning, it’s not difficult for oscilloscope designers to include a frequency counter built into the oscilloscope. It may sometimes require extra hardware, but the essentials already exist.

 

The most important specification of a counter is accuracy - the higher the precision of the time base, the more accurate the counter. Oscilloscopes also have to have a highly accurate time base, so a built in counter can just use the scope’s clock.

 

Finally, an oscilloscope’s trigger circuitry typically has its own special signal path designed specifically to extract the core signal and block out noise and unwanted frequency components. Unlike the oscilloscope’s acquisition circuitry, the trigger circuitry doesn’t need to recreate the signal with high accuracy, it only needs to do a fantastic job of finding edges. So, a frequency counter can use a scope’s trigger signal path instead of the acquisition signal path and get a higher fidelity edge.

 

The Spec Hack

Let’s put it all together. So far we’ve learned a few things:

 

  1. You can see signals (or signal edges) higher than the bandwidth of your oscilloscope, but it may have an attenuated amplitude.
  2. Frequency counters just need to count edges; they don’t care very much about the amplitude of the signal.
  3. Oscilloscopes have a dedicated, specially conditioned signal path dedicated to identifying edges.

 

So, a frequency counter built into an oscilloscope can measure frequencies higher than the bandwidth of the oscilloscope. The question is, how much higher?

 

So, a frequency counter built into an oscilloscope can measure frequencies higher than the bandwidth of the oscilloscope. The question is, how much higher?

 

One of the perks of working at Keysight is that that’s an easy question to answer. I pulled out my 100 MHz Keysight 1000 X-Series low-cost oscilloscope and a grossly unnecessary Keysight 67 GHz PSG (because hey, why not?) (Fig. 4) and ran a frequency sweep to see just how fast of a signal the oscilloscope’s frequency counter could measure.

 

Figure 4: a PSG producing a 529 MHz sine wave

 

The results blew me away!

 

The 100 MHz oscilloscope’s counter was able to measure up to 529 MHz! That’s over 5x the bandwidth of the oscilloscope (Fig. 5).

 

Figure 5: A screenshot of the frequency counter measuring 529 MHz. 

 

The lesson? Know your equipment!

It’s always fun to find a little, hidden gem in your test equipment. Sometimes it’s an Easter egg mini game hidden away in a secret menu; sometimes it’s a measurement you had no idea you could make. Having a good understanding of the fundamentals of the equipment you use will not only help you make better, more accurate measurements but also help you avoid any traps that might lead you down the wrong test path.

 

Having a good understanding of the fundamentals of the equipment you use will not only help you make better, more accurate measurements but also help you avoid any traps that might lead you down the wrong test path.

 

Are there any spec hacks that you’ve found? Be sure to let me know in the comments below or on Twitter (@Keysight_Daniel). Happy testing!