Post Go back to editing

JESD204B Link between ZCU102 & AD9154-FMC-EBZ stuck in CGS

Category: Hardware
Product Number: AD9154-FMC-EBZ

Hello,

I am trying to bring up a JESD204B Link between a ZCU102 (TX) and AD9154 on the FMC-EBZ card (RX.) I am using the following parameters: LMFS = 8411, K=32, N=NP=16, subclass 0.

As a reference for the FPGA design, I took the analogdevices/hdl/projects/dac_fmc_ebz/zcu102 design and configured it for mode 0 with the AD9154. For the clocks, I am giving a 93.75MHz input on tx_refclk. According to the documentation for the transport and transmit IP's, the device clock that comes in needs to be 1/40 of the lane rate. From simulation I am observing that the lane rate is 1.875Gbps and thus I divide the output tx_out_clk_0 from the util_dac_jesd204_xcvr by 2 using a clocking wizard before feeding it into the link clock and device clock on the Transport and Transmit blocks.

I ran the simulation in which I configure the TX pipeline IP's according to the example testbench in analogdevices/testbenches/jesd_loopback project. Since my simulation only contains the TX side with no receiver, I externally toggle the SYNC line. When doing this I can see observe the TX side transition from stages 0 to 3 in the link procedure. I can see the lanes transmit the /K28.5/ character and then, if I manually toggle the SYNC line low and then sometime later high, the transceiver enters the ILAS stage and then the DATA stage.

On the DAC side, I am configuring the AD9154 with the following settings: 750MHz fed to the CLK_N/_P input, internal PLL off. The clocks comes from the onboard AD9516 which gives the 750MHz to the DAC and the 93.75MHz to the FPGA.

The issue I am observing is that on real hardware, the FPGA never progresses past the CGS stage. I verify the stage by looking at register 0x280 in the Transmit peripheral. Here are the register dumps from all 3 of the IP's (Transport, Transmit and Transceiver):

Transport IP Transmit IP Transceiver IP
0x0000: 0x00090162 0x0000: 0x00010361 0x0000: 0x00110161
0x0004: 0x00000000 0x0004: 0x00000000 0x0004: 0x00000000
0x0008: 0x00000000 0x0008: 0x00000000 0x0008: 0x00000000
0x000c: 0x00000001 0x000c: 0x32303454 0x0010: 0x00000001
0x0010: 0x00000000 0x0010: 0x00000008 0x0014: 0x00000001
0x001c: 0x03041403 0x0014: 0x00000002 0x001c: 0x03041403
0x0040: 0x00000003 0x0018: 0x00000101 0x0020: 0x00001003
0x0044: 0x00000000 0x0080: 0x00000000 0x0024: 0x00180108
0x0048: 0x00000000 0x0084: 0x00000000 0x0040: 0x00000000
0x004c: 0x00000000 0x0088: 0x00000000 0x0044: 0x00000000
0x0050: 0x00000000 0x00c0: 0x00000000 0x0048: 0x00000000
0x0054: 0x00007803 0x00c4: 0x00000000 0x0060: 0x00000000
0x0058: 0x00000004 0x00c8: 0x00007803 0x0064: 0x00000000
0x005c: 0x00000001 0x00cc: 0x00000000 0x0068: 0x00000000
0x0060: 0x00000000 0x0100: 0x00000001 0x0080: 0x00000000
0x0068: 0x00000000 0x0104: 0x00000000 0x00a0: 0x00000000
0x0070: 0x00000000 0x0108: 0x00000001 0x00a4: 0x00000000
0x0074: 0x00020000 0x0200: 0x00000000 0x00a8: 0x00000000
0x0078: 0x00000000 0x0210: 0x0000001f 0x00ac: 0x00000000
0x007c: 0x00000000 0x0214: 0x00000000 0x00b0: 0x00000000
0x0088: 0x00000000 0x0218: 0x00000000 0x00b4: 0x00000000
0x00a0: 0x00000004 0x021c: 0x00000000 0x00b8: 0x00000000
0x00b8: 0x00000000 0x0240: 0x00000000 0x00bc: 0x00000000
0x00bc: 0x00000000 0x0244: 0x00000003 0x00c0: 0x00000008
0x0200: 0x00000000 0x0248: 0x00000000 0x00c4: 0x00000000
0x0204: 0x00000001 0x0280: 0x00000001 0x00c8: 0x00000000
0x0240: 0x01010804 0x0140: 0x00000352
0x0244: 0x00001010 0x0180: 0x00000000
0x0248: 0x00000000 0x0184: 0x00000000
0x0250: 0x00000000
0x0400: 0x00000000
0x0404: 0x00000000
0x0408: 0x00000000
0x040c: 0x00000000
0x0410: 0x00000000
0x0414: 0x00000000
0x0418: 0x00000002
0x041c: 0x00000000
0x0420: 0x01001010
0x0424: 0x00010001
0x0428: 0x00000000

After reading these registers and configuring the FPGA PL, the clock AD9516 clock chip and the DAC, I took a high speed scope and probed the physical lanes coming into the DAC chip on the AD9154-DAC-FMC board. I probed the SYNC line and saw that it was indeed held low at around 200mV. I then probed the lanes and saw that they did not look right (there was no /K28.5/ character and the lane rate did not look correct either.) On the photos below, image 0 shows lane 0, image 1 shows lane 1 and image 2 shows lane 6 - the rest of the lanes look very similar to lane 0. From the simulation, the link is supposed to be running at 1.875Gbps but none of the images seem to show this. The timescale on the images is 500ps/box.

Here are the util_adxcvr_v1_0 settings that I am using:

It seems from the images that something about the transceiver isn't set up properly but it is not obvious to me what it is. In the transceiver, register 0x14 set to 1 indicates that PLL locked and things should be running but that does not seem to be the case. Should I be able to see the /K28.5/ characters clearly using an oscilloscope? Is there something else I can look at to debug this and understand the difference between the simulation and reality?

My understanding is that if the RX side keeps the SYNC low then the transmitter should be sending a continuous stream of /K28.5/ characters which I should be able to observe, right?

Thank you for your help,

svv9

Top Replies

    •  Analog Employees 
    Jun 9, 2022 in reply to svv9 +1 verified
    I am still not clear on how the lane rate from the adxcvr is controlled, and what determines it

    The lane rate is defined by the  CPLL/QPLL output frequency and the channel divider.

    See the transceiver…

  • Hello,

    What software are you using ?  Are you writing manually the registers ? There should  be also reference software available for the DACs here https://github.com/analogdevicesinc/no-OS/tree/master/projects/ad9172

    You can adapt that for AD9154.

    1. Based on register 0x280 from the transmit IP, the SYNC~ is low (active)  and the state is CGS, meaning the core should send only the /K28.5/ characters.

    2.

    I divide the output tx_out_clk_0 from the util_dac_jesd204_xcvr by 2 using a clocking wizard

    You can divide the clock internally in the XCVR don't need the clocking wizard. Set  the OUTCLK_SEL field of CONTROL (0x20) register of the transceiver IP to 4.    (0x0020: 0x00001004)

    3. I never probed the serial lanes, but since the lane rate is relatively low you should be seeing similar transitions as in the simulation.

    4.

    The clocks comes from the onboard AD9516 which gives the 750MHz to the DAC and the 93.75MHz to the FPGA.

    750 MHz should be your DAC rate, but the sample rate at JESD level should be lower right ?  Are you using interpolation  of  4x  ?  Only in that case I see the lane rate would be 1.875Gbps. Can you confirm ? 

    5. TX_CLK25_DIV seems to be high,   according the user guide should be 4

    The general recommendation is to regenerate the XCVR parameters based on the Xilinx wizard with focus on TX and CPLL parameters. See this page for reference.

    Thank you.

    Laszlo

  • Hi Laszlo,

    Thanks for the quick response and pointers.

    1. To get the register configurations that I want, I used an ADS8 and the AD9154 SPI GUI tool (an old Labview based GUI that comes together with the DPGDownloader) to get a working setup and then get a register dump. I then followed the Example start-up sequence in the AD9154 datasheet and substituted the values from the GUI where necessary. I am not 100% this is correct because I do not know some of the finer parameters in the ADS8 transmitter side versus the example zcu102 project but I thought it would be a decent start. - also, thanks for the tip about the no-os project, I will try to use it.

    2. Thanks for this tip - I will implement this and see if it makes a difference.

    3. Yes, I intentionally made the rates low so that I could look at things with a scope.

    4. I do not know what the sample rate should be here - my plan was to get the setup working and then feed a known pattern from the PL and see what the output frequency would be, and use that to figure out the sample rate... (still learning this stuff and that seemed like a decent approach!)
    I was able to confirm the 1.875Gbps in the simulation, however, so am pretty confident in that number. Does the sample rate need and interpolation to be specified in the transceiver IP configuration somewhere?

    5. In this project I have the util_dac_jesd204_xcvr which I understand is an ADI wrapper for the Xilinx GT wizard? When I re-customize the util_dac_jesd204_xcvr IP I only see the list of parameters that I shared in the original post. My understanding was that these were populated when I built the project in the analogdevices/hdl/projects/dac_fmc_ebz folder with the configuration for AD9154 mode 0 so I thought that these should be correct? I didn't think that I need to manually add any more IP (such as the Transceiver Wizard,) do I?

    thank you for your help,

    svv9

  • Does the sample rate need and interpolation to be specified in the transceiver IP configuration somewhere?

    The data rate at the JESD link endpoints should match. Suppose the lane rate is 1.875Gbps and you have mode LMFS = 8411  it means we have 1.875Gbps * (8 / 4) * (8 / 10)  / 16  =  187.5 MSPS per channel at the input of the JESD link.  If you provide 750 MHz to the DAC it means you need to set x4 interpolation to get up to 750MSPS  (187.5 x 4)

    The transceiver IP is agnostic to this interpolation since this happens outside of the JESD. but at the endpoints of the link the lane rate (data rate) setting must match,  other case you are stuck in CGS.

    n this project I have the util_dac_jesd204_xcvr which I understand is an ADI wrapper for the Xilinx GT wizard?

    The util_dac_jeasd204_xcvr is a wrapper of the GTH transceiver primitives which have a lot of parameters that can be extracted only using the wizard. They are undocumented in the user guide of the transceiver.

    I didn't think that I need to manually add any more IP (such as the Transceiver Wizard,) do I?

    No, just you have to use the wizard to get the parameters. Unfortunately this is still a manual process.

    Thank you,

    Laszlo

  • Laszlo,

    Thank you, this is very insightful - a few things to clarify:

    1. "If you provide 750 MHz to the DAC it means you need to set x4 interpolation to get up to 750MSPS  (187.5 x 4)" - So you mean that the FDAC is the actual Sampling frequency also? That is very good to know, that part was also not clear to me before.

    2. Based on your explanation of the GT Transceiver Wizard, it seems that just building the example project from analogdevices/hdl/projects/dac_fmc_ebz/zcu102 using the "make" command is not enough - you then need to instantiate an Ultrascale+ GT wizard and modify the parameters of the GT's to be correct also. Is that right?

    -svv9

  • So you mean that the FDAC is the actual Sampling frequency also? That is very good to know, that part was also not clear to me before.

    I would confirm that with someone from the High Speed DACs forum and read through the datasheet   since here we support only the FPGA side of  the design and can give only general support about the converters.

    "make" command is not enough -

    The default parameters of the xcvr are set for the highest lane rate,  which are different from your use case so the parameters needs to be updated.

    Alternatively you can instantiate a Xilinx PHY instead of the util_adxcvr, Historically we preferred the util_adxcvr due maintainability and control but the cores should be pin compatible.

    Laszlo

  • Hi Laszlo,

    Ok, that makes sense. I think that I am going to go with the default zcu102 dac-fmc-ebz project and try to get it working with the minimal number of changes for AD9154 and mode 0.

    I looked at the AD9172 no-os project's main.c and can use the ADI IP AXI settings when configuring the IP's.

    One last question - I am still not clear on how the lane rate from the adxcvr is controlled, and what determines it. In the dac-fmc-ebz project the refclk is 385MHz. Is the lane rate automatically set to the input on the cpll/qpll_ref_clk * 40? Or are there internal settings in the IP that set this?
    If it's the latter, then is it entirely set by those cryptic parameters in the adxcvr (the ones that you said are hidden from view with the Xilinx PHY?) So your HDL team derived those values for the project and then, during the build process via make file, those fields are populated with those predetermined values?

    Thanks,
    svv9

  • I am still not clear on how the lane rate from the adxcvr is controlled, and what determines it

    The lane rate is defined by the  CPLL/QPLL output frequency and the channel divider.

    See the transceiver user guide https://docs.xilinx.com/v/u/en-US/ug576-ultrascale-gth-transceivers page 46.

    These settings (CPLL_REFCLK_DIV,CPLL_FBDIV,CPLL_FBDIV_45,TXOUT_DIV) can be controlled through DRP accesses by the software and their initial default values are set through those cryptic parameters.

    Thank you,

    Laszlo