Post Go back to editing

Scalability of the IP core "util_adrv9009_xcvr" from managing 8 SoCs ADRV9009 to 4 SoCs ADRV9009

Category: Hardware
Product Number: util_adrv9009_xcvr

Good morning everyone,

I have taken over a project that features a JESD204 link with 16 channels (managing 8 ADRV9009 chips).

This project originated from Analog Devices' reference design, which was modified to work with 16 channels.

We are now creating a new project with a similar architecture, but the number of channels is reduced from 16 to 8, meaning we will only control 4 ADRV9009 chips.

The FPGA target is XCZU9EG-FFVB1156-2-e

Thus, the previous JESD204 parameters, which were:

  1. Number of Lanes (L) = 16
  2. Number of Converters (M) = 32
  3. Bits per Sample (N') = 16
  4. Converter Resolution (N) = 16
  5. Samples per Frame (S) = 1
  6. Octets per Beat = 4

have now been changed to:

  1. Number of Lanes (L) = 8
  2. Number of Converters (M) = 16
  3. Bits per Sample (N') = 16
  4. Converter Resolution (N) = 16
  5. Samples per Frame (S) = 1
  6. Octets per Beat = 4

The outputs of the IP core "util_adrv9009_xcvr" (such as tx_0_p, tx_0_n, etc.) are connected to specific pins that are fixed at the PCB level and route the signals to the different ADRV9009 chips. Here’s the current mapping:

#set_property -dict {PACKAGE_PIN K6} [get_ports tx_data_a_p[0]] (tx_0_p)
#set_property -dict {PACKAGE_PIN K5} [get_ports tx_data_a_n[0]] (tx_0_n)
#set_property -dict {PACKAGE_PIN H6} [get_ports tx_data_a_p[1]] (tx_1_p)
#set_property -dict {PACKAGE_PIN H5} [get_ports tx_data_a_n[1]] (tx_1_n)

#set_property -dict {PACKAGE_PIN K2} [get_ports rx_data_a_p[0]] (rx_0_n)
#set_property -dict {PACKAGE_PIN K1} [get_ports rx_data_a_n[0]] (rx_0_n)
#set_property -dict {PACKAGE_PIN J4} [get_ports rx_data_a_p[1]] (rx_1_p)
#set_property -dict {PACKAGE_PIN J3} [get_ports rx_data_a_n[1]] (rx_1_n)

With this configuration, the project synthesizes successfully. However, I need to remap the tx_0_p, tx_0_n, etc., to different pins, such as the tx_data_c_p pins, and when I make this change, the project no longer implements, and I get the following error:

[Place 30-510] Unroutable Placement! A GTHE_COMMON / GTHE_CHANNEL clock component pair is not placed in a routable site pair. The GTHE_COMMON component can use the dedicated path between the GTHE_COMMON and the GTHE_CHANNEL if both are placed in the same clock region. If this suboptimal condition is acceptable for this design, you may use the CLOCK_DEDICATED_ROUTE constraint in the .xdc file to demote this message to a WARNING. However, the use of this override is highly discouraged. These examples can be used directly in the .xdc file to override this clock rule.
< set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets i_system_wrapper/system_i/util_adrv9009_xcvr/inst/i_xcm_0/qpll2ch_clk] >

i_system_wrapper/system_i/util_adrv9009_xcvr/inst/i_xcm_0/i_gthe4_common (GTHE4_COMMON.QPLL0OUTCLK) is provisionally placed by clockplacer on GTHE4_COMMON_X1Y3
i_system_wrapper/system_i/util_adrv9009_xcvr/inst/i_xch_0/i_gthe4_channel (GTHE4_CHANNEL.QPLL0CLK) is locked to GTHE4_CHANNEL_X1Y5
i_system_wrapper/system_i/util_adrv9009_xcvr/inst/i_xch_1/i_gthe4_channel (GTHE4_CHANNEL.QPLL0CLK) is locked to GTHE4_CHANNEL_X1Y4
i_system_wrapper/system_i/util_adrv9009_xcvr/inst/i_xch_2/i_gthe4_channel (GTHE4_CHANNEL.QPLL0CLK) is locked to GTHE4_CHANNEL_X0Y9
i_system_wrapper/system_i/util_adrv9009_xcvr/inst/i_xch_3/i_gthe4_channel (GTHE4_CHANNEL.QPLL0CLK) is locked to GTHE4_CHANNEL_X0Y8

I understand that there is an issue with the GTHE components, but I am unsure how to instruct Vivado which specific GTHE to use for the designated output pins. The "util_adrv9009_xcvr" IP core does not offer many configurable options beyond a few parameters related to frequency management. How should I proceed to resolve this issue?

I appreciate your help in advance.

Thread Notes

  • Here is the list of GTHE4 modules being used, along with their corresponding TX and RX data pins. The issue is that I am unable to manually select the GTHE4 modules utilized in the IP core, and furthermore, the associated clocks for the Common block are unclear.

  • util_advr9009_xcvr
    For 8 advr9009 Config. For 4 advr9009 Config.
    GTHE4 CHANNEL GTH4 COMMON GTHE4 CHANNEL GTH4 COMMON
    JESD204 LINE SYNTHESIS LEVEL PLACE & ROUTE LELVEL SYNTHESIS LEVEL PLACE & ROUTE LELVEL
    0 CHANNELL X1Y9 COMMON X1Y2 CHANNELL X1Y4 ?
    1 CHANNELL X1Y8 COMMON X1Y2 CHANNELL X1Y5 ?
    2 CHANNELL X1Y11 COMMON X1Y2 CHANNELL X0Y8 ?
    3 CHANNELL X1Y10 COMMON X1Y2 CHANNELL X0Y9 ?
            ?
    4 CHANNELL X1Y5 COMMON X1Y1 CHANNELL X1Y10 ?
    5 CHANNELL X1Y4 COMMON X1Y1 CHANNELL X1Y11 ?
    6 CHANNELL X1Y7 COMMON X1Y1 CHANNELL X0Y14 ?
    7 CHANNELL X1Y6 COMMON X1Y1 CHANNELL X0Y15 ?
             
    8 CHANNELL X0Y13 COMMON X0Y3    
    9 CHANNELL X0Y12 COMMON X0Y3    
    10 CHANNELL X0Y15 COMMON X0Y3    
    11 CHANNELL X0Y14 COMMON X0Y3    
             
    12 CHANNELL X0Y9 COMMON X0Y2    
    13 CHANNELL X0Y8 COMMON X0Y2    
    14 CHANNELL X0Y11 COMMON X0Y2    
    15 CHANNELL X0Y10 COMMON X0Y2    
  • Hi,

    Apologies for the delayed reply.

    Could you take a look at page 29 and page 15? https://docs.amd.com/v/u/en-US/ug578-ultrascale-gty-transceivers 

    And check carefully what quad you select since the clock and the data should be as close as possible.

    Using the GT Wizard from Vivado GUI might help you to choose the right pins/quads (looks like this https://docs.amd.com/viewer/attachment/U2GQR49CyPTcdLAya2_2Xg/ppnGOig73uz9Jtlw8tHPkw)

    Here we have an example with 4 devices: https://github.com/analogdevicesinc/hdl/blob/main/projects/adrv9009zu11eg/adrv2crr_fmcomms8/system_project.tcl 

    Best regards,
    Iulia

  • i need to force the util_adrv9009_xcvr ip core to use the following GTHE4 Channels:

    X1Y11; X1Y10 => QUAD X1Y2 => external clock coming to CLK0

    X1Y5; X1Y4    => QUAD X1Y1 => clock from CLK0 of QUAD X1Y2

    X0Y13; X0Y12 => QUAD X0Y3 => external clock coming to CLK0

    X0Y9; X0Y8    => QUAD X0Y2 => clock from CLK0 of QUAD X0Y3

    how i can achieve this configuration?

    thanks.

  • Hi,

    It is just a mater of constraints, chose the corresponding pins.
    https://github.com/analogdevicesinc/hdl/blob/main/projects/adrv9009zu11eg/adrv2crr_fmcomms8/fmcomms8_constr.xdc#L29-L44

    As Iulia pointed out, you should check the transceiver user guide, You will find there that the reference clock from a quad can drive only the adjacent quads.

    Andrei

  • The UTIL_ADXCVR is a wrapper that instantiates transceiver quads, which are also full banks. For 4 channels it instantiates a quad common. The transceiver quads have fixed pinouts for each specific channel. When a channel is instantiated, the pinout for both TX and RX are fixed, there is not flexibility as compared with other general FPGA pins.

    When doing the PCB design, ideally you should use pins from the same quad and adjacent quads. The reference clock should come from the dedicated clock pins of the quad. Depending on the lane rate, you can use also clocks from the quad above and quad below or quad 2 positions above and below.

    Can you tell me what banks you have in your design and where does the clock comes from?

  • Certainly! Below is the FPGA pin configuration (xczu9eg-ffvb1156-2-e) used in conjunction with the GTH transceivers for the initial project, which featured a JESD204 interface driving 8 SoCs. This configuration has been tested and is functioning correctly:

    JESD204 DATA FPGA PIN

    FPGA BANK

    GTHE4

    QUAD

    CLOCK

    Util Xcvr TX0

    Util Xcvr RX0

    229

    X1Y9

    X1Y2

    REF_CLK_A

    Pin P => G8

    Pin N =>G7

    P

    N

    P

    N

    K6

    K5

    K2

    K1

    Util Xcvr TX1

    Util Xcvr RX1

    X1Y8

    P

    N

    P

    N

    H6

    H5

    J4

    J3

    Util Xcvr TX2

    Util Xcvr RX2

    X1Y11

    P

    N

    P

    N

    G4

    G3

    H2

    H1

    Util Xcvr TX3

    Util Xcvr RX3

    X1Y10

    P

    N

    P

    N

    F6

    F5

    F2

    F1

    Util Xcvr TX4

    Util Xcvr RX4

    228

    X1Y5

    X1Y1

    P

    N

    P

    N

    R4

    R3

    T2

    T1

    Util Xcvr TX5

    Util Xcvr RX5

    X1Y4

    P

    N

    P

    N

    P6

    P5

    P2

    P1

    Util Xcvr TX6

    Util Xcvr RX6

    X1Y7

    P

    N

    P

    N

    N4

    N3

    M2

    M1

    Util Xcvr TX7

    Util Xcvr RX7

    X1Y6

    P

    N

    P

    N

    M6

    M5

    L4

    L3

    Util Xcvr TX8

    Util Xcvr RX8

    130

    X0Y13

    X0Y3

    REF_CLK_B

    Pin P => G27

    Pin N =>G28

    P

    N

    P

    N

    F29

    F30

    E31

    E32

    Util Xcvr TX9

    Util Xcvr RX9

    X0Y12

    P

    N

    P

    N

    D29

    D30

    D33

    D34

    Util Xcvr TX10

    Util Xcvr RX10

    X0Y15

    P

    N

    P

    N

    B29

    B30

    C31

    C32

    Util Xcvr TX11

    Util Xcvr RX11

    X0Y14

    P

    N

    P

    N

    A31

    A32

    B33

    B34

    Util Xcvr TX12

    Util Xcvr RX12

    129

    X0Y9

    X0Y2

    P

    N

    P

    N

    K29

    K30

    L31

    L32

    Util Xcvr TX13

    Util Xcvr RX13

    X0Y8

    P

    N

    P

    N

    J31

    J32

    K33

    K34

    Util Xcvr TX14

    Util Xcvr RX14

    X0Y11

    P

    N

    P

    N

    M29

    M30

    M33

    M34

    Util Xcvr TX15

    Util Xcvr RX15

    X0Y10

    P

    N

    P

    N

    G31

    G32

    E33

    E34

    However, in the new project, the board will remain unchanged, but the FPGA configuration will be modified. The desired configuration is as follows:

    JESD204 DATA FPGA PIN

    FPGA BANK

    GTHE4

    QUAD

    CLOCK

    229

    X1Y9

    X1Y2

    REF_CLK_A

    Pin P => G8

    Pin N =>G7

    X1Y8

    Util Xcvr TX0

    Util Xcvr RX0

    X1Y11

    P

    N

    P

    N

    G4

    G3

    H2

    H1

    Util Xcvr TX1

    Util Xcvr RX1

    X1Y10

    P

    N

    P

    N

    F6

    F5

    F2

    F1

    Util Xcvr TX2

    Util Xcvr RX2

    228

    X1Y5

    X1Y1

    Clock coming from QUAD X1Y2 (REF_CLK_A)

    P

    N

    P

    N

    R4

    R3

    T2

    T1

    Util Xcvr TX3

    Util Xcvr RX3

    X1Y4

    P

    N

    P

    N

    P6

    P5

    P2

    P1

    X1Y7

    X1Y6

    130

    X0Y13

    X0Y3

    REF_CLK_B

    Pin P => G27

    Pin N =>G28

    X0Y12

    Util Xcvr TX4

    Util Xcvr RX4

    X0Y15

    P

    N

    P

    N

    B29

    B30

    C31

    C32

    Util Xcvr TX5

    Util Xcvr RX5

    X0Y14

    P

    N

    P

    N

    A31

    A32

    B33

    B34

    Util Xcvr TX6

    Util Xcvr RX6

    129

    X0Y9

    X0Y2

    Clock coming from QUAD X0Y3 (REF_CLK_B)

    P

    N

    P

    N

    K29

    K30

    L31

    L32

    Util Xcvr TX7

    Util Xcvr RX7

    X0Y8

    P

    N

    P

    N

    J31

    J32

    K33

    K34

    X0Y11

    X0Y10

    But Vivado gives me an error when I try to synthesize and implement it this way... assuming I have configured the util_xcvr block correctly.

  • We need some tests, but for this pinout you'll need to instantiate all 4 quads and it will complicated to use the other transceivers for other applications.

    Is there any way to change the PCB pinout ?

    Regards,

    Adrian

  • Unfortunately, the board, and therefore the PCB, cannot be modified, meaning the routing of the various chips on the PCB cannot be changed. We need to explore an FPGA-based solution, where feasible, to implement the configuration outlined in the table.

  • One option would be to instantiate the util_adxcvr similar with the  older solution (16 channels) and only connect the pins you are using. Not sure how the tool will handle the other channels, so needs some exploration.

    Another option would be to split the design similar with what we've done here: AD9208-DUAL-EBZ HDL reference design [Analog Devices Wiki] so you'll have a physical layer per adjacent banks. in this case you can probably instantiate the number of lanes to 6 per util_adxcvr, which will allow you to have two quad common and 4 channels.

    On top of the vivado implication, there may be also software implications when you bring up the system.

    Regards,

    Adrian