Post Go back to editing

Scalability of the IP core "util_adrv9009_xcvr" from managing 8 SoCs ADRV9009 to 4 SoCs ADRV9009

Category: Hardware
Product Number: util_adrv9009_xcvr

Good morning everyone,

I have taken over a project that features a JESD204 link with 16 channels (managing 8 ADRV9009 chips).

This project originated from Analog Devices' reference design, which was modified to work with 16 channels.

We are now creating a new project with a similar architecture, but the number of channels is reduced from 16 to 8, meaning we will only control 4 ADRV9009 chips.

The FPGA target is XCZU9EG-FFVB1156-2-e

Thus, the previous JESD204 parameters, which were:

  1. Number of Lanes (L) = 16
  2. Number of Converters (M) = 32
  3. Bits per Sample (N') = 16
  4. Converter Resolution (N) = 16
  5. Samples per Frame (S) = 1
  6. Octets per Beat = 4

have now been changed to:

  1. Number of Lanes (L) = 8
  2. Number of Converters (M) = 16
  3. Bits per Sample (N') = 16
  4. Converter Resolution (N) = 16
  5. Samples per Frame (S) = 1
  6. Octets per Beat = 4

The outputs of the IP core "util_adrv9009_xcvr" (such as tx_0_p, tx_0_n, etc.) are connected to specific pins that are fixed at the PCB level and route the signals to the different ADRV9009 chips. Here’s the current mapping:

#set_property -dict {PACKAGE_PIN K6} [get_ports tx_data_a_p[0]] (tx_0_p)
#set_property -dict {PACKAGE_PIN K5} [get_ports tx_data_a_n[0]] (tx_0_n)
#set_property -dict {PACKAGE_PIN H6} [get_ports tx_data_a_p[1]] (tx_1_p)
#set_property -dict {PACKAGE_PIN H5} [get_ports tx_data_a_n[1]] (tx_1_n)

#set_property -dict {PACKAGE_PIN K2} [get_ports rx_data_a_p[0]] (rx_0_n)
#set_property -dict {PACKAGE_PIN K1} [get_ports rx_data_a_n[0]] (rx_0_n)
#set_property -dict {PACKAGE_PIN J4} [get_ports rx_data_a_p[1]] (rx_1_p)
#set_property -dict {PACKAGE_PIN J3} [get_ports rx_data_a_n[1]] (rx_1_n)

With this configuration, the project synthesizes successfully. However, I need to remap the tx_0_p, tx_0_n, etc., to different pins, such as the tx_data_c_p pins, and when I make this change, the project no longer implements, and I get the following error:

[Place 30-510] Unroutable Placement! A GTHE_COMMON / GTHE_CHANNEL clock component pair is not placed in a routable site pair. The GTHE_COMMON component can use the dedicated path between the GTHE_COMMON and the GTHE_CHANNEL if both are placed in the same clock region. If this suboptimal condition is acceptable for this design, you may use the CLOCK_DEDICATED_ROUTE constraint in the .xdc file to demote this message to a WARNING. However, the use of this override is highly discouraged. These examples can be used directly in the .xdc file to override this clock rule.
< set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets i_system_wrapper/system_i/util_adrv9009_xcvr/inst/i_xcm_0/qpll2ch_clk] >

i_system_wrapper/system_i/util_adrv9009_xcvr/inst/i_xcm_0/i_gthe4_common (GTHE4_COMMON.QPLL0OUTCLK) is provisionally placed by clockplacer on GTHE4_COMMON_X1Y3
i_system_wrapper/system_i/util_adrv9009_xcvr/inst/i_xch_0/i_gthe4_channel (GTHE4_CHANNEL.QPLL0CLK) is locked to GTHE4_CHANNEL_X1Y5
i_system_wrapper/system_i/util_adrv9009_xcvr/inst/i_xch_1/i_gthe4_channel (GTHE4_CHANNEL.QPLL0CLK) is locked to GTHE4_CHANNEL_X1Y4
i_system_wrapper/system_i/util_adrv9009_xcvr/inst/i_xch_2/i_gthe4_channel (GTHE4_CHANNEL.QPLL0CLK) is locked to GTHE4_CHANNEL_X0Y9
i_system_wrapper/system_i/util_adrv9009_xcvr/inst/i_xch_3/i_gthe4_channel (GTHE4_CHANNEL.QPLL0CLK) is locked to GTHE4_CHANNEL_X0Y8

I understand that there is an issue with the GTHE components, but I am unsure how to instruct Vivado which specific GTHE to use for the designated output pins. The "util_adrv9009_xcvr" IP core does not offer many configurable options beyond a few parameters related to frequency management. How should I proceed to resolve this issue?

I appreciate your help in advance.

Parents Reply
  • The UTIL_ADXCVR is a wrapper that instantiates transceiver quads, which are also full banks. For 4 channels it instantiates a quad common. The transceiver quads have fixed pinouts for each specific channel. When a channel is instantiated, the pinout for both TX and RX are fixed, there is not flexibility as compared with other general FPGA pins.

    When doing the PCB design, ideally you should use pins from the same quad and adjacent quads. The reference clock should come from the dedicated clock pins of the quad. Depending on the lane rate, you can use also clocks from the quad above and quad below or quad 2 positions above and below.

    Can you tell me what banks you have in your design and where does the clock comes from?

Children
  • Certainly! Below is the FPGA pin configuration (xczu9eg-ffvb1156-2-e) used in conjunction with the GTH transceivers for the initial project, which featured a JESD204 interface driving 8 SoCs. This configuration has been tested and is functioning correctly:

    JESD204 DATA FPGA PIN

    FPGA BANK

    GTHE4

    QUAD

    CLOCK

    Util Xcvr TX0

    Util Xcvr RX0

    229

    X1Y9

    X1Y2

    REF_CLK_A

    Pin P => G8

    Pin N =>G7

    P

    N

    P

    N

    K6

    K5

    K2

    K1

    Util Xcvr TX1

    Util Xcvr RX1

    X1Y8

    P

    N

    P

    N

    H6

    H5

    J4

    J3

    Util Xcvr TX2

    Util Xcvr RX2

    X1Y11

    P

    N

    P

    N

    G4

    G3

    H2

    H1

    Util Xcvr TX3

    Util Xcvr RX3

    X1Y10

    P

    N

    P

    N

    F6

    F5

    F2

    F1

    Util Xcvr TX4

    Util Xcvr RX4

    228

    X1Y5

    X1Y1

    P

    N

    P

    N

    R4

    R3

    T2

    T1

    Util Xcvr TX5

    Util Xcvr RX5

    X1Y4

    P

    N

    P

    N

    P6

    P5

    P2

    P1

    Util Xcvr TX6

    Util Xcvr RX6

    X1Y7

    P

    N

    P

    N

    N4

    N3

    M2

    M1

    Util Xcvr TX7

    Util Xcvr RX7

    X1Y6

    P

    N

    P

    N

    M6

    M5

    L4

    L3

    Util Xcvr TX8

    Util Xcvr RX8

    130

    X0Y13

    X0Y3

    REF_CLK_B

    Pin P => G27

    Pin N =>G28

    P

    N

    P

    N

    F29

    F30

    E31

    E32

    Util Xcvr TX9

    Util Xcvr RX9

    X0Y12

    P

    N

    P

    N

    D29

    D30

    D33

    D34

    Util Xcvr TX10

    Util Xcvr RX10

    X0Y15

    P

    N

    P

    N

    B29

    B30

    C31

    C32

    Util Xcvr TX11

    Util Xcvr RX11

    X0Y14

    P

    N

    P

    N

    A31

    A32

    B33

    B34

    Util Xcvr TX12

    Util Xcvr RX12

    129

    X0Y9

    X0Y2

    P

    N

    P

    N

    K29

    K30

    L31

    L32

    Util Xcvr TX13

    Util Xcvr RX13

    X0Y8

    P

    N

    P

    N

    J31

    J32

    K33

    K34

    Util Xcvr TX14

    Util Xcvr RX14

    X0Y11

    P

    N

    P

    N

    M29

    M30

    M33

    M34

    Util Xcvr TX15

    Util Xcvr RX15

    X0Y10

    P

    N

    P

    N

    G31

    G32

    E33

    E34

    However, in the new project, the board will remain unchanged, but the FPGA configuration will be modified. The desired configuration is as follows:

    JESD204 DATA FPGA PIN

    FPGA BANK

    GTHE4

    QUAD

    CLOCK

    229

    X1Y9

    X1Y2

    REF_CLK_A

    Pin P => G8

    Pin N =>G7

    X1Y8

    Util Xcvr TX0

    Util Xcvr RX0

    X1Y11

    P

    N

    P

    N

    G4

    G3

    H2

    H1

    Util Xcvr TX1

    Util Xcvr RX1

    X1Y10

    P

    N

    P

    N

    F6

    F5

    F2

    F1

    Util Xcvr TX2

    Util Xcvr RX2

    228

    X1Y5

    X1Y1

    Clock coming from QUAD X1Y2 (REF_CLK_A)

    P

    N

    P

    N

    R4

    R3

    T2

    T1

    Util Xcvr TX3

    Util Xcvr RX3

    X1Y4

    P

    N

    P

    N

    P6

    P5

    P2

    P1

    X1Y7

    X1Y6

    130

    X0Y13

    X0Y3

    REF_CLK_B

    Pin P => G27

    Pin N =>G28

    X0Y12

    Util Xcvr TX4

    Util Xcvr RX4

    X0Y15

    P

    N

    P

    N

    B29

    B30

    C31

    C32

    Util Xcvr TX5

    Util Xcvr RX5

    X0Y14

    P

    N

    P

    N

    A31

    A32

    B33

    B34

    Util Xcvr TX6

    Util Xcvr RX6

    129

    X0Y9

    X0Y2

    Clock coming from QUAD X0Y3 (REF_CLK_B)

    P

    N

    P

    N

    K29

    K30

    L31

    L32

    Util Xcvr TX7

    Util Xcvr RX7

    X0Y8

    P

    N

    P

    N

    J31

    J32

    K33

    K34

    X0Y11

    X0Y10

    But Vivado gives me an error when I try to synthesize and implement it this way... assuming I have configured the util_xcvr block correctly.

  • We need some tests, but for this pinout you'll need to instantiate all 4 quads and it will complicated to use the other transceivers for other applications.

    Is there any way to change the PCB pinout ?

    Regards,

    Adrian

  • Unfortunately, the board, and therefore the PCB, cannot be modified, meaning the routing of the various chips on the PCB cannot be changed. We need to explore an FPGA-based solution, where feasible, to implement the configuration outlined in the table.

  • One option would be to instantiate the util_adxcvr similar with the  older solution (16 channels) and only connect the pins you are using. Not sure how the tool will handle the other channels, so needs some exploration.

    Another option would be to split the design similar with what we've done here: AD9208-DUAL-EBZ HDL reference design [Analog Devices Wiki] so you'll have a physical layer per adjacent banks. in this case you can probably instantiate the number of lanes to 6 per util_adxcvr, which will allow you to have two quad common and 4 channels.

    On top of the vivado implication, there may be also software implications when you bring up the system.

    Regards,

    Adrian

  • Regarding the first option, we have already tried implementing a similar solution in Vivado, but it didn't work as expected. Vivado seems to struggle with it, and, from a software perspective, managing it would be overly complicated.

    The second option, splitting the design and following the AD9208-DUAL-EBZ HDL reference design approach, could be a viable path. However, this would require significant software development effort to accommodate the change, so it’s important to assess whether we can handle the workload involved.

    What I would like to understand more clearly is whether it is possible to synthesize and implement the project with the configuration we have in mind for the util_xcvr, as summarized in the simplified table below:

    • X1Y11; X1Y10 => QUAD X1Y2 => external clock to CLK0
    • X1Y5; X1Y4 => QUAD X1Y1 => clock from CLK0 of QUAD X1Y2
    • X0Y13; X0Y12 => QUAD X0Y3 => external clock to CLK0
    • X0Y9; X0Y8 => QUAD X0Y2 => clock from CLK0 of QUAD X0Y3

    I need to determine whether this configuration is feasible, as it directly impacts the project. If this approach is not possible, we would likely need to redesign the PCB, which would lead to significant development costs and delays. The sooner we can confirm this, the better.

    Additionally, I have another question: If we were to only implement the following configuration:

    • X1Y11; X1Y10 => QUAD X1Y2 => external clock to CLK0
    • X0Y13; X0Y12 => QUAD X0Y3 => external clock to CLK0

    Would Vivado be able to synthesize and implement this setup successfully? Understanding this will help us determine if the implementation issue is due to using adjacent QUADs. According to Vivado’s wizard for managing GTHE4, it should be possible to supply the clock from one QUAD to the next, either from above or below, so I’d like to confirm if this holds true in practice.

    I appreciate your insights on these points.

  • Regarding this:

    • X1Y11; X1Y10 => QUAD X1Y2 => external clock to CLK0
    • X1Y5; X1Y4 => QUAD X1Y1 => clock from CLK0 of QUAD X1Y2
    • X0Y13; X0Y12 => QUAD X0Y3 => external clock to CLK0
    • X0Y9; X0Y8 => QUAD X0Y2 => clock from CLK0 of QUAD X0Y3

    You mean to create 4 util_adxcvr's each with two channels ?  All of these inside a single util_adxcvr won't be possible, except when instantiating for 16 channels.

    I think the reference clocks should be fine when comming from adjacent quad.

    Regards,

    Adrian

  • No, I mean to create a single util_adxcvr that can manage the configuration I described to you.

    What solution would you suggest?