Help with two ADRV9008-2W/PCBZ on ZCU102

Hi,

I know this topic pops up regularly on EngineerZone forums. I have read a lot of posts and tried to follow all advices. Still, I am stuck at the moment and hope I can get some help to move further.

What I am trying to do is to make a ZCU102 with 2 ADRV9008-2 eval boards working with an adapted HDL reference design and AD Linux with an adapted device tree. My experiments are based on the 2019-R2 release.

What I did so far:

- Modified the ADRV9008-2W/PCBZ hardware to feed dev_clk and dev_sysref signals from the AD9528 on the first card (on FMC HPC0) to the second card (on FMC HPC1) according to instructions from Design support forum. The signals are there but still I am not sure if the hardware setup is correct.

- Modified the HDL reference design to include a second ADRV9008-2W/PCBZ on HPC0, the double number of lanes and converters, and hopefully connected all lanes, clocks, handshake signals etc. accordingly. The first card at HPC0 is connected to SPI0, the second one at HPC1 to SPI1. The design looks reasonable and builds fine, but still there might be errors.

- Use zynqmp-zcu102-rev10-adrv9008-2-jesd204-fsm.dts and includes as a start for a modified device tree. I added  clk1_ad9528:ad9528-2 and trx1_adrv9009:adrv9009-phy-b at spi1 and followed the example in zynqmp-adrv9009-zu11eg-revb-adrv2crr-fmc-revb-jesd204-fsm.dts for the JESD204 setup.

The success is limited so far, see kernel log below. The second ad9528 is found, so spi2 seems to work. Apparently, there are problems with the JESD. The second adrv9009 is not found either.

My questions:

- Does any obvious configuration error spring to mind yet?

- What would be the next step to debug?

- What about MultichipSync initialization as required by UG1295? Do I have to do it explicitely (how?) or does the Linux kernel care for it?

Thank you in advance!

Best regards,

Erik

[ 1.530305] jesd204: created con: id=0, topo=0, link=0, /amba/spi@ff040000/ad9528-1@0 <-> /fpga-axi@0/axi-adxcvr-tx@84a80000
[ 1.540914] jesd204: created con: id=1, topo=0, link=2, /amba/spi@ff040000/ad9528-1@0 <-> /fpga-axi@0/axi-adxcvr-rx-os@84a50000
[ 1.552332] jesd204: created con: id=2, topo=0, link=2, /fpga-axi@0/axi-adxcvr-rx-os@84a50000 <-> /fpga-axi@0/axi-jesd204-rx-os@84ab0000
[ 1.564527] jesd204: created con: id=3, topo=0, link=0, /fpga-axi@0/axi-adxcvr-tx@84a80000 <-> /fpga-axi@0/axi-jesd204-tx@84a90000
[ 1.576206] jesd204: created con: id=4, topo=0, link=0, /fpga-axi@0/axi-jesd204-tx@84a90000 <-> /fpga-axi@0/axi-adrv9009-tx-hpc@84a04000
[ 1.588420] jesd204: created con: id=5, topo=0, link=2, /fpga-axi@0/axi-jesd204-rx-os@84ab0000 <-> /amba/spi@ff050000/adrv9009-phy-b@1
[ 1.600436] jesd204: created con: id=6, topo=0, link=0, /fpga-axi@0/axi-adrv9009-tx-hpc@84a04000 <-> /amba/spi@ff050000/adrv9009-phy-b@1
[ 1.612666] jesd204: created con: id=7, topo=0, link=2, /amba/spi@ff050000/adrv9009-phy-b@1 <-> /amba/spi@ff040000/adrv9009-phy@1
[ 1.624241] jesd204: created con: id=8, topo=0, link=0, /amba/spi@ff050000/adrv9009-phy-b@1 <-> /amba/spi@ff040000/adrv9009-phy@1
[ 1.635819] jesd204: /amba/spi@ff040000/adrv9009-phy@1: JESD204[0] transition uninitialized -> initialized
[ 1.645408] jesd204: /amba/spi@ff040000/adrv9009-phy@1: JESD204[2] transition uninitialized -> initialized
[ 1.655009] jesd204: found 8 devices and 1 topologies
[ 3.013579] adi-axi-clkgen 83c00000.axi-clkgen: failed to get s_axi_aclk
[ 3.019176] adi-axi-clkgen 83c20000.axi-clkgen: failed to get s_axi_aclk
[ 3.412148] axi_sysid 85000000.axi-sysid-0: [adrv9009] on [zcu102] git <0> dirty [2021-04-29 12:11:06] UTC
[ 3.753705] zynqmp_clk_divider_set_rate() set divider failed for spi1_ref_div1, ret = -13
[ 4.114962] ad9528 spi1.0: spi1.0 supply vcc not found, using dummy regulator
[ 4.122124] ad9528 spi1.0: Linked as a consumer to regulator.0
[ 4.152846] jesd204: /amba/spi@ff040000/ad9528-1@0,jesd204:0,parent=spi1.0: Using as SYSREF provider
[ 4.162306] adrv9009 spi1.1: adrv9009_probe : enter
[ 4.172677] ad9528 spi2.0: spi2.0 supply vcc not found, using dummy regulator
[ 4.179854] ad9528 spi2.0: Linked as a consumer to regulator.0
[ 4.211481] adrv9009 spi2.1: adrv9009_probe : enter
[ 4.216558] adrv9009: probe of spi2.1 failed with error -2
[ 5.055369] cf_axi_dds 84a04000.axi-adrv9009-tx-hpc: Analog Devices CF_AXI_DDS_DDS MASTER (9.01.b) at 0x84A04000 mapped to 0x(____ptrval____), probed DDS AD9371
[ 5.072127] axi_adxcvr 84a50000.axi-adxcvr-rx-os: AXI-ADXCVR-RX (17.01.a) using CPLL on GTH4 at 0x84A50000. Number of lanes: 4.
[ 5.084505] axi_adxcvr 84a80000.axi-adxcvr-tx: AXI-ADXCVR-TX (17.01.a) using QPLL on GTH4 at 0x84A80000. Number of lanes: 8.
[ 19.532190] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] invalid link state: initialized, exp: idle, nxt: idle
[ 19.532201] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: Rolling back from 'idle', got error -22
[ 19.532206] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: FSM completed with error -22
[ 19.532230] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] invalid link state: initialized, exp: idle, nxt: idle
[ 19.532236] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: Rolling back from 'idle', got error -22
[ 19.532241] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: FSM completed with error -22

Parents
  • 0
    •  Analog Employees 
    on Apr 30, 2021 11:32 AM

    Hi Erik,

    If you implement the jesd204-fsm, the driver and jesd204 subsystem will take are of multichip sync.

    You should first review this: https://wiki.analog.com/resources/tools-software/linux-drivers/jesd204/jesd204-fsm-framework

    BTW - there is a similar effort discussed here:

    https://ez.analog.com/linux-software-drivers/f/q-a/543712/how-to-configure-device-tree-to-use-more-than-one-adrv9009

    Maybe you get some pointers from it.

    Without your devicetrees it's hard to tell what's going on.

    -Michael

  • Hi Michael,

    thanks for the answer. I have already read the above links and tried to heed all advice as good as I could.

    My device tree is below, i.e. only the differences to the 2019-R2 distribution. The ### comments indicate that the following blocks are copied from the spi0 branches. I thought, in this short form it is more readable and hope I do not have forgotten something important. The gpio numbers reflect my HDL design. I have removed the adrv9009_gpio_?? lines since I could not find any usage of them within Linux driver. I will need some GPIOs for other purposes.

    As a next step I will check the ref_clk and sysref signals on the ADRC9008.2 boards using an oscilloscope, just in case.

    Best regards,

    Erik

    #include "zynqmp-zcu102-rev10-adrv9008-2-jesd204-fsm.dts"
    
    &spi1 {
        status = "okay";
    
        clk1_ad9528: ad9528-2@0 {
            ### ...
            ### copy of clk0_ad9528, changing ad9528-1 to ad9528-2
            ### only output channels 12 and 13
            ### ...
    	};
        trx1_adrv9009: adrv9009-phy-b@1 {
    		### ...
            ### copy of trx0_adrv9009	
            ### ...
        }
    }
    
    &trx0_adrv9009 {
        clock-output-names = "rx0_sampl_clk", "rx0_os_sampl_clk", "tx0_sampl_clk";
    
        jesd204-inputs =
            <&trx1_adrv9009 0 FRAMER_LINK_ORX>,
            <&trx1_adrv9009 0 DEFRAMER_LINK_TX>;
    };
    
    &trx1_adrv9009 {
        compatible = "adrv9008-2";
    
        clock-names = "jesd_tx_clk", "jesd_rx_os_clk",
            "dev_clk", "fmc_clk", "sysref_dev_clk", "sysref_fmc_clk";
    
        clocks = <&axi_adrv9009_tx_jesd>, <&axi_adrv9009_rx_os_jesd>,
            <&clk0_ad9528 8>, <&clk0_ad9528 5>, <&clk1_ad9528 12>, 
            <&clk1_ad9528 3>;
        clock-names = "jesd_tx_clk", "jesd_rx_os_clk",
            "dev_clk_1", "fmc_clk_1", "dev_sysref", "fmc_sysrev";
        clock-output-names = "rx1_sampl_clk", "rx1_os_sampl_clk", "tx1_sampl_clk";
    
        reset-gpios = <&gpio 121 0>;
        test-gpios = <&gpio 122 0>;
        sysref-req-gpios = <&gpio 127 0>;
        tx2-enable-gpios = <&gpio 125 0>;
        tx1-enable-gpios = <&gpio 126 0>;
    
        jesd204-device;
        jesd204-inputs =
            <&axi_adrv9009_rx_os_jesd 0 FRAMER_LINK_ORX>,
            <&axi_adrv9009_core_tx 0 DEFRAMER_LINK_TX>;
        /delete-property/ interrupts;
    };
    
    &clk0_ad9528 {
        ad9528_0_c5: channel@5 {
            reg = <5>;
            adi,extended-name = "FMC_CLK_1";
            adi,driver-mode = <DRIVER_MODE_LVDS>;
            adi,divider-phase = <0>;
            adi,channel-divider = <5>;
            adi,signal-source = <SOURCE_VCO>;
        };
    
        ad9528_0_c8: channel@8 {
            reg = <8>;
            adi,extended-name = "DEV_CLK_1";
            adi,driver-mode = <DRIVER_MODE_LVDS>;
            adi,divider-phase = <0>;
            adi,channel-divider = <5>;
            adi,signal-source = <SOURCE_VCO>;
        };
    
        ad9528_0_c7: channel@7 {
            reg = <7>;
            adi,extended-name = "DEV_SYSREF_1";
            adi,driver-mode = <DRIVER_MODE_LVDS>;
            adi,divider-phase = <0>;
            adi,channel-divider = <5>;
            adi,signal-source = <SOURCE_SYSREF_VCO>;
        };
    };
    
    &clk1_ad9528 {
        adi,sysref-src = <SYSREF_SRC_EXTERNAL>;
        adi,sysref-pattern-mode = <SYSREF_PATTERN_NSHOT>;
        /delete-property/ adi,sysref-request-enable;
        adi,sysref-nshot-mode = <SYSREF_NSHOT_8_PULSES>;
    };
    

  • Still some issues.

    If I set adi,sysref-pattern-mode  to SYSREF_PATTERN_NSHOT, as it is the default for ADRV9009, I get : adrv9009 spi2.1: Link2 TAL_FRAMER_A framerStatus 0x6F

    and jesd_status looks like below.

    If I use SYSREF_PATTERN_CONTINUOUS, the error is gone and the latency values of the rx-os link are better, but the tx link is still deasserted. In case the SYNC has something to do with the sync lines of the FMC: I have ANDed both tx_sync anc connected the output to axi_adrv9009_tx_jesd/sync of the PL. Hope this is correct.

    Any ideas?

    Thank you, and best regards,

    Erik

  • +1
    •  Analog Employees 
    on May 4, 2021 4:16 PM in reply to EHeinz689

    Hi Erik,

    First of all SYNC de-asserted is good. Your links are in DATA - that's where you want to be.

    If SYNC is asserted you're typically in CGS, that's not so good.

    The framer errors are explained here: 

    https://github.com/analogdevicesinc/linux/blob/master/drivers/iio/adc/talise/talise_jesd204.h#L255

    You're getting a "SYSREF phase error -  a new SYSREF had different timing than the first that set the LMFC timing."

    Most likely on the part that is connected via the external SYSREF input. You should probe the SYSREFs on both cards.

    Then make sure that they assert the same time - I guess the ones on the slave part have some delays. Use output delays to compensate for that. Make sure dev_clk is also compensated. Both parts on both cards need to be in sync both in terms of dev_clk and SYSREF.

    -Michael

  • It took me some time to measure the signals correctly and to understand the AD9528 details. I am now able to compensate the delay and to align the SYSREFs.

    Whatever I do, however, the  adrv9009 spi2.1: Link2 TAL_FRAMER_A framerStatus 0x6F message is there. It only goes away if I switch to continuous SYSREF. Anyway, the ADRVs both seem to work and there are no other errors. May be, I could ignore the 0x6F?

    I found one more issue when I tried to change the ADRV9008-2 profiles to 491.520 Msps. Changing the profiles at runtime using /sys/bus/iio/devices/iio:device2/profile_config changes only the profile of the first ADRV. The messages are:

    [ 104.452939] adrv9009 spi2.1: Link2 TAL_FRAMER_A framerStatus 0x6F
    [ 104.452985] adrv9009 spi2.1: Link0 TAL_DEFRAMER_A deframerStatus 0x12
    [ 104.453031] adrv9009 spi1.1: Link0 TAL_DEFRAMER_A deframerStatus 0x12
    [ 104.453112] adrv9009 spi1.1: Link2 TAL_FRAMER_A framerStatus 0x2F

    Configuring the 491.520 Msps profile at boot via the device tree works, though. Again there are some messages regarding the framesStatus:

    [ 42.118306] adrv9009 spi2.1: Link2 TAL_FRAMER_A framerStatus 0x2F
    [ 42.118491] adrv9009 spi1.1: Link2 TAL_FRAMER_A framerStatus 0x2F

    but both ADRV work correctly at 491.52 MHz sampling rate.

    So far, so good.

    Best regards,

    Erik

  • +1
    •  Analog Employees 
    on May 6, 2021 10:44 AM in reply to EHeinz689
    Whatever I do, however, the  adrv9009 spi2.1: Link2 TAL_FRAMER_A framerStatus 0x6F message is there. It only goes away if I switch to continuous SYSREF. Anyway, the ADRVs both seem to work and there are no other errors. May be, I could ignore the 0x6F?

    You may need to see if there is a glitch, or some floating SYSREF when you use NSHOT SYSREF.

    Try to probe the SYSREF outputs on both clock chips and see if you can find irregularities.

    I found one more issue when I tried to change the ADRV9008-2 profiles to 491.520 Msps. Changing the profiles at runtime using /sys/bus/iio/devices/iio:device2/profile_config changes only the profile of the first ADRV.

    You need to write the same profile to each device. And you need to make sure that you write the profile to the TOP device last. This way all devices have the same config when the link restart is triggered at the TOP device.

    -Michael

Reply
  • +1
    •  Analog Employees 
    on May 6, 2021 10:44 AM in reply to EHeinz689
    Whatever I do, however, the  adrv9009 spi2.1: Link2 TAL_FRAMER_A framerStatus 0x6F message is there. It only goes away if I switch to continuous SYSREF. Anyway, the ADRVs both seem to work and there are no other errors. May be, I could ignore the 0x6F?

    You may need to see if there is a glitch, or some floating SYSREF when you use NSHOT SYSREF.

    Try to probe the SYSREF outputs on both clock chips and see if you can find irregularities.

    I found one more issue when I tried to change the ADRV9008-2 profiles to 491.520 Msps. Changing the profiles at runtime using /sys/bus/iio/devices/iio:device2/profile_config changes only the profile of the first ADRV.

    You need to write the same profile to each device. And you need to make sure that you write the profile to the TOP device last. This way all devices have the same config when the link restart is triggered at the TOP device.

    -Michael

Children