Help with two ADRV9008-2W/PCBZ on ZCU102

Hi,

I know this topic pops up regularly on EngineerZone forums. I have read a lot of posts and tried to follow all advices. Still, I am stuck at the moment and hope I can get some help to move further.

What I am trying to do is to make a ZCU102 with 2 ADRV9008-2 eval boards working with an adapted HDL reference design and AD Linux with an adapted device tree. My experiments are based on the 2019-R2 release.

What I did so far:

- Modified the ADRV9008-2W/PCBZ hardware to feed dev_clk and dev_sysref signals from the AD9528 on the first card (on FMC HPC0) to the second card (on FMC HPC1) according to instructions from Design support forum. The signals are there but still I am not sure if the hardware setup is correct.

- Modified the HDL reference design to include a second ADRV9008-2W/PCBZ on HPC0, the double number of lanes and converters, and hopefully connected all lanes, clocks, handshake signals etc. accordingly. The first card at HPC0 is connected to SPI0, the second one at HPC1 to SPI1. The design looks reasonable and builds fine, but still there might be errors.

- Use zynqmp-zcu102-rev10-adrv9008-2-jesd204-fsm.dts and includes as a start for a modified device tree. I added  clk1_ad9528:ad9528-2 and trx1_adrv9009:adrv9009-phy-b at spi1 and followed the example in zynqmp-adrv9009-zu11eg-revb-adrv2crr-fmc-revb-jesd204-fsm.dts for the JESD204 setup.

The success is limited so far, see kernel log below. The second ad9528 is found, so spi2 seems to work. Apparently, there are problems with the JESD. The second adrv9009 is not found either.

My questions:

- Does any obvious configuration error spring to mind yet?

- What would be the next step to debug?

- What about MultichipSync initialization as required by UG1295? Do I have to do it explicitely (how?) or does the Linux kernel care for it?

Thank you in advance!

Best regards,

Erik

[ 1.530305] jesd204: created con: id=0, topo=0, link=0, /amba/spi@ff040000/ad9528-1@0 <-> /fpga-axi@0/axi-adxcvr-tx@84a80000
[ 1.540914] jesd204: created con: id=1, topo=0, link=2, /amba/spi@ff040000/ad9528-1@0 <-> /fpga-axi@0/axi-adxcvr-rx-os@84a50000
[ 1.552332] jesd204: created con: id=2, topo=0, link=2, /fpga-axi@0/axi-adxcvr-rx-os@84a50000 <-> /fpga-axi@0/axi-jesd204-rx-os@84ab0000
[ 1.564527] jesd204: created con: id=3, topo=0, link=0, /fpga-axi@0/axi-adxcvr-tx@84a80000 <-> /fpga-axi@0/axi-jesd204-tx@84a90000
[ 1.576206] jesd204: created con: id=4, topo=0, link=0, /fpga-axi@0/axi-jesd204-tx@84a90000 <-> /fpga-axi@0/axi-adrv9009-tx-hpc@84a04000
[ 1.588420] jesd204: created con: id=5, topo=0, link=2, /fpga-axi@0/axi-jesd204-rx-os@84ab0000 <-> /amba/spi@ff050000/adrv9009-phy-b@1
[ 1.600436] jesd204: created con: id=6, topo=0, link=0, /fpga-axi@0/axi-adrv9009-tx-hpc@84a04000 <-> /amba/spi@ff050000/adrv9009-phy-b@1
[ 1.612666] jesd204: created con: id=7, topo=0, link=2, /amba/spi@ff050000/adrv9009-phy-b@1 <-> /amba/spi@ff040000/adrv9009-phy@1
[ 1.624241] jesd204: created con: id=8, topo=0, link=0, /amba/spi@ff050000/adrv9009-phy-b@1 <-> /amba/spi@ff040000/adrv9009-phy@1
[ 1.635819] jesd204: /amba/spi@ff040000/adrv9009-phy@1: JESD204[0] transition uninitialized -> initialized
[ 1.645408] jesd204: /amba/spi@ff040000/adrv9009-phy@1: JESD204[2] transition uninitialized -> initialized
[ 1.655009] jesd204: found 8 devices and 1 topologies
[ 3.013579] adi-axi-clkgen 83c00000.axi-clkgen: failed to get s_axi_aclk
[ 3.019176] adi-axi-clkgen 83c20000.axi-clkgen: failed to get s_axi_aclk
[ 3.412148] axi_sysid 85000000.axi-sysid-0: [adrv9009] on [zcu102] git <0> dirty [2021-04-29 12:11:06] UTC
[ 3.753705] zynqmp_clk_divider_set_rate() set divider failed for spi1_ref_div1, ret = -13
[ 4.114962] ad9528 spi1.0: spi1.0 supply vcc not found, using dummy regulator
[ 4.122124] ad9528 spi1.0: Linked as a consumer to regulator.0
[ 4.152846] jesd204: /amba/spi@ff040000/ad9528-1@0,jesd204:0,parent=spi1.0: Using as SYSREF provider
[ 4.162306] adrv9009 spi1.1: adrv9009_probe : enter
[ 4.172677] ad9528 spi2.0: spi2.0 supply vcc not found, using dummy regulator
[ 4.179854] ad9528 spi2.0: Linked as a consumer to regulator.0
[ 4.211481] adrv9009 spi2.1: adrv9009_probe : enter
[ 4.216558] adrv9009: probe of spi2.1 failed with error -2
[ 5.055369] cf_axi_dds 84a04000.axi-adrv9009-tx-hpc: Analog Devices CF_AXI_DDS_DDS MASTER (9.01.b) at 0x84A04000 mapped to 0x(____ptrval____), probed DDS AD9371
[ 5.072127] axi_adxcvr 84a50000.axi-adxcvr-rx-os: AXI-ADXCVR-RX (17.01.a) using CPLL on GTH4 at 0x84A50000. Number of lanes: 4.
[ 5.084505] axi_adxcvr 84a80000.axi-adxcvr-tx: AXI-ADXCVR-TX (17.01.a) using QPLL on GTH4 at 0x84A80000. Number of lanes: 8.
[ 19.532190] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] invalid link state: initialized, exp: idle, nxt: idle
[ 19.532201] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: Rolling back from 'idle', got error -22
[ 19.532206] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: FSM completed with error -22
[ 19.532230] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] invalid link state: initialized, exp: idle, nxt: idle
[ 19.532236] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: Rolling back from 'idle', got error -22
[ 19.532241] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: FSM completed with error -22

Parents
  • 0
    •  Analog Employees 
    on Apr 30, 2021 11:32 AM

    Hi Erik,

    If you implement the jesd204-fsm, the driver and jesd204 subsystem will take are of multichip sync.

    You should first review this: https://wiki.analog.com/resources/tools-software/linux-drivers/jesd204/jesd204-fsm-framework

    BTW - there is a similar effort discussed here:

    https://ez.analog.com/linux-software-drivers/f/q-a/543712/how-to-configure-device-tree-to-use-more-than-one-adrv9009

    Maybe you get some pointers from it.

    Without your devicetrees it's hard to tell what's going on.

    -Michael

  • Hi Michael,

    thanks for the answer. I have already read the above links and tried to heed all advice as good as I could.

    My device tree is below, i.e. only the differences to the 2019-R2 distribution. The ### comments indicate that the following blocks are copied from the spi0 branches. I thought, in this short form it is more readable and hope I do not have forgotten something important. The gpio numbers reflect my HDL design. I have removed the adrv9009_gpio_?? lines since I could not find any usage of them within Linux driver. I will need some GPIOs for other purposes.

    As a next step I will check the ref_clk and sysref signals on the ADRC9008.2 boards using an oscilloscope, just in case.

    Best regards,

    Erik

    #include "zynqmp-zcu102-rev10-adrv9008-2-jesd204-fsm.dts"
    
    &spi1 {
        status = "okay";
    
        clk1_ad9528: ad9528-2@0 {
            ### ...
            ### copy of clk0_ad9528, changing ad9528-1 to ad9528-2
            ### only output channels 12 and 13
            ### ...
    	};
        trx1_adrv9009: adrv9009-phy-b@1 {
    		### ...
            ### copy of trx0_adrv9009	
            ### ...
        }
    }
    
    &trx0_adrv9009 {
        clock-output-names = "rx0_sampl_clk", "rx0_os_sampl_clk", "tx0_sampl_clk";
    
        jesd204-inputs =
            <&trx1_adrv9009 0 FRAMER_LINK_ORX>,
            <&trx1_adrv9009 0 DEFRAMER_LINK_TX>;
    };
    
    &trx1_adrv9009 {
        compatible = "adrv9008-2";
    
        clock-names = "jesd_tx_clk", "jesd_rx_os_clk",
            "dev_clk", "fmc_clk", "sysref_dev_clk", "sysref_fmc_clk";
    
        clocks = <&axi_adrv9009_tx_jesd>, <&axi_adrv9009_rx_os_jesd>,
            <&clk0_ad9528 8>, <&clk0_ad9528 5>, <&clk1_ad9528 12>, 
            <&clk1_ad9528 3>;
        clock-names = "jesd_tx_clk", "jesd_rx_os_clk",
            "dev_clk_1", "fmc_clk_1", "dev_sysref", "fmc_sysrev";
        clock-output-names = "rx1_sampl_clk", "rx1_os_sampl_clk", "tx1_sampl_clk";
    
        reset-gpios = <&gpio 121 0>;
        test-gpios = <&gpio 122 0>;
        sysref-req-gpios = <&gpio 127 0>;
        tx2-enable-gpios = <&gpio 125 0>;
        tx1-enable-gpios = <&gpio 126 0>;
    
        jesd204-device;
        jesd204-inputs =
            <&axi_adrv9009_rx_os_jesd 0 FRAMER_LINK_ORX>,
            <&axi_adrv9009_core_tx 0 DEFRAMER_LINK_TX>;
        /delete-property/ interrupts;
    };
    
    &clk0_ad9528 {
        ad9528_0_c5: channel@5 {
            reg = <5>;
            adi,extended-name = "FMC_CLK_1";
            adi,driver-mode = <DRIVER_MODE_LVDS>;
            adi,divider-phase = <0>;
            adi,channel-divider = <5>;
            adi,signal-source = <SOURCE_VCO>;
        };
    
        ad9528_0_c8: channel@8 {
            reg = <8>;
            adi,extended-name = "DEV_CLK_1";
            adi,driver-mode = <DRIVER_MODE_LVDS>;
            adi,divider-phase = <0>;
            adi,channel-divider = <5>;
            adi,signal-source = <SOURCE_VCO>;
        };
    
        ad9528_0_c7: channel@7 {
            reg = <7>;
            adi,extended-name = "DEV_SYSREF_1";
            adi,driver-mode = <DRIVER_MODE_LVDS>;
            adi,divider-phase = <0>;
            adi,channel-divider = <5>;
            adi,signal-source = <SOURCE_SYSREF_VCO>;
        };
    };
    
    &clk1_ad9528 {
        adi,sysref-src = <SYSREF_SRC_EXTERNAL>;
        adi,sysref-pattern-mode = <SYSREF_PATTERN_NSHOT>;
        /delete-property/ adi,sysref-request-enable;
        adi,sysref-nshot-mode = <SYSREF_NSHOT_8_PULSES>;
    };
    

  • 0
    •  Analog Employees 
    on Apr 30, 2021 3:33 PM in reply to EHeinz689

    I've never tested the external SYSREF input feature of the AD9528. You might need some extra 

    register settings to make it work.

    Please consult the datasheet, you can use iio_reg or the OSC debug tab to play with these registers.

    -Michael

  • I came up with this sysref-loop-through scheme since the ADRV9008-2W/PCBZ supports external input to SYSREF_CLK_IN of the AD9528 (by adding/moving some components), not to SYSREFIN of the ADRV9008-2 directly. The latter would not be so simple: it would need to attach wires to the PCB (nasty and might be unreliable), would need an additional signal for the FMC_SYSREF etc. etc. Moreover, this solution has been repeatedly suggested in the ADRV9009 Design Forum. So I guess it is supposed to work somehow.

    My plan for a next step is to monitor the sysref signals from the PL side using Xilinx ILA. The resolution then is limited to the ref_clk, though. Do you think this is sufficient to judge the sysref? Measuring physically is tricky, as I told.

    Since the problem seems to boil down to a sysref issue, I mark the thread as "answered". I will come back when I have a solution or additional Linux-related questions.

    Thanks a lot again.

    Best regards,

    Erik

  • Hi Michael,

    I am trying to understand AD9528 and SYSREF scheme to find out why external SYSREF source does not work for me. I am not there yet, but have a related question.

    I am puzzled about SYSREF frequency of the internal SYSREF. The Linux device tree for ADRV9009 has set adi,sysref-k-div = <512>; for ad9528-1. This should set registers 0x401/0x400 of the AD9528 to 0x200 and result in a sysref frequency of 120 kHz, shouldn't it?

    Actually, when booting Linux 2019_R2 with the default device tree, registers 0x401/0x400 of the AD9528 have a value of 8 and the SYSREF frequency is 7.68 MHz.

    Why is this and is it intended?

    Thank you,

    Erik

  • 0
    •  Analog Employees 
    on May 4, 2021 10:04 AM in reply to EHeinz689

    When using the AD9528 with the jesd204-fsm - the k divider value from the devicetree is ignored.

    The AD9528 driver get's the link parameters of all links on the topology then either computes the LMFC or LMEC (for jesd204c) later it computes the GCD of all LMFCs and then programs the k divider accordingly (suitable for all links)

    https://github.com/analogdevicesinc/linux/blob/master/drivers/iio/frequency/ad9528.c#L1257

    https://github.com/analogdevicesinc/linux/blob/master/drivers/iio/frequency/ad9528.c#L1329

    Hope this explains it.

    -Michael

  • Thanks. I found the problem with external SYSREF. To enable SYSREF_IN input, register 0x0404 must be set to 1 (in case of a differential signal). Since this has to be done during initialization, I made a little patch:

    *** drivers/iio/frequency/ad9528.c.ORIG	2021-04-23 12:56:11.000000000 +0200
    --- drivers/iio/frequency/ad9528.c	2021-05-04 12:56:53.684915076 +0200
    ***************
    *** 70,75 ****
    --- 70,76 ----
      
      #define AD9528_SYSREF_K_DIVIDER			AD9528_2B(0x400)
      #define AD9528_SYSREF_CTRL			AD9528_2B(0x402)
    + #define AD9528_SYSREF_IN_CTRL			AD9528_1B(0x404)
      
      #define AD9528_PD_EN				AD9528_1B(0x500)
      #define AD9528_CHANNEL_PD_EN			AD9528_2B(0x501)
    ***************
    *** 1164,1169 ****
    --- 1165,1176 ----
      	if (ret < 0)
      		return ret;
      
    + 	if (pdata->sysref_src == SYSREF_SRC_EXTERNAL) {
    + 		ret = ad9528_write(indio_dev, AD9528_SYSREF_IN_CTRL, 1);
    + 		if (ret < 0)
    + 			return ret;
    + 	}
    + 
      	ret = ad9528_write(indio_dev, AD9528_PD_EN, AD9528_PD_BIAS |
      			   AD_IF(pll1_bypass_en, AD9528_PD_PLL1) |
      			   AD_IF(pll2_bypass_en, AD9528_PD_PLL2));
      			   

    This should be made a bit more general, but it seems to work. Error messages are gone now (see below), and oscilloscope seems to work at first glance.

    How can I check if JESD204 setup is correct?

    Best regards,

    Erik

    [    1.530545] jesd204: created con: id=0, topo=0, link=0, /amba/spi@ff040000/ad9528-1@0 <-> /fpga-axi@0/axi-adxcvr-tx@84a80000
    [    1.541156] jesd204: created con: id=1, topo=0, link=2, /amba/spi@ff040000/ad9528-1@0 <-> /fpga-axi@0/axi-adxcvr-rx-os@84a50000
    [    1.552573] jesd204: created con: id=2, topo=0, link=2, /fpga-axi@0/axi-adxcvr-rx-os@84a50000 <-> /fpga-axi@0/axi-jesd204-rx-os@84ab0000
    [    1.564769] jesd204: created con: id=3, topo=0, link=0, /fpga-axi@0/axi-adxcvr-tx@84a80000 <-> /fpga-axi@0/axi-jesd204-tx@84a90000
    [    1.576447] jesd204: created con: id=4, topo=0, link=0, /fpga-axi@0/axi-jesd204-tx@84a90000 <-> /fpga-axi@0/axi-adrv9009-tx-hpc@84a04000
    [    1.588662] jesd204: created con: id=5, topo=0, link=2, /fpga-axi@0/axi-jesd204-rx-os@84ab0000 <-> /amba/spi@ff050000/adrv9009-phy-b@1
    [    1.600677] jesd204: created con: id=6, topo=0, link=0, /fpga-axi@0/axi-adrv9009-tx-hpc@84a04000 <-> /amba/spi@ff050000/adrv9009-phy-b@1
    [    1.612907] jesd204: created con: id=7, topo=0, link=2, /amba/spi@ff050000/adrv9009-phy-b@1 <-> /amba/spi@ff040000/adrv9009-phy@1
    [    1.624483] jesd204: created con: id=8, topo=0, link=0, /amba/spi@ff050000/adrv9009-phy-b@1 <-> /amba/spi@ff040000/adrv9009-phy@1
    [    1.636060] jesd204: /amba/spi@ff040000/adrv9009-phy@1: JESD204[0] transition uninitialized -> initialized
    [    1.645650] jesd204: /amba/spi@ff040000/adrv9009-phy@1: JESD204[2] transition uninitialized -> initialized
    [    1.655251] jesd204: found 8 devices and 1 topologies
    [    3.400378] axi_sysid 85000000.axi-sysid-0: [adrv9009] on [zcu102] git <0> dirty [2021-04-29 12:11:06] UTC
    [    3.741272] zynqmp_clk_divider_set_rate() set divider failed for spi1_ref_div1, ret = -13
    [    5.002298] ad9528 spi1.0: spi1.0 supply vcc not found, using dummy regulator
    [    5.009473] ad9528 spi1.0: Linked as a consumer to regulator.0
    [    5.037279] jesd204: /amba/spi@ff040000/ad9528-1@0,jesd204:0,parent=spi1.0: Using as SYSREF provider
    [    5.046738] adrv9009 spi1.1: adrv9009_probe : enter
    [    5.057222] ad9528 spi2.0: spi2.0 supply vcc not found, using dummy regulator
    [    5.064382] ad9528 spi2.0: Linked as a consumer to regulator.0
    [    5.092685] adrv9009 spi2.1: adrv9009_probe : enter
    [    5.935780] cf_axi_dds 84a04000.axi-adrv9009-tx-hpc: Analog Devices CF_AXI_DDS_DDS MASTER (9.01.b) at 0x84A04000 mapped to 0x(____ptrval____), probed DDS AD9371
    [    5.952133] axi_adxcvr 84a50000.axi-adxcvr-rx-os: AXI-ADXCVR-RX (17.01.a) using CPLL on GTH4 at 0x84A50000. Number of lanes: 4.
    [    5.964395] axi_adxcvr 84a80000.axi-adxcvr-tx: AXI-ADXCVR-TX (17.01.a) using QPLL on GTH4 at 0x84A80000. Number of lanes: 8.
    [    5.977756] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition initialized -> probed
    [    5.988899] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition initialized -> probed
    [    6.000034] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition probed -> idle
    [    6.010553] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition probed -> idle
    [    6.021070] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition idle -> device_init
    [    6.032020] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition idle -> device_init
    [    6.042973] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition device_init -> link_init
    [    6.054356] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition device_init -> link_init
    [    6.065747] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_init -> link_supported
    [    6.077390] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_init -> link_supported
    [    6.093823] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_supported -> link_pre_setup
    [    6.105909] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_supported -> link_pre_setup
    [    6.117992] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_pre_setup -> clk_sync_stage1
    [    6.130157] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_pre_setup -> clk_sync_stage1
    [    6.142323] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage1 -> clk_sync_stage2
    [    6.154576] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage1 -> clk_sync_stage2
    [    6.166829] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage2 -> clk_sync_stage3
    [    6.179084] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage2 -> clk_sync_stage3
    [    6.191352] jesd204: /fpga-axi@0/axi-jesd204-rx-os@84ab0000,jesd204:5,parent=84ab0000.axi-jesd204-rx-os: Possible instantiation for multiple chips; HDL lanes 4, Link[2] lanes 2
    [    6.207089] jesd204: /fpga-axi@0/axi-jesd204-tx@84a90000,jesd204:4,parent=84a90000.axi-jesd204-tx: Possible instantiation for multiple chips; HDL lanes 8, Link[0] lanes 4
    [    6.223412] adrv9009 spi2.1: ADIHAL_resetHw
    [    6.543506] adrv9009 spi1.1: ADIHAL_resetHw
    [    6.863431] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage3 -> link_setup
    [    6.875263] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage3 -> link_setup
    [    6.887550] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_setup -> opt_setup_stage1
    [    6.899458] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_setup -> opt_setup_stage1
    [   11.569182] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage1 -> opt_setup_stage2
    [   11.581615] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage1 -> opt_setup_stage2
    [   11.594274] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage2 -> opt_setup_stage3
    [   11.606703] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage2 -> opt_setup_stage3
    [   11.619458] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage3 -> opt_setup_stage4
    [   11.631885] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage3 -> opt_setup_stage4
    [   16.146477] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage4 -> opt_setup_stage5
    [   16.158906] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage4 -> opt_setup_stage5
    [   16.172746] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage5 -> clocks_enable
    [   16.184914] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage5 -> clocks_enable
    [   16.197420] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clocks_enable -> link_enable
    [   16.209150] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clocks_enable -> link_enable
    [   16.221298] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_enable -> link_running
    [   16.232945] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_enable -> link_running
    [   16.345997] adrv9009 spi2.1: adrv9009_info: adrv9008-2 Rev 192, Firmware 6.0.2 API version: 3.6.0.5 successfully initialized via jesd204-fsm
    [   16.460951] adrv9009 spi1.1: adrv9009_info: adrv9008-2 Rev 192, Firmware 6.0.2 API version: 3.6.0.5 successfully initialized via jesd204-fsm
    [   16.473556] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_running -> opt_post_running_stage
    [   16.486156] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_running -> opt_post_running_stage
    [   23.940985] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_post_running_stage -> opt_post_running_stage
    [   23.940995] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_post_running_stage -> opt_post_running_stage
    [   23.941008] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_post_running_stage -> link_running
    [   23.941014] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_post_running_stage -> link_running
    [   23.941030] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_running -> link_enable
    [   23.941035] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_running -> link_enable
    [   23.941054] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_enable -> clocks_enable
    [   23.941059] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_enable -> clocks_enable
    [   23.941070] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clocks_enable -> opt_setup_stage5
    [   23.941075] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clocks_enable -> opt_setup_stage5
    [   23.941085] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage5 -> opt_setup_stage4
    [   23.941091] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage5 -> opt_setup_stage4
    [   23.941101] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage4 -> opt_setup_stage3
    [   23.941107] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage4 -> opt_setup_stage3
    [   23.941117] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage3 -> opt_setup_stage2
    [   23.941122] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage3 -> opt_setup_stage2
    [   23.941133] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage2 -> opt_setup_stage1
    [   23.941138] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage2 -> opt_setup_stage1
    [   23.941152] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage1 -> link_setup
    [   23.941158] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage1 -> link_setup
    [   23.941168] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_setup -> clk_sync_stage3
    [   23.941173] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_setup -> clk_sync_stage3
    [   23.941182] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage3 -> clk_sync_stage2
    [   23.941188] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage3 -> clk_sync_stage2
    [   23.941198] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage2 -> clk_sync_stage1
    [   23.941203] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage2 -> clk_sync_stage1
    [   23.941213] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage1 -> link_pre_setup
    [   23.941219] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage1 -> link_pre_setup
    [   23.941230] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_pre_setup -> link_supported
    [   23.941235] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_pre_setup -> link_supported
    [   23.941246] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_supported -> link_init
    [   23.941251] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_supported -> link_init
    [   23.941266] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_init -> device_init
    [   23.941271] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_init -> device_init
    [   23.941281] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition device_init -> idle
    [   23.941286] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition device_init -> idle
    [   23.941305] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition idle -> idle
    [   23.941311] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition idle -> idle
    [   23.941321] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition idle -> device_init
    [   23.941327] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition idle -> device_init
    [   23.941338] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition device_init -> link_init
    [   23.941343] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition device_init -> link_init
    [   23.941358] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_init -> link_supported
    [   23.941364] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_init -> link_supported
    [   24.175872] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_supported -> link_pre_setup
    [   24.175881] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_supported -> link_pre_setup
    [   24.175895] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_pre_setup -> clk_sync_stage1
    [   24.175900] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_pre_setup -> clk_sync_stage1
    [   24.175910] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage1 -> clk_sync_stage2
    [   24.175916] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage1 -> clk_sync_stage2
    [   24.175926] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage2 -> clk_sync_stage3
    [   24.175931] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage2 -> clk_sync_stage3
    [   24.175941] jesd204: /fpga-axi@0/axi-jesd204-rx-os@84ab0000,jesd204:5,parent=84ab0000.axi-jesd204-rx-os: Possible instantiation for multiple chips; HDL lanes 4, Link[2] lanes 2
    [   24.175959] jesd204: /fpga-axi@0/axi-jesd204-tx@84a90000,jesd204:4,parent=84a90000.axi-jesd204-tx: Possible instantiation for multiple chips; HDL lanes 8, Link[0] lanes 4
    [   24.176984] adrv9009 spi2.1: ADIHAL_resetHw
    [   25.770155] adrv9009 spi1.1: ADIHAL_resetHw
    [   26.108220] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage3 -> link_setup
    [   26.108229] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage3 -> link_setup
    [   26.108503] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_setup -> opt_setup_stage1
    [   26.108509] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_setup -> opt_setup_stage1
    [   36.366302] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage1 -> opt_setup_stage2
    [   36.366311] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage1 -> opt_setup_stage2
    [   36.366659] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage2 -> opt_setup_stage3
    [   36.366666] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage2 -> opt_setup_stage3
    [   36.367029] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage3 -> opt_setup_stage4
    [   36.367035] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage3 -> opt_setup_stage4
    [   40.784946] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage4 -> opt_setup_stage5
    [   40.784955] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage4 -> opt_setup_stage5
    [   40.786285] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage5 -> clocks_enable
    [   40.786294] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage5 -> clocks_enable
    [   40.786592] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clocks_enable -> link_enable
    [   40.786599] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clocks_enable -> link_enable
    [   40.787007] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_enable -> link_running
    [   40.787013] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_enable -> link_running
    [   40.891734] adrv9009 spi2.1: adrv9009_info: adrv9008-2 Rev 192, Firmware 6.0.2 API version: 3.6.0.5 successfully initialized via jesd204-fsm
    [   40.994251] adrv9009 spi1.1: adrv9009_info: adrv9008-2 Rev 192, Firmware 6.0.2 API version: 3.6.0.5 successfully initialized via jesd204-fsm
    [   40.994269] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_running -> opt_post_running_stage
    [   40.994276] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_running -> opt_post_running_stage
    

Reply
  • Thanks. I found the problem with external SYSREF. To enable SYSREF_IN input, register 0x0404 must be set to 1 (in case of a differential signal). Since this has to be done during initialization, I made a little patch:

    *** drivers/iio/frequency/ad9528.c.ORIG	2021-04-23 12:56:11.000000000 +0200
    --- drivers/iio/frequency/ad9528.c	2021-05-04 12:56:53.684915076 +0200
    ***************
    *** 70,75 ****
    --- 70,76 ----
      
      #define AD9528_SYSREF_K_DIVIDER			AD9528_2B(0x400)
      #define AD9528_SYSREF_CTRL			AD9528_2B(0x402)
    + #define AD9528_SYSREF_IN_CTRL			AD9528_1B(0x404)
      
      #define AD9528_PD_EN				AD9528_1B(0x500)
      #define AD9528_CHANNEL_PD_EN			AD9528_2B(0x501)
    ***************
    *** 1164,1169 ****
    --- 1165,1176 ----
      	if (ret < 0)
      		return ret;
      
    + 	if (pdata->sysref_src == SYSREF_SRC_EXTERNAL) {
    + 		ret = ad9528_write(indio_dev, AD9528_SYSREF_IN_CTRL, 1);
    + 		if (ret < 0)
    + 			return ret;
    + 	}
    + 
      	ret = ad9528_write(indio_dev, AD9528_PD_EN, AD9528_PD_BIAS |
      			   AD_IF(pll1_bypass_en, AD9528_PD_PLL1) |
      			   AD_IF(pll2_bypass_en, AD9528_PD_PLL2));
      			   

    This should be made a bit more general, but it seems to work. Error messages are gone now (see below), and oscilloscope seems to work at first glance.

    How can I check if JESD204 setup is correct?

    Best regards,

    Erik

    [    1.530545] jesd204: created con: id=0, topo=0, link=0, /amba/spi@ff040000/ad9528-1@0 <-> /fpga-axi@0/axi-adxcvr-tx@84a80000
    [    1.541156] jesd204: created con: id=1, topo=0, link=2, /amba/spi@ff040000/ad9528-1@0 <-> /fpga-axi@0/axi-adxcvr-rx-os@84a50000
    [    1.552573] jesd204: created con: id=2, topo=0, link=2, /fpga-axi@0/axi-adxcvr-rx-os@84a50000 <-> /fpga-axi@0/axi-jesd204-rx-os@84ab0000
    [    1.564769] jesd204: created con: id=3, topo=0, link=0, /fpga-axi@0/axi-adxcvr-tx@84a80000 <-> /fpga-axi@0/axi-jesd204-tx@84a90000
    [    1.576447] jesd204: created con: id=4, topo=0, link=0, /fpga-axi@0/axi-jesd204-tx@84a90000 <-> /fpga-axi@0/axi-adrv9009-tx-hpc@84a04000
    [    1.588662] jesd204: created con: id=5, topo=0, link=2, /fpga-axi@0/axi-jesd204-rx-os@84ab0000 <-> /amba/spi@ff050000/adrv9009-phy-b@1
    [    1.600677] jesd204: created con: id=6, topo=0, link=0, /fpga-axi@0/axi-adrv9009-tx-hpc@84a04000 <-> /amba/spi@ff050000/adrv9009-phy-b@1
    [    1.612907] jesd204: created con: id=7, topo=0, link=2, /amba/spi@ff050000/adrv9009-phy-b@1 <-> /amba/spi@ff040000/adrv9009-phy@1
    [    1.624483] jesd204: created con: id=8, topo=0, link=0, /amba/spi@ff050000/adrv9009-phy-b@1 <-> /amba/spi@ff040000/adrv9009-phy@1
    [    1.636060] jesd204: /amba/spi@ff040000/adrv9009-phy@1: JESD204[0] transition uninitialized -> initialized
    [    1.645650] jesd204: /amba/spi@ff040000/adrv9009-phy@1: JESD204[2] transition uninitialized -> initialized
    [    1.655251] jesd204: found 8 devices and 1 topologies
    [    3.400378] axi_sysid 85000000.axi-sysid-0: [adrv9009] on [zcu102] git <0> dirty [2021-04-29 12:11:06] UTC
    [    3.741272] zynqmp_clk_divider_set_rate() set divider failed for spi1_ref_div1, ret = -13
    [    5.002298] ad9528 spi1.0: spi1.0 supply vcc not found, using dummy regulator
    [    5.009473] ad9528 spi1.0: Linked as a consumer to regulator.0
    [    5.037279] jesd204: /amba/spi@ff040000/ad9528-1@0,jesd204:0,parent=spi1.0: Using as SYSREF provider
    [    5.046738] adrv9009 spi1.1: adrv9009_probe : enter
    [    5.057222] ad9528 spi2.0: spi2.0 supply vcc not found, using dummy regulator
    [    5.064382] ad9528 spi2.0: Linked as a consumer to regulator.0
    [    5.092685] adrv9009 spi2.1: adrv9009_probe : enter
    [    5.935780] cf_axi_dds 84a04000.axi-adrv9009-tx-hpc: Analog Devices CF_AXI_DDS_DDS MASTER (9.01.b) at 0x84A04000 mapped to 0x(____ptrval____), probed DDS AD9371
    [    5.952133] axi_adxcvr 84a50000.axi-adxcvr-rx-os: AXI-ADXCVR-RX (17.01.a) using CPLL on GTH4 at 0x84A50000. Number of lanes: 4.
    [    5.964395] axi_adxcvr 84a80000.axi-adxcvr-tx: AXI-ADXCVR-TX (17.01.a) using QPLL on GTH4 at 0x84A80000. Number of lanes: 8.
    [    5.977756] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition initialized -> probed
    [    5.988899] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition initialized -> probed
    [    6.000034] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition probed -> idle
    [    6.010553] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition probed -> idle
    [    6.021070] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition idle -> device_init
    [    6.032020] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition idle -> device_init
    [    6.042973] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition device_init -> link_init
    [    6.054356] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition device_init -> link_init
    [    6.065747] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_init -> link_supported
    [    6.077390] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_init -> link_supported
    [    6.093823] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_supported -> link_pre_setup
    [    6.105909] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_supported -> link_pre_setup
    [    6.117992] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_pre_setup -> clk_sync_stage1
    [    6.130157] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_pre_setup -> clk_sync_stage1
    [    6.142323] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage1 -> clk_sync_stage2
    [    6.154576] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage1 -> clk_sync_stage2
    [    6.166829] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage2 -> clk_sync_stage3
    [    6.179084] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage2 -> clk_sync_stage3
    [    6.191352] jesd204: /fpga-axi@0/axi-jesd204-rx-os@84ab0000,jesd204:5,parent=84ab0000.axi-jesd204-rx-os: Possible instantiation for multiple chips; HDL lanes 4, Link[2] lanes 2
    [    6.207089] jesd204: /fpga-axi@0/axi-jesd204-tx@84a90000,jesd204:4,parent=84a90000.axi-jesd204-tx: Possible instantiation for multiple chips; HDL lanes 8, Link[0] lanes 4
    [    6.223412] adrv9009 spi2.1: ADIHAL_resetHw
    [    6.543506] adrv9009 spi1.1: ADIHAL_resetHw
    [    6.863431] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage3 -> link_setup
    [    6.875263] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage3 -> link_setup
    [    6.887550] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_setup -> opt_setup_stage1
    [    6.899458] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_setup -> opt_setup_stage1
    [   11.569182] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage1 -> opt_setup_stage2
    [   11.581615] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage1 -> opt_setup_stage2
    [   11.594274] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage2 -> opt_setup_stage3
    [   11.606703] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage2 -> opt_setup_stage3
    [   11.619458] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage3 -> opt_setup_stage4
    [   11.631885] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage3 -> opt_setup_stage4
    [   16.146477] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage4 -> opt_setup_stage5
    [   16.158906] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage4 -> opt_setup_stage5
    [   16.172746] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage5 -> clocks_enable
    [   16.184914] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage5 -> clocks_enable
    [   16.197420] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clocks_enable -> link_enable
    [   16.209150] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clocks_enable -> link_enable
    [   16.221298] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_enable -> link_running
    [   16.232945] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_enable -> link_running
    [   16.345997] adrv9009 spi2.1: adrv9009_info: adrv9008-2 Rev 192, Firmware 6.0.2 API version: 3.6.0.5 successfully initialized via jesd204-fsm
    [   16.460951] adrv9009 spi1.1: adrv9009_info: adrv9008-2 Rev 192, Firmware 6.0.2 API version: 3.6.0.5 successfully initialized via jesd204-fsm
    [   16.473556] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_running -> opt_post_running_stage
    [   16.486156] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_running -> opt_post_running_stage
    [   23.940985] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_post_running_stage -> opt_post_running_stage
    [   23.940995] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_post_running_stage -> opt_post_running_stage
    [   23.941008] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_post_running_stage -> link_running
    [   23.941014] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_post_running_stage -> link_running
    [   23.941030] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_running -> link_enable
    [   23.941035] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_running -> link_enable
    [   23.941054] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_enable -> clocks_enable
    [   23.941059] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_enable -> clocks_enable
    [   23.941070] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clocks_enable -> opt_setup_stage5
    [   23.941075] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clocks_enable -> opt_setup_stage5
    [   23.941085] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage5 -> opt_setup_stage4
    [   23.941091] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage5 -> opt_setup_stage4
    [   23.941101] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage4 -> opt_setup_stage3
    [   23.941107] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage4 -> opt_setup_stage3
    [   23.941117] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage3 -> opt_setup_stage2
    [   23.941122] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage3 -> opt_setup_stage2
    [   23.941133] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage2 -> opt_setup_stage1
    [   23.941138] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage2 -> opt_setup_stage1
    [   23.941152] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage1 -> link_setup
    [   23.941158] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage1 -> link_setup
    [   23.941168] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_setup -> clk_sync_stage3
    [   23.941173] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_setup -> clk_sync_stage3
    [   23.941182] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage3 -> clk_sync_stage2
    [   23.941188] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage3 -> clk_sync_stage2
    [   23.941198] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage2 -> clk_sync_stage1
    [   23.941203] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage2 -> clk_sync_stage1
    [   23.941213] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage1 -> link_pre_setup
    [   23.941219] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage1 -> link_pre_setup
    [   23.941230] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_pre_setup -> link_supported
    [   23.941235] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_pre_setup -> link_supported
    [   23.941246] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_supported -> link_init
    [   23.941251] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_supported -> link_init
    [   23.941266] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_init -> device_init
    [   23.941271] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_init -> device_init
    [   23.941281] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition device_init -> idle
    [   23.941286] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition device_init -> idle
    [   23.941305] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition idle -> idle
    [   23.941311] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition idle -> idle
    [   23.941321] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition idle -> device_init
    [   23.941327] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition idle -> device_init
    [   23.941338] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition device_init -> link_init
    [   23.941343] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition device_init -> link_init
    [   23.941358] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_init -> link_supported
    [   23.941364] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_init -> link_supported
    [   24.175872] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_supported -> link_pre_setup
    [   24.175881] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_supported -> link_pre_setup
    [   24.175895] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_pre_setup -> clk_sync_stage1
    [   24.175900] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_pre_setup -> clk_sync_stage1
    [   24.175910] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage1 -> clk_sync_stage2
    [   24.175916] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage1 -> clk_sync_stage2
    [   24.175926] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage2 -> clk_sync_stage3
    [   24.175931] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage2 -> clk_sync_stage3
    [   24.175941] jesd204: /fpga-axi@0/axi-jesd204-rx-os@84ab0000,jesd204:5,parent=84ab0000.axi-jesd204-rx-os: Possible instantiation for multiple chips; HDL lanes 4, Link[2] lanes 2
    [   24.175959] jesd204: /fpga-axi@0/axi-jesd204-tx@84a90000,jesd204:4,parent=84a90000.axi-jesd204-tx: Possible instantiation for multiple chips; HDL lanes 8, Link[0] lanes 4
    [   24.176984] adrv9009 spi2.1: ADIHAL_resetHw
    [   25.770155] adrv9009 spi1.1: ADIHAL_resetHw
    [   26.108220] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clk_sync_stage3 -> link_setup
    [   26.108229] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clk_sync_stage3 -> link_setup
    [   26.108503] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_setup -> opt_setup_stage1
    [   26.108509] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_setup -> opt_setup_stage1
    [   36.366302] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage1 -> opt_setup_stage2
    [   36.366311] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage1 -> opt_setup_stage2
    [   36.366659] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage2 -> opt_setup_stage3
    [   36.366666] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage2 -> opt_setup_stage3
    [   36.367029] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage3 -> opt_setup_stage4
    [   36.367035] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage3 -> opt_setup_stage4
    [   40.784946] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage4 -> opt_setup_stage5
    [   40.784955] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage4 -> opt_setup_stage5
    [   40.786285] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition opt_setup_stage5 -> clocks_enable
    [   40.786294] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition opt_setup_stage5 -> clocks_enable
    [   40.786592] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition clocks_enable -> link_enable
    [   40.786599] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition clocks_enable -> link_enable
    [   40.787007] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_enable -> link_running
    [   40.787013] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_enable -> link_running
    [   40.891734] adrv9009 spi2.1: adrv9009_info: adrv9008-2 Rev 192, Firmware 6.0.2 API version: 3.6.0.5 successfully initialized via jesd204-fsm
    [   40.994251] adrv9009 spi1.1: adrv9009_info: adrv9008-2 Rev 192, Firmware 6.0.2 API version: 3.6.0.5 successfully initialized via jesd204-fsm
    [   40.994269] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[0] transition link_running -> opt_post_running_stage
    [   40.994276] jesd204: /amba/spi@ff040000/adrv9009-phy@1,jesd204:1,parent=spi1.1: JESD204[2] transition link_running -> opt_post_running_stage
    

Children
  • Still some issues.

    If I set adi,sysref-pattern-mode  to SYSREF_PATTERN_NSHOT, as it is the default for ADRV9009, I get : adrv9009 spi2.1: Link2 TAL_FRAMER_A framerStatus 0x6F

    and jesd_status looks like below.

    If I use SYSREF_PATTERN_CONTINUOUS, the error is gone and the latency values of the rx-os link are better, but the tx link is still deasserted. In case the SYNC has something to do with the sync lines of the FMC: I have ANDed both tx_sync anc connected the output to axi_adrv9009_tx_jesd/sync of the PL. Hope this is correct.

    Any ideas?

    Thank you, and best regards,

    Erik

  • +1
    •  Analog Employees 
    on May 4, 2021 4:16 PM in reply to EHeinz689

    Hi Erik,

    First of all SYNC de-asserted is good. Your links are in DATA - that's where you want to be.

    If SYNC is asserted you're typically in CGS, that's not so good.

    The framer errors are explained here: 

    https://github.com/analogdevicesinc/linux/blob/master/drivers/iio/adc/talise/talise_jesd204.h#L255

    You're getting a "SYSREF phase error -  a new SYSREF had different timing than the first that set the LMFC timing."

    Most likely on the part that is connected via the external SYSREF input. You should probe the SYSREFs on both cards.

    Then make sure that they assert the same time - I guess the ones on the slave part have some delays. Use output delays to compensate for that. Make sure dev_clk is also compensated. Both parts on both cards need to be in sync both in terms of dev_clk and SYSREF.

    -Michael

  • It took me some time to measure the signals correctly and to understand the AD9528 details. I am now able to compensate the delay and to align the SYSREFs.

    Whatever I do, however, the  adrv9009 spi2.1: Link2 TAL_FRAMER_A framerStatus 0x6F message is there. It only goes away if I switch to continuous SYSREF. Anyway, the ADRVs both seem to work and there are no other errors. May be, I could ignore the 0x6F?

    I found one more issue when I tried to change the ADRV9008-2 profiles to 491.520 Msps. Changing the profiles at runtime using /sys/bus/iio/devices/iio:device2/profile_config changes only the profile of the first ADRV. The messages are:

    [ 104.452939] adrv9009 spi2.1: Link2 TAL_FRAMER_A framerStatus 0x6F
    [ 104.452985] adrv9009 spi2.1: Link0 TAL_DEFRAMER_A deframerStatus 0x12
    [ 104.453031] adrv9009 spi1.1: Link0 TAL_DEFRAMER_A deframerStatus 0x12
    [ 104.453112] adrv9009 spi1.1: Link2 TAL_FRAMER_A framerStatus 0x2F

    Configuring the 491.520 Msps profile at boot via the device tree works, though. Again there are some messages regarding the framesStatus:

    [ 42.118306] adrv9009 spi2.1: Link2 TAL_FRAMER_A framerStatus 0x2F
    [ 42.118491] adrv9009 spi1.1: Link2 TAL_FRAMER_A framerStatus 0x2F

    but both ADRV work correctly at 491.52 MHz sampling rate.

    So far, so good.

    Best regards,

    Erik

  • +1
    •  Analog Employees 
    on May 6, 2021 10:44 AM in reply to EHeinz689
    Whatever I do, however, the  adrv9009 spi2.1: Link2 TAL_FRAMER_A framerStatus 0x6F message is there. It only goes away if I switch to continuous SYSREF. Anyway, the ADRVs both seem to work and there are no other errors. May be, I could ignore the 0x6F?

    You may need to see if there is a glitch, or some floating SYSREF when you use NSHOT SYSREF.

    Try to probe the SYSREF outputs on both clock chips and see if you can find irregularities.

    I found one more issue when I tried to change the ADRV9008-2 profiles to 491.520 Msps. Changing the profiles at runtime using /sys/bus/iio/devices/iio:device2/profile_config changes only the profile of the first ADRV.

    You need to write the same profile to each device. And you need to make sure that you write the profile to the TOP device last. This way all devices have the same config when the link restart is triggered at the TOP device.

    -Michael