ADRV9009-ZU11EG Multi-SOM: 3 SOMs delayed from other 3

I'm running 6 SOMs (without FMCOMMS8) simultaneously, following the multi-SOM guide here.

I'm finding that when I run multi.rx, the second, third, and fifth SOMs are all delayed by approximately 138ns.  So what I have is one set of 3 that are in sync, and another set of 3 that are in sync but delayed.

I've tried adding adding a coarse digital delay in the devicetree and that seemed to make no difference.

I've also tried using the multi.hmc7044_ext_output_delay() function within my python script, which also seems to make no difference.

Noting that my VCO frequency is 122.88 MHz, the 138ns delay I'm seeing would appear to be 17 cycles, which leads me to believe this is related to the coarse digital delay.

Here's the portion of my primary device tree that pertains to hmc7044-ext:

			hmc7044-ext@4 {
				#address-cells = < 0x01 >;
				#size-cells = < 0x00 >;
				#clock-cells = < 0x01 >;
				compatible = "adi,hmc7044";
				reg = < 0x04 >;
				spi-max-frequency = < 0xf4240 >;
				adi,pll1-clkin-frequencies = < 0x00 0x1d4c000 0x00 0x00 >;
				adi,pll1-loop-bandwidth-hz = < 0xc8 >;
				adi,pfd1-maximum-limit-frequency-hz = < 0x3a9800 >;
				adi,vcxo-frequency = < 0x7530000 >;
				adi,pll2-output-frequency = < 0xafc80000 >;
				adi,sysref-timer-divider = < 0xf00 >;
				adi,pulse-generator-mode = < 0x01 >;
				adi,oscin-buffer-mode = < 0x15 >;
				adi,clkin1-buffer-mode = < 0x07 >;
				adi,sync-pin-mode = < 0x01 >;
				adi,gpi-controls = < 0x00 0x00 0x00 0x00 >;
				adi,gpo-controls = < 0x1f 0x2b 0x00 0x00 >;
				clock-output-names = "hmc7044_e_out0_REFCLK_OUT0\0hmc7044_e_out1_SYNC_OUT0\0hmc7044_e_out2_REFCLK_OUT1\0hmc7044_e_out3_SYNC_OUT1\0hmc7044_e_out4_REFCLK_OUT2\0hmc7044_e_out5_SYNC_OUT2\0hmc7044_e_out6_REFCLK_OUT3\0hmc7044_e_out7_SYNC_OUT3\0hmc7044_e_out8_REFCLK_OUT4\0hmc7044_e_out9_SYNC_OUT4\0hmc7044_e_out10_REFCLK_OUT5\0hmc7044_e_out11_SYNC_OUT5\0hmc7044_e_out12_REFCLK_OUT6\0hmc7044_e_out13_SYNC_OUT6";
				jesd204-device;
				#jesd204-cells = < 0x02 >;
				jesd204-sysref-provider;
				adi,hmc-two-level-tree-sync-en;
				phandle = < 0x25 >;

				channel@0 {
					reg = < 0x00 >;
					adi,extended-name = "REFCLK_OUT0";
					adi,divider = < 0x60 >;
					adi,driver-mode = < 0x01 >;
					phandle = < 0x95 >;
				};

				channel@2 {
					reg = < 0x02 >;
					adi,extended-name = "REFCLK_OUT1";
					adi,divider = < 0x60 >;
					adi,driver-mode = < 0x01 >;
					phandle = < 0x96 >;
				};

				channel@5 {
					reg = < 0x05 >;
					adi,extended-name = "SYNC_OUT2";
					adi,divider = < 0xf00 >;
					adi,driver-mode = < 0x03 >;
					adi,startup-mode-dynamic-enable;
					adi,high-performance-mode-disable;
					adi,driver-impedance-mode = < 0x03 >;
					phandle = < 0x97 >;
				};

				channel@6 {
					reg = < 0x06 >;
					adi,extended-name = "REFCLK_OUT3";
					adi,divider = < 0x60 >;
					adi,driver-mode = < 0x01 >;
					phandle = < 0x98 >;
				};

				channel@11 {
					reg = < 0x0b >;
					adi,extended-name = "SYNC_OUT5";
					adi,divider = < 0xf00 >;
					adi,driver-mode = < 0x03 >;
					adi,startup-mode-dynamic-enable;
					adi,high-performance-mode-disable;
					adi,driver-impedance-mode = < 0x03 >;
					phandle = < 0x99 >;
				};

				channel@1 {
					reg = < 0x01 >;
					adi,extended-name = "SYNC_OUT0";
					adi,divider = < 0xf00 >;
					adi,driver-mode = < 0x03 >;
					adi,startup-mode-dynamic-enable;
					adi,high-performance-mode-disable;
					adi,driver-impedance-mode = < 0x03 >;
					phandle = < 0x9a >;
				};

				channel@3 {
					reg = < 0x03 >;
					adi,extended-name = "SYNC_OUT1";
					adi,divider = < 0xf00 >;
					adi,driver-mode = < 0x03 >;
					adi,startup-mode-dynamic-enable;
					adi,high-performance-mode-disable;
					adi,driver-impedance-mode = < 0x03 >;
					phandle = < 0x9b >;
				};

				channel@4 {
					reg = < 0x04 >;
					adi,extended-name = "REFCLK_OUT2";
					adi,divider = < 0x60 >;
					adi,driver-mode = < 0x01 >;
					phandle = < 0x9c >;
				};

				channel@7 {
					reg = < 0x07 >;
					adi,extended-name = "SYNC_OUT3";
					adi,divider = < 0xf00 >;
					adi,driver-mode = < 0x03 >;
					adi,startup-mode-dynamic-enable;
					adi,high-performance-mode-disable;
					adi,driver-impedance-mode = < 0x03 >;
					phandle = < 0x9d >;
				};

				channel@8 {
					reg = < 0x08 >;
					adi,extended-name = "REFCLK_OUT4";
					adi,divider = < 0x60 >;
					adi,driver-mode = < 0x01 >;
					phandle = < 0x9e >;
				};

				channel@9 {
					reg = < 0x09 >;
					adi,extended-name = "SYNC_OUT4";
					adi,divider = < 0xf00 >;
					adi,driver-mode = < 0x03 >;
					adi,startup-mode-dynamic-enable;
					adi,high-performance-mode-disable;
					adi,driver-impedance-mode = < 0x03 >;
					phandle = < 0x9f >;
				};

				channel@10 {
					reg = < 0x0a >;
					adi,extended-name = "REFCLK_OUT5";
					adi,divider = < 0x60 >;
					adi,driver-mode = < 0x01 >;
					phandle = < 0xa0 >;
				};

				channel@12 {
					reg = < 0x0c >;
					adi,extended-name = "REFCLK_OUT6";
					adi,divider = < 0x60 >;
					adi,driver-mode = < 0x01 >;
					phandle = < 0xa1 >;
				};

				channel@13 {
					reg = < 0x0d >;
					adi,extended-name = "SYNC_OUT6";
					adi,divider = < 0xf00 >;
					adi,driver-mode = < 0x03 >;
					adi,startup-mode-dynamic-enable;
					adi,high-performance-mode-disable;
					adi,driver-impedance-mode = < 0x03 >;
					phandle = < 0xa2 >;
				};
			};
		};

Any suggestions?

Parents
  • An update: I have probed the SYNC lines on the clock board, and it appears that the signal is inverted on the ports that exhibit the delay.

    The rising clock edges are 138ns apart so I feel pretty certain that this is the cause.

    Is there a setting that has inverted 3 of my SYNC lines?  Which of the two is the proper SYNC signal?

  • 0
    •  Analog Employees 
    on Apr 29, 2021 6:53 AM in reply to mstone

    Hmm - I'm a bit confused now.

    HMC7044 SYNC signal should be CMOS this looks differential.

    In your device tree you correctly use:

    adi,driver-mode = <3>;

    The output buffers seem to activate/deactivate in sync, like you say they are just inverted.

    Wondering if you probe on one board OUTCLKx_P and on the other OUTCLKx_N?   

Reply
  • 0
    •  Analog Employees 
    on Apr 29, 2021 6:53 AM in reply to mstone

    Hmm - I'm a bit confused now.

    HMC7044 SYNC signal should be CMOS this looks differential.

    In your device tree you correctly use:

    adi,driver-mode = <3>;

    The output buffers seem to activate/deactivate in sync, like you say they are just inverted.

    Wondering if you probe on one board OUTCLKx_P and on the other OUTCLKx_N?   

Children
  • This isn't a differential signal.  This is one SYNC line (SCLKOUT7_P) on top of a different one (SCLKOUT5_P).

    They are both the P lines. I have replaced the in-line capacitor at the output with a 0-ohm resistor on both as well.

    When I look at the N lines, the signals aren't inversions of these.  Rather they are the same signal as the P without the DC offset, since I never replaced the capacitors on them.

    This trend is consistent with my other clock lines as well, inasmuch as I could probe them.  SCLKOUT7_P and SCLKOUT1_P are oriented one way, while SCLKOUT5_P and SCLKOUT3_P are inversions of that signal.  I expect this pattern follows on the remaining two ports I'm using but couldn't explicitly verify that.

  • 0
    •  Analog Employees 
    on Apr 29, 2021 8:54 PM in reply to mstone

    Well - I think I need some LAB time.

    I'll get back on this soon.

  • 0
    •  Analog Employees 
    on Apr 30, 2021 3:50 PM in reply to mstone

    Ok I spend some time looking into this.

    I think we're missing two channel attributes for SYSREF in CMOS mode.

    From e26c56c2119b6969af72706b8d9a9409002ab857 Mon Sep 17 00:00:00 2001
    From: Michael Hennerich <michael.hennerich@analog.com>
    Date: Fri, 30 Apr 2021 17:44:12 +0200
    Subject: [PATCH] dts: zynqmp-adrv9009-zu11eg-reva-adrv2crr-fmc-reva: HMC7044
     fix SYNC_OUT
    
    This tries to fix the CMOS output of the SYNC_OUT outputs.
    
    Signed-off-by: Michael Hennerich <michael.hennerich@analog.com>
    ---
     ...adrv9009-zu11eg-reva-adrv2crr-fmc-reva.dts | 20 +++++++++++++------
     1 file changed, 14 insertions(+), 6 deletions(-)
    
    diff --git a/arch/arm64/boot/dts/xilinx/zynqmp-adrv9009-zu11eg-reva-adrv2crr-fmc-reva.dts b/arch/arm64/boot/dts/xilinx/zynqmp-adrv9009-zu11eg-reva-adrv2crr-fmc-reva.dts
    index c57364f1ee86..de84eeb5aaaa 100644
    --- a/arch/arm64/boot/dts/xilinx/zynqmp-adrv9009-zu11eg-reva-adrv2crr-fmc-reva.dts
    +++ b/arch/arm64/boot/dts/xilinx/zynqmp-adrv9009-zu11eg-reva-adrv2crr-fmc-reva.dts
    @@ -411,20 +411,24 @@
     			reg = <5>;
     			adi,extended-name = "SYNC_OUT1";
     			adi,divider = <3840>;	// 768000
    -			adi,driver-mode = <3>;
    +			adi,driver-mode = <HMC7044_DRIVER_MODE_CMOS>;
     			adi,startup-mode-dynamic-enable;
     			adi,high-performance-mode-disable;
    -			adi,driver-impedance-mode = <3>;
    +			adi,dynamic-driver-enable;
    +			adi,force-mute-enable;
    +			adi,driver-impedance-mode = <HMC7044_DRIVER_IMPEDANCE_50_OHM>; /* Don't touch */
     		};
     
     		hmc7044_car_c6: channel@6 {
     			reg = <6>;
     			adi,extended-name = "SYNC_OUT2";
     			adi,divider = <3840>;	// 768000
    -			adi,driver-mode = <3>;
    +			adi,driver-mode = <HMC7044_DRIVER_MODE_CMOS>;
     			adi,startup-mode-dynamic-enable;
     			adi,high-performance-mode-disable;
    -			adi,driver-impedance-mode = <3>;
    +			adi,dynamic-driver-enable;
    +			adi,force-mute-enable;
    +			adi,driver-impedance-mode = <HMC7044_DRIVER_IMPEDANCE_50_OHM>; /* Don't touch */
     
     		};
     
    @@ -573,7 +577,9 @@
     			adi,driver-mode = <HMC7044_DRIVER_MODE_CMOS>;
     			adi,startup-mode-dynamic-enable;
     			adi,high-performance-mode-disable;
    -			adi,driver-impedance-mode = <3>; /* Don't touch */
    +			adi,dynamic-driver-enable;
    +			adi,force-mute-enable;
    +			adi,driver-impedance-mode = <HMC7044_DRIVER_IMPEDANCE_50_OHM>; /* Don't touch */
     		};
     
     		hmc7044_ext_c6: channel@6 {
    @@ -583,7 +589,9 @@
     			adi,driver-mode = <HMC7044_DRIVER_MODE_CMOS>;
     			adi,startup-mode-dynamic-enable;
     			adi,high-performance-mode-disable;
    -			adi,driver-impedance-mode = <3>; /* Don't touch */
    +			adi,dynamic-driver-enable;
    +			adi,force-mute-enable;
    +			adi,driver-impedance-mode = <HMC7044_DRIVER_IMPEDANCE_50_OHM>; /* Don't touch */
     		};
     
     		hmc7044_ext_c11: channel@11 {
    -- 
    2.25.1
    

    With this patch, the inactive signal level is no longer having some common mode voltage.

    However I have not yet tested how that influences synchronization.

    The other logic level inversion issue. I have not been able to see it on my end.

    Maybe I haven't looked at the right outputs. We use OUT5 and OUT6 and they don't show this.

    The only thing that might explain this is that the channel output dividers are not synced or reset.

    root@analog:~# iio_attr -d hmc7044-ext reset_dividers_request 1
    0
    
    root@analog:~# iio_attr -d hmc7044-ext reseed_request 1
    0
    
    root@analog:~# iio_attr -D hmc7044-ext status                             
    --- PLL1 ---
    Status: Locked
    Using:  CLKIN1 @ 30720000 Hz
    PFD:    3840 kHz
    --- PLL2 ---
    Status: Locked (Synchronized)
    Frequency:      2949120000 Hz (Autocal cap bank value: 13)
    SYSREF Status:  Valid & Locked
    SYNC Status:    Synchronized
    Lock Status:    PLL1 & PLL2 Locked
    root@analog:~#
    

    Can you try this, and see if your PLLs are locked, SYSREF Status is valid & locked?

    -Michael

  • Before implementing the patch in any way, here's what your commands yield:

    root@analog:~# iio_attr -D hmc7044-ext status
    --- PLL1 ---
    Status:	Holdover
    Using:	CLKIN1 @ 30720000 Hz
    PFD:	3840 kHz
    --- PLL2 ---
    Status:	Locked (Unsynchronized)
    Frequency:	2949120000 Hz (Autocal cap bank value: 13)
    SYSREF Status:	Invalid
    SYNC Status:	Unsynchronized
    Lock Status:	Unlocked
    
    root@analog:~# iio_attr -d hmc7044-ext reset_dividers_request 1
    0
    
    root@analog:~# iio_attr -d hmc7044-ext reseed_request 1
    0
    
    root@analog:~# iio_attr -D hmc7044-ext status
    --- PLL1 ---
    Status:	Holdover
    Using:	CLKIN1 @ 30720000 Hz
    PFD:	3840 kHz
    --- PLL2 ---
    Status:	Locked (Synchronized)
    Frequency:	2949120000 Hz (Autocal cap bank value: 13)
    SYSREF Status:	Valid & Locked
    SYNC Status:	Synchronized
    Lock Status:	Unlocked
    
    root@analog:~# 
    

  • After implementing the patch to primary and all secondary images, the time delay is unimproved.

    The other main change I have made within the device tree is to add support for the AD9545 clock chip:

    				i2c@1 {
    					#address-cells = <0x1>;
    					#size-cells = <0x0>;
    					reg = <0x1>;
    
    					ad9545@4A {
    						compatible = "adi,ad9545";
    						reg = <0x4a>;
    						#address-cells = <0x1>;
    						#size-cells = <0x0>;
    						adi,ref-crystal;
    						adi,ref-frequency-hz = <0x2ee0000>;
    						#clock-cells = <0x2>;
    						assigned-clocks = <0x18 0x2 0x0 0x18 0x1 0x1 0x18 0x0 0x6 0x18 0x0 0x8>;
    						assigned-clock-rates = <0x2710 0x6fc23ac0 0x9502f90 0x9502f90>;
    						assigned-clock-phases = <0x0 0x0 0x0 0xb4>;
    						phandle = <0x18>;
    
    						aux-nco-clk@0 {
    							reg = <0x0>;
    							adi,freq-lock-threshold-ps = <0xf42400>;
    							adi,phase-lock-threshold-ps = <0xf42400>;
    						};
    
    						pll-clk@1 {
    							reg = <0x1>;
    							#address-cells = <0x1>;
    							#size-cells = <0x0>;
    							phandle = <0x75>;
    
    							profile@0 {
    								reg = <0x0>;
    								adi,pll-source = <0x4>;
    								adi,profile-priority = <0x0>;
    								adi,pll-loop-bandwidth-uhz = <0xbebc200>;
    							};
    						};
    
    						output-clk@6 {
    							reg = <0x6>;
    							adi,output-mode = <0x0>;
    							adi,current-source-microamp = <0x3a98>;
    						};
    
    						output-clk@8 {
    							reg = <0x8>;
    							adi,output-mode = <0x0>;
    							adi,current-source-microamp = <0x3a98>;
    						};
    					};
    				};
    

    I don't think it likely, but is there any way this could be impacting the way the hmc7044-ext device is behaving?