ZC706_GTH_QPLL+ADRV9009-OBS450MSPS

Michael,

    I have been working on ORX491SPS profile almost one month and I am hungry for your guides. 

    HW: We are using ZC706(ZYNQ with speed rate 3) + our ADRV9009 daughter card. We are following your HDL design on github (branch "2018_R2") and your linux-kernel design (branch "2018_R2"). Our JESD204B subsystem has 4 Tx-lanes, 2 Rx-lanes, and 2 ORx-lanes. All of them shares one GTH quad of Xilinx-7z045. Our RF-FrontEnd Feedback path connects to ADRV9009 ORx2 rather than ORx1.Yes, our HW guy didn’t notice the limitation of GTH CPLL at the beginning.

    Debug Phase1 for function bringup: I customized "zynq-zc706-adv7511-adrv9009.dts" (which is following the profile “Tx_BW100_IR122p88_Rx_BW100_OR122p88_ORx_BW100_OR122p88_DC122p88”) into "fhk_adrv9009_122m.dtsi", with AD9528 output changes, our FPGA design changes, ORx-selection 2 and our specific JESD204B lanes layout(TX:M=4,L=4; Rx:M=4,L=2;ORx:M=2,L=2) etc. The whole system works normally: all JESD204B lanes can link up; initCal can pass; iio-scope GUI can get Rx/ORx signals.

    Debug Phase2: to get the profile “Tx_BW200_IR245p76_Rx_BW200_OR245p76_ORx_BW200_OR245p76_DC245p76” (neither dts nor txt style) workable, I root-caused this issue: The lane rate for Rx should be 9.8GHz, but it beyond the CPLL ability of Xilinx-7serial GTH. Our HW guy changed the FPGA design to use QPLL rather than CPLL for JESD204B Rx/ORx. I changed dts accordingly, and also changed "axi_adxcvr.c" to get the exected lane-rate. Eventally all JESD204B Tx/Rx/ORx lanes can initialize correctly, ADRV9009 can boot up, and all JESD204B lanes can link up correctly.

root@analog:~# grep "" /sys/bus/platform/devices/*jesd*/status
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:Link is enabled
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:Measured Link Clock: 122.882 MHz
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:Reported Link Clock: 122.880 MHz
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:Lane rate: 4915.200 MHz	#### 245.76*20 #### Yes
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:Lane rate / 40: 122.880 MHz
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:LMFC rate: 7.680 MHz
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:SYNC~: deasserted
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:Link status: DATA
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:SYSREF captured: Yes
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:SYSREF alignment error: No

/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:Link is enabled
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:Measured Link Clock: 245.766 MHz
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:Reported Link Clock: 245.760 MHz
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:Lane rate: 9830.400 MHz	#### 245.76*40 #### Yes
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:Lane rate / 40: 245.760 MHz
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:LMFC rate: 7.680 MHz
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:Link status: DATA
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:SYSREF captured: Yes
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:SYSREF alignment error: No

/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:Link is enabled
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:Measured Link Clock: 122.882 MHz
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:Reported Link Clock: 122.880 MHz
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:Lane rate: 4915.200 MHz	#### 245.76*20 #### Yes
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:Lane rate / 40: 122.880 MHz
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:LMFC rate: 7.680 MHz
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:Link status: DATA
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:SYSREF captured: Yes
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:SYSREF alignment error: No

    Debug Phase3: Our target application needs “Tx_BW450_IR491p52_Rx_BW200_OR245pY6_ORx_BW450_OR491p52_DC491p52”. I produced the corresponding profile with the FilterWizard at first, then converted this profile to "fhk_adrv9009_491m_dc245.dtsi", do necessary adjustments per the patch which Michael written in the original post. It was found that the lane-rate for JESD204B-Rx failed. The root-cause is the limitation of GTH CPLL.Then Our HW guy changed the FPGA design to use QPLL rather than CPLL for JESD204B Rx/ORx. I changed dts accordingly, and then changed "axi_adxcvr.c" so eventally got the Tx/Rx/ORx lanes linkup with the expected rate correctly when I skip "TALISE_runInitCals()" in "adrv9009.c".

root@analog:~# grep "" /sys/bus/platform/devices/*jesd*/status
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:Link is enabled
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:Measured Link Clock: 245.763 MHz
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:Reported Link Clock: 245.760 MHz
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:Lane rate: 9830.400 MHz	#### 491.52*20 #### Yes
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:Lane rate / 40: 245.760 MHz
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:LMFC rate: 15.360 MHz
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:SYNC~: deasserted
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:Link status: DATA
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:SYSREF captured: Yes
/sys/bus/platform/devices/44a90000.jesd204tx-layer2/status:SYSREF alignment error: No

/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:Link is enabled
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:Measured Link Clock: 245.763 MHz
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:Reported Link Clock: 245.760 MHz
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:Lane rate: 9830.400 MHz	#### 245.76*40 #### Yes
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:Lane rate / 40: 245.760 MHz
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:LMFC rate: 7.680 MHz
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:Link status: DATA
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:SYSREF captured: Yes
/sys/bus/platform/devices/44aa0000.jesd204rx-layer2/status:SYSREF alignment error: No

/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:Link is enabled
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:Measured Link Clock: 245.763 MHz
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:Reported Link Clock: 245.760 MHz
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:Lane rate: 9830.400 MHz	#### 491.52*20 #### Yes
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:Lane rate / 40: 245.760 MHz
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:LMFC rate: 15.360 MHz
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:Link status: DATA
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:SYSREF captured: Yes
/sys/bus/platform/devices/44ab0000.jesd204ro-layer2/status:SYSREF alignment error: No

But, after "TALISE_runInitCals()" is open, "TALISE_waitInitCals()" failed with "ERROR: 294: TALISE_waitArmCmdStatus() failed due to thrown ARM error. Is device in correct state for calling command?". I modified the mask with opening calibration items one by one, it’s found that the 3rd item "TAL_TIA_3DB_CORNER" lead to this error.

There are some codes for the stitching mode in "adrv9009.c", it will force M=2, F=2, and obsRxChannelsEnable=1. After I disabled "phy->talInit.obsRx.obsRxChannelsEnable = 1;", everything goes well. I understand "M=2, F=2" is kind of a fixup for the actual application with "4 converters and/or 4 lanes" but I am confused why "obsRxChannelsEnable=1" is forced as well. Could anyone explain further?

	if (orx_adc_stitching_enabled) {
		if (phy->talInit.obsRx.framerSel != 1) {
			dev_warn(&phy->spi->dev, "%s:%d: Can't apply fixup",
				 __func__, __LINE__);
		} else {
			if (framer_b_m != 2 || framer_b_f != 2 ||
				orx_channel_enabled != 1)
				dev_warn(&phy->spi->dev,
					 "%s:%d: ORx samples might be incorrect",
					 __func__, __LINE__);

			phy->talInit.jesd204Settings.framerB.M = 2;
			phy->talInit.jesd204Settings.framerB.F = 2;
			phy->talInit.obsRx.obsRxChannelsEnable = 1;
		}
	}

Although there is no error reported any more during adrv9009 initialization, I still hope that you could review our HW design changes and our SW design changes descripted here (includes 1, using the shared QPLL for Tx/Rx/ORx rather than seperated QPLL for Tx + CPLL for Rx/ORx; 2, using ORx2 rather than ORx1 in the stitch mode.), and please give your professional comments about "what should be tested and estimated further" and "what's the risk and how to manage the risk further".



InitCal error was fixed just now, I update the overall debug procedure in time sequence. Change the specific questions at last.
[edited by: jacky168wang at 5:34 AM (GMT 0) on 8 Apr 2019]