Post Go back to editing

AD9144 sometimes goes to blanking state

Category: Software
Product Number: AD9144

Hello,

we are experiencing a problem similar to the one described in this post:

https://ez.analog.com/data_converters/high-speed_dacs/f/q-a/22410/ad9144-blanking-state-machine-behavior/412114

we have a custom board that uses the AD9144. We were able to set up the JESD link successfully for several configurations, but one of them (that happens to be the one that we want to use) has an unstable behavior. Upon boot, the system starts correctly roughly 1 time out of 3 (randomly). To see if the system is OK, we check the default 40 MHz sine wave from the internal DDS on the oscilloscope. When it starts correclty, the 40 MHz sine wave is visible and it remains OK until the next reboot. When it fails, no output is visible on the oscilloscope and it stays this way until the next reboot.

When it fails, the jesd_status ouptut looks fine (all "green", SYSREF captured, etc), but checking the AD9144 SPI registers reveals that "blanking mode" is activated (see linked post).

The only differences when I compare the SPI registers (the ones listed in the AD9144 datasheet) are:

0x038 SYNC_LASTERR_L (good: 0xF4   bad: 0xEB)

0x147 SOFTBLANKRB (good: 0xC0 bad: 0x00)

0x302 DYN_LINK_LATENCY_0 (good: 0x0 bad: 0xF)

Some details about the configuration:

The configuration of the AD9144 we want is with 128 MHz data input rate, interpolation x2 (done with the AD9144 internal interpolator), this gives a 256 MHz final sample rate.

The relevant portion of our device tree is the following:

 

&axi_quad_spi_daq {
        #address-cells = <0x1>;
        #size-cells = <0x0>;
        ad9144_spi: ad9144@1 {

                #address-cells = <0x1>;
                #size-cells = <0x0>;
                compatible = "ad9144";
                reg = <0x1>;
                spi-cpol;
                spi-cpha;
                spi-max-frequency = <1000000>;
                clocks = <&axi_ad9144_jesd>, <&idt8t49n1012 0>, <&idt8t49n1012 1>;
                clock-names = "jesd_dac_clk", "dac_clk", "dac_sysref";
                txen-gpios = <&axi_gpio_1 41 8>, <&axi_gpio_1 42 8>;
                reset-gpios = <&axi_gpio_1 40 8>;
                irq-gpios = <&axi_gpio_1 34 8>;
                adi,interpolation = <2>;
                adi,jesd-xbar-lane0-sel = <0x0>;
                adi,jesd-xbar-lane1-sel = <0x1>;
                adi,jesd-xbar-lane2-sel = <0x2>;
                adi,jesd-xbar-lane3-sel = <0x3>;
                adi,jesd-xbar-lane4-sel = <0x4>;
                adi,jesd-xbar-lane5-sel = <0x5>;
                adi,jesd-xbar-lane6-sel = <0x6>;
                adi,jesd-xbar-lane7-sel = <0x7>;
                adi,jesd-link-mode = <6>; // 4 is the default
                adi,jesd-subclass = <1>;
                };

};

&amba_pl {
                /delete-node/ ad_ip_jesd204_tpl_dac@44a50000;  // this becomes axi-ad9144-hpc@44a04000
                /delete-node/ axi_adxcvr@44a30000;

                axi_adxcvr_0: axi-ad9144-adxcvr@44a30000 {
                        compatible = "adi,axi-adxcvr-1.0";
                        reg = <0x44a30000 0x10000>;
                        clocks = <&idt8t49n1012 2>;
                        clock-names = "conv";
                        adi,sys-clk-select = <XCVR_CPLL>;
                        adi,out-clk-select = <XCVR_REFCLK_DIV2>;
                        adi,use-lpm-enable;
                        #clock-cells = <0x1>;
                        clock-output-names = "dac_gt_clk", "tx_out_clk";
                };


                axi_ad9144_jesd: axi-jesd204-tx@44a20000 {
                        compatible = "adi,axi-jesd204-tx-1.0";
                        reg = <0x44a20000 0x4000>;
                        interrupt-parent = <&axi_intc_0>;
                        interrupts = <22 2>; 
                        clocks = <&clk_bus_0 &axi_adxcvr_0 1 &axi_adxcvr_0 0>;
                        clock-names = "s_axi_aclk", "device_clk", "lane_clk";
                        adi,octets-per-frame = <2>;
                        adi,frames-per-multiframe = <0x20>;
                        adi,converter-resolution = <16>;
                        adi,bits-per-sample = <16>;
                        adi,converters-per-device = <2>;
                        #clock-cells = <0x0>;
                        clock-output-names = "jesd_dac_lane_clk";
                        adi,subclass = <1>;
                };

                
                axi-ad9144-hpc@44a50000 {
                        compatible = "adi,axi-ad9144-1.0";
                        reg = <0x44a50000 0x10000>;
                        spibus-connected = <&ad9144_spi>;
                        adi,axi-pl-fifo-enable;
                };
};

The JESD configuration we are trying to use is number "6" that has 2 converters and 2 lanes (M=2, L=2, S=1, F=2, HD=0, N=16, NP=16).

On the FPGA side, we modified the design accordingly and regenerated the transceiver configuration with the Xilinx transceiver wizard for our target configuration. We are using the CPLL, but switching to QPLL seems to give the same behavior.

To feed the input clock we are using an IDT8T49N1012 clock generator. We are sending 256 MHz as JESD refclk for the AD9144 and 128 MHz for the FPGA, SYSREF = 2MHz for both (side note: to make the system work we had to write a kernel module for the IDT8T49N1012 that is able to respond to "clk_set_rate" and "clk_get_rate" API calls required for the JESD kernel drivers). 

I'd like to mention that we tested other configurations that didn't have this problem, for example this one worked fine: 2 converters @256 MHz data rate with x2 interpolation (512 MHz final sample rate).

Another point, I noticed that on the datasheet there is no mention of the minimum DAC sampling rate and I'm not sure if ours (256 MHz sample rate after interpolation) is acutally supported.

The questions is: do you have any suggestions to debug this issue?



cleaned up gpios in device tree
[edited by: andrearu at 1:29 PM (GMT -4) on 6 Jul 2022]
Parents
  • Hi Andrearu,

    Sorry for the delayed reply.

    The JESD configuration we are trying to use is number "6" that has 2 converters and 2 lanes (M=2, L=2, S=1, F=2, HD=0, N=16, NP=16).
    I'd like to mention that we tested other configurations that didn't have this problem, for example this one worked fine: 2 converters @256 MHz data rate with x2 interpolation (512 MHz final sample rate).

    Since your setup is working in other JESD modes and not working in mode 6, it might be because the resulting JESD lane rate in mode 6 is not within 1.44Gbps and 12.4Gbps. Please confirm using the formula in page 31 of the datasheet.


    Regards,
    Shine

  • Hello, thank you for replying. In my case the lane rate is 2.56 GHz, so it's within the limits.

    Best,

    Andrea

  • Hi Andrea,

    Another thing: If you're using the DAC Clock PLL, please check if the resulting fVCO is within the range shown below.




    Regards,
    Shine

  • Hi Shine,

    in my case I'm feeding a 256 MHz clock from my clock generator (IDT8T49N1012) directly to the AD9144 without using the DAC internal PLL (otherwise the "adi,pll-enable" and "adi,pll-frequency" entries would be included in the device tree).

    Regards,

    Andrea

  • Hi Andrea,

    Thank you for providing the details I asked for.

    I tried replicating your settings with our eval platform. However, we are still waiting for the AD9144 FMC board to be returned, so I used the AD9152 FMC board. These devices are from the same DAC family and only differs in the speed grade and no. of DAC channels.



    I disabled the BSM or Digital Gain, in particular, by disconnecting the P2 jumper on the eval board, effectively leaving TXEN high.





    I tried the following:
    1. JESD Mode 6 resulting to 5.12Gsps Lane Rate, Subclass 0 operation;
    2. JESD Mode 5 resulting to 2.56Gsps Lane Rate, Subclass 0 operation; and
    3. JESD Mode 5 resulting to 2.56Gsps Lane Rate, Subclass 1 operation.

    All didn't have problems on output generation.

    I attached the register read and write sequence files generated from the ACE software. The files follow the format below:


    For setup 3, I included the register readback file with following format:


    I used this wiki page as guide during evaluation.

    Hope these are helpful.

    Regards,
    Shine

    JESDMode6_5.12GspsLaneRate_Subclass0.zip
    JESDMode5_2.56GspsLaneRate_Subclass0.zip
    8780.JESDMode5_2.56GspsLaneRate_Subclass1.zip

  • Hi Shine,

    thank you for taking the time to test this.

    The configuration that has troubles is mode 6 with 2.56 GHz lane rate with subclass 0 or 1. It fails to start about 50% of the times. It seems this configuration is not in your list.

    I mentioned a configuration with double the data rate (so mode 6 with 5.12 GHz lane rate) but that had no problems.

    If possible, could you please try mode 6 with 2.56 GHz lane rate (either with subclass 0 or 1) a few times to see if it's stable?

    By the way I checked the digital gain register and is set to 1 (SPI register 0x111 is set to 0xA4).

    TXen are set both high by the driver.

    Thank you,

    regards,

    Andrea.

  • Assumed answered. Please let us know if there are further questions.

  • Dear deljones,

    the problem is not yet solved. Since you guys took the effort to test a chip of the same family of the AD9144 (AD9152), could you please verify that mode 6 with 2.56 GHz lane rate is stable with subclass 0 or 1? Three different setups were tested (thanks for that!) but not the problematic combination I reported.

    Thank you,

    regards.

    Andrea

  • Hi Andrea,

    I think we have finally replicated your problem. We noticed the difference between the AD9152 eval board I used earlier with the AD9144 board I used just recently. The AD9152 eval board has a pullup resistor for the TXEN pin while the AD9144 does not. This is what I did to the AD9144 eval board:


    This is test circuit 2 while the original schematic is test circuit 1.

    If on startup the TXEN pin ramps up to logic high then held low, we allow the DAC to flush out invalid data. So that when the TXEN pin is enabled again, it outputs waveforms corresponding to the latest data sent to the DAC.

    Please see attached software screenshots and waveform captures.

    test_circuit_1.zip

    test_circuit_2.zip

    By the way, here is the resulting lane rate with FDAC = 260MHz and with no interpolation:

     

     

    Data Rate = 260MHz

    Lane Rate = (20 *260MHz * 2)/2 = 5.2GHz


    Please do let us know if this solves your problem.

    Regards,
    Shine

  • Dear Shine,

    thanks for the helpful information! So if I understand correctly the schematic, I should tie TXEN0 and TXEN2 high with a pull up, and disconnect the controlling logic. The two red crosses in the schematic mean "disconnect" from the contorlling logic TXEN_0 and TXEN_1, right?

    On our custom board, TXEN0 and TXEN1 are connected to GPIOs of the FPGA, the Linux driver sets them to "high" when the module is probed. So, we will try to force them always high.

    ad9144.c
    
    	for (i = 0; i < 2; i++) {
    		conv->txen_gpio[i] = devm_gpiod_get_index_optional(&spi->dev,
    			"txen", i, GPIOD_OUT_HIGH);
    		if (IS_ERR(conv->txen_gpio[i]))
    			return PTR_ERR(conv->txen_gpio[i]);
    	}

    Thank you,

    Andrea

  • Hello,

    we forced both TXENs to "1" in the FPGA constraints, and also put an hardware pull up on both pins (soldered 1k resistor to 1.8V in our case). The statistics seems to improve (the number of successful startup attempts was higher) but sometimes the system started with blanking activated, unfortunately.

    Anyway, you said you could reproduce our problem. Did you get blanking=ON in certain conditions? What was the JESD lane rate when it happened? 

    Thank you,

    best regards,

    Andrea 

  • Hi,

    it looks like we found the problem. The AD9144 datasheet describes a procedure to calibrate some delays called LMFCDEL and LMFCVAR. These delays are related to the lane synchronization stage. The problem was, we were using the Linux kernel module and we were erreneously assuming that the kernel module took care of that automatically. That is not possible, because calibrating them requires multiple reboots and monitoring (see section "Link delay setup" and "Without Known Delays"). The misleading part is also the fact that these settings should be exposed to the device tree, but they aren't, some values are hardcoded in the kernel module instead (see ad9144.c source). I patched the kernel module and put there my constants, but as I said it would be better if they were editable via the device tree.

    To my understanding, this problem can affect not only custom boards, but also the evaluation board DAQ2 for example, depending on the clock configuration. So users should be warned.

    	/* LMFC settings for link 0 */
    	regmap_write(map, 0x304, 0x00);	// lmfc delay
    	regmap_write(map, 0x306, 0x0a);	// receive buffer delay
    	if (AD9144_ID_GET_PRODUCT_ID(st->id) == 0x9144 ||
    	    AD9144_ID_GET_PRODUCT_ID(st->id) == 0x9154) {
    		/* LMFC settings for link 1 */
    		regmap_write(map, 0x305, 0x00);	// lmfc delay
    		regmap_write(map, 0x307, 0x0a);	// receive buffer delay
    	}
    

    Regards,

    Andrea

Reply
  • Hi,

    it looks like we found the problem. The AD9144 datasheet describes a procedure to calibrate some delays called LMFCDEL and LMFCVAR. These delays are related to the lane synchronization stage. The problem was, we were using the Linux kernel module and we were erreneously assuming that the kernel module took care of that automatically. That is not possible, because calibrating them requires multiple reboots and monitoring (see section "Link delay setup" and "Without Known Delays"). The misleading part is also the fact that these settings should be exposed to the device tree, but they aren't, some values are hardcoded in the kernel module instead (see ad9144.c source). I patched the kernel module and put there my constants, but as I said it would be better if they were editable via the device tree.

    To my understanding, this problem can affect not only custom boards, but also the evaluation board DAQ2 for example, depending on the clock configuration. So users should be warned.

    	/* LMFC settings for link 0 */
    	regmap_write(map, 0x304, 0x00);	// lmfc delay
    	regmap_write(map, 0x306, 0x0a);	// receive buffer delay
    	if (AD9144_ID_GET_PRODUCT_ID(st->id) == 0x9144 ||
    	    AD9144_ID_GET_PRODUCT_ID(st->id) == 0x9154) {
    		/* LMFC settings for link 1 */
    		regmap_write(map, 0x305, 0x00);	// lmfc delay
    		regmap_write(map, 0x307, 0x0a);	// receive buffer delay
    	}
    

    Regards,

    Andrea

Children
No Data