enabling ad9144 as a module

Hi, 

I'm trying to debug a setup with an ad9136 and to help with this, we're trying to build all the drivers associated with this chip as modules.

I was wondering if this has been tried before and if there are things I should be careful about.

At the moment, it looks like I might have an issue with the order in which the modules are loaded.

Thanks for your time, 

Liam

Top Replies

Parents
  • 0
    •  Analog Employees 
    on Aug 14, 2020 8:06 AM 6 months ago

    Hi Liam,

    We usually test the drivers by statically linking in the kernel. As you say some issues might arise with the order that you load the drivers. What drivers are you trying as a module and what is your exact order ?

    Regards,

    Mircea

  • Hi Mircea,

    At the moment, we're trying to boot with all drivers loaded as modules. Since we are using an AD9136-FMC-EBZ, this also includes the drivers for the ad9516-1.

    here is a list of the module names:

    - ad9517

    - ad9144

    - cf_axi_dds_drv

    axi_jesd204_tx

    - axi_adxcvr_drv

    - dma_axi_dmac

    As for the loading order, I'm relying on the devicetree to load everything automatically.

    Thanks,

    Liam

  • 0
    •  Analog Employees 
    on Aug 17, 2020 10:16 AM 6 months ago in reply to lvb

    Hi Liam,

    Do you have a dmesg log output with the error you are getting ?

  • Hi Mircea,

    Here is a snippet of my latest bootlog:

    ad9517 spi0.1: AD9517 successfully initialized
    ltc2632 spi4.0: ltc2634-l12 device registered, 4 channels
    axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_recalc_rate: Parent Rate 184320000 Hz
    axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_set_rate: Rate 8847360 Hz Parent Rate 184320000 Hz
    axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_recalc_rate: Parent Rate 184320000 Hz
    axi_adxcvr_drv b0070000.axi_adxcvr: AXI-ADXCVR-TX (17.01.a) using GTH4 at 0xB0070000 mapped to 0xffffff800ee30000. Number of lanes: 8.
    ad9144 spi0.0: Setting lane rate 7372800 kHz
    axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_round_rate: Rate 7372800 Hz Parent Rate 184320000 Hz
    axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_set_rate: Rate 7372800 Hz Parent Rate 184320000 Hz
    axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_recalc_rate: Parent Rate 184320000 Hz
    axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_enable: TX
    cf_axi_dds b0098000.ad_ip_jesd204_tpl_dac: Device Tree Probing 'ad_ip_jesd204_tpl_dac'
    cf_axi_dds b0098000.ad_ip_jesd204_tpl_dac: failed to get converter
    ad9144 spi0.0: DAC-0 calibration successful
    ad9144 spi0.0: DAC-1 calibration successful
    ad9144 spi0.0: Probed.

    I forgot to mention but I'm working with the kernel based on 2018.2 (6bfe5ed3d71e1f4e0164817e0dcb0d67da8c3a6e).

    I've enabled debug in the ad9144, cf_axi_dds_drv and the axi_adxcvr_drv.

    I've also added debug in the `cf_axi_dds_drv` and it seems like it fails during probe when getting the converter.

    6bfe5ed3d7:drivers/iio/frequency/cf_axi_dds.c#L1391

    master:drivers/iio/frequency/cf_axi_dds.c#L2047

    looking at the log, it seems like we try to call spi_get_drvdata() in cf_axi_dds.c before it's set in ad9144.c 

    Thanks,

    Liam

  • Mircea,

    Also, We're debugging using modules because the kernel seems to hang when accessing some AXI IP cores. Is that something you've seen before?

    The setup I have right now also supports an ad9371-FMC-EBZ. when all the drivers are built-in, the kernel hangs on the first AXI read on an axi-clkgen (this also seems to happen with other IP cores including the axi-quad-qspi).

    In this configuration, we're able to read and write directly to the registers from U-Boot and Linux, and disabling nodes in the devicetree allows us to boot into the rootfs.

    Loading everything as modules seems to workaround the hang issue but seems to bring a whole lot of other problems.

    would you have any suggestions on how to debug this boot hanging issue?

    Thanks again for your time,

    Liam

  • 0
    •  Analog Employees 
    on Aug 18, 2020 6:57 AM 6 months ago in reply to lvb

    Hi Liam,

    I saw that you said that you are using 2018.2 kernel. What HDL version are you using ? Is it the same 2018.2 ?

    You have to match both versions to be the same release otherwise you could have issues as the ones that you described.

    What device tree are you using ?

    Maybe the answer to these questions will point us in the right direction.

    Regards,

    Mircea

  • Hi Mircea,

    Yes, we are using the 2018.2 kernel along with the 2018.2 HDL and devicetree.

    This same setup was initially tested on a ZCU102 (all the same software/HDL versions) and we never encountered the boot hanging issue.

    Thanks,

    Liam

Reply Children
  • 0
    •  Analog Employees 
    on Aug 19, 2020 8:57 AM 6 months ago in reply to lvb

    With which version was this working before?

    Also, We're debugging using modules because the kernel seems to hang when accessing some AXI IP cores. Is that something you've seen before?

    It's not unheard of that in some setups the probe order can influence behavior.

    Usually for the IP cores to be able to read regs, the axi-clk needs to be enabled. This can be done in the driver, or the core can have this enabled by default.

    Maybe check that the clock is enabled (clk_prepare_enable() called) before reading anything from the IP core.

    One example is:

    https://github.com/analogdevicesinc/linux/commit/d78a31ef3583d58f8af9a6a385bc6308f897496c#diff-4024e54ca1569d5a29c5abc04ffbb1a2

    Someone else had some issue with a kernel hang (with our spi-axi-engine), because in some setups, the clock isn't enabled early in the probe.

    No idea if this also fixes this for you, but maybe do a check for this.

  • Hi ,

    With which version was this working before?

    Sorry for the confusion, I was initially testing on a ZCU102 with the default configuration (with all drivers builtin). With this setup, we managed to get everything working and were able to test using `jesd_status` and other custom test scripts.

    We then switched to a custom board (which has been tested extensively on other setups) and we noticed the kernel seems to hang when the axi-clkgen driver attempts its first AXI read. I disabled most PL nodes in the devicetree and got the board to boot into rootfs.

    As an attempt to debug the setup, we decided to build the drivers as modules and have them loaded automatically at boot. This effectively works around the boot hanging issue but lead us to a different set of issues, the order in which the drivers are loaded sometimes prevent the DAC from being initialized properly.

    To fix this I'm testing a patch of the cf_axi_dds where we check if the converter is setup earlier in the probe and return -EPROBE_DEFER if not.

    NOTE: all this was tested on our version of the kernel (based on adi's 2018.2 kernel) and vivado 2018.2.

    Maybe check that the clock is enabled (clk_prepare_enable() called) before reading anything from the IP core.

    Thanks for pointing that out, I'll give that a try as soon as possible.

    As another attempt to fix this, I was going to try to change the initcall priority of the driver setting up the axi-clk, would that make sense?

    Thanks again for your suggestions, I'll post the results as soon as possible,

    Liam 

  • Hi ,

    Maybe check that the clock is enabled (clk_prepare_enable() called) before reading anything from the IP core.

    Thanks for this suggestion, it seems to have fixed my issue! I've submitted a draft pull request to fix this in the axi-clkgen driver.

    Simply adding a reference to the axi_clk in the axi_quad_spi node fixed the issue with that driver too.

    Thanks again for your help,

    Liam

  • 0
    •  Analog Employees 
    on Aug 21, 2020 7:49 AM 6 months ago in reply to lvb

    Your PR is the correct fix.
    We will discuss on GIthub more.