enabling ad9144 as a module

Hi Liam,

We usually test the drivers by statically linking in the kernel. As you say some issues might arise with the order that you load the drivers. What drivers are you trying as a module and what is your exact order ?

Regards,

Mircea

Hi Mircea,

At the moment, we're trying to boot with all drivers loaded as modules. Since we are using an AD9136-FMC-EBZ, this also includes the drivers for the ad9516-1.

here is a list of the module names:

- ad9517

- ad9144

- cf_axi_dds_drv

- axi_jesd204_tx

- axi_adxcvr_drv

- dma_axi_dmac

As for the loading order, I'm relying on the devicetree to load everything automatically.

Thanks,

Liam

Hi Liam,

Do you have a dmesg log output with the error you are getting ?

Hi Mircea,

Here is a snippet of my latest bootlog:

ad9517 spi0.1: AD9517 successfully initialized
ltc2632 spi4.0: ltc2634-l12 device registered, 4 channels
axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_recalc_rate: Parent Rate 184320000 Hz
axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_set_rate: Rate 8847360 Hz Parent Rate 184320000 Hz
axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_recalc_rate: Parent Rate 184320000 Hz
axi_adxcvr_drv b0070000.axi_adxcvr: AXI-ADXCVR-TX (17.01.a) using GTH4 at 0xB0070000 mapped to 0xffffff800ee30000. Number of lanes: 8.
ad9144 spi0.0: Setting lane rate 7372800 kHz
axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_round_rate: Rate 7372800 Hz Parent Rate 184320000 Hz
axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_set_rate: Rate 7372800 Hz Parent Rate 184320000 Hz
axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_recalc_rate: Parent Rate 184320000 Hz
axi_adxcvr_drv b0070000.axi_adxcvr: adxcvr_clk_enable: TX
cf_axi_dds b0098000.ad_ip_jesd204_tpl_dac: Device Tree Probing 'ad_ip_jesd204_tpl_dac'
cf_axi_dds b0098000.ad_ip_jesd204_tpl_dac: failed to get converter
ad9144 spi0.0: DAC-0 calibration successful
ad9144 spi0.0: DAC-1 calibration successful
ad9144 spi0.0: Probed.

I forgot to mention but I'm working with the kernel based on 2018.2 (6bfe5ed3d71e1f4e0164817e0dcb0d67da8c3a6e).

I've enabled debug in the ad9144, cf_axi_dds_drv and the axi_adxcvr_drv.

I've also added debug in the `cf_axi_dds_drv` and it seems like it fails during probe when getting the converter.

- 6bfe5ed3d7:drivers/iio/frequency/cf_axi_dds.c#L1391

- master:drivers/iio/frequency/cf_axi_dds.c#L2047

looking at the log, it seems like we try to call spi_get_drvdata() in cf_axi_dds.c before it's set in ad9144.c 

Thanks,

Liam

Mircea,

Also, We're debugging using modules because the kernel seems to hang when accessing some AXI IP cores. Is that something you've seen before?

The setup I have right now also supports an ad9371-FMC-EBZ. when all the drivers are built-in, the kernel hangs on the first AXI read on an axi-clkgen (this also seems to happen with other IP cores including the axi-quad-qspi).

In this configuration, we're able to read and write directly to the registers from U-Boot and Linux, and disabling nodes in the devicetree allows us to boot into the rootfs.

Loading everything as modules seems to workaround the hang issue but seems to bring a whole lot of other problems.

would you have any suggestions on how to debug this boot hanging issue?

Thanks again for your time,

Liam

Hi Liam,

I saw that you said that you are using 2018.2 kernel. What HDL version are you using ? Is it the same 2018.2 ?

You have to match both versions to be the same release otherwise you could have issues as the ones that you described.

What device tree are you using ?

Maybe the answer to these questions will point us in the right direction.

Regards,

Mircea

Hi Mircea,

Yes, we are using the 2018.2 kernel along with the 2018.2 HDL and devicetree.

This same setup was initially tested on a ZCU102 (all the same software/HDL versions) and we never encountered the boot hanging issue.

Thanks,

Liam

With which version was this working before?

lvb said:

Also, We're debugging using modules because the kernel seems to hang when accessing some AXI IP cores. Is that something you've seen before?

It's not unheard of that in some setups the probe order can influence behavior.

Usually for the IP cores to be able to read regs, the axi-clk needs to be enabled. This can be done in the driver, or the core can have this enabled by default.

Maybe check that the clock is enabled (clk_prepare_enable() called) before reading anything from the IP core.

One example is:

https://github.com/analogdevicesinc/linux/commit/d78a31ef3583d58f8af9a6a385bc6308f897496c#diff-4024e54ca1569d5a29c5abc04ffbb1a2

Someone else had some issue with a kernel hang (with our spi-axi-engine), because in some setups, the clock isn't enabled early in the probe.

No idea if this also fixes this for you, but maybe do a check for this.

Hi ,

Unknown said:

With which version was this working before?

Sorry for the confusion, I was initially testing on a ZCU102 with the default configuration (with all drivers builtin). With this setup, we managed to get everything working and were able to test using `jesd_status` and other custom test scripts.

We then switched to a custom board (which has been tested extensively on other setups) and we noticed the kernel seems to hang when the axi-clkgen driver attempts its first AXI read. I disabled most PL nodes in the devicetree and got the board to boot into rootfs.

As an attempt to debug the setup, we decided to build the drivers as modules and have them loaded automatically at boot. This effectively works around the boot hanging issue but lead us to a different set of issues, the order in which the drivers are loaded sometimes prevent the DAC from being initialized properly.

To fix this I'm testing a patch of the cf_axi_dds where we check if the converter is setup earlier in the probe and return -EPROBE_DEFER if not.

NOTE: all this was tested on our version of the kernel (based on adi's 2018.2 kernel) and vivado 2018.2.

Unknown said:

Maybe check that the clock is enabled (clk_prepare_enable() called) before reading anything from the IP core.

Thanks for pointing that out, I'll give that a try as soon as possible.

As another attempt to fix this, I was going to try to change the initcall priority of the driver setting up the axi-clk, would that make sense?

Thanks again for your suggestions, I'll post the results as soon as possible,

Liam 

Hi ,

Unknown said:

With which version was this working before?

Sorry for the confusion, I was initially testing on a ZCU102 with the default configuration (with all drivers builtin). With this setup, we managed to get everything working and were able to test using `jesd_status` and other custom test scripts.

We then switched to a custom board (which has been tested extensively on other setups) and we noticed the kernel seems to hang when the axi-clkgen driver attempts its first AXI read. I disabled most PL nodes in the devicetree and got the board to boot into rootfs.

As an attempt to debug the setup, we decided to build the drivers as modules and have them loaded automatically at boot. This effectively works around the boot hanging issue but lead us to a different set of issues, the order in which the drivers are loaded sometimes prevent the DAC from being initialized properly.

To fix this I'm testing a patch of the cf_axi_dds where we check if the converter is setup earlier in the probe and return -EPROBE_DEFER if not.

NOTE: all this was tested on our version of the kernel (based on adi's 2018.2 kernel) and vivado 2018.2.

Unknown said:

Maybe check that the clock is enabled (clk_prepare_enable() called) before reading anything from the IP core.

Thanks for pointing that out, I'll give that a try as soon as possible.

As another attempt to fix this, I was going to try to change the initcall priority of the driver setting up the axi-clk, would that make sense?

Thanks again for your suggestions, I'll post the results as soon as possible,

Liam