Post Go back to editing

ADSD3100 / I.MX8MP uboot requirements

Category: Software
Product Number: eval-adsd3100-nxz
Software Version: 5.10.72

We are currently working on the development of a product centered around the adsd3100 sensor module and have been using the eval-adsd3100-nxz evaluation kit for this purpose. Our team has successfully ported the embedded Linux builds from your SDK to Yocto, allowing us to build our own software platform. However, we have encountered issues with image corruption when moving to other versions of u-boot. We are concerned about this as we plan to use a different SOM/baseboard in the future and would like to address any potential issues beforehand.

We have been using version 3.0.0 of the SDK as it has been working well for us during our tests. We have also examined the TOF git repository and noticed that the adsd3100 drivers have not been updated since the 3.0.0 tag. Our team has moved the kernel drivers for the adsd3100 and other patches from your SDK to a Yocto build based on a SolidRun layer for the same iMX8 SOM used in the evaluation kit. The kernel version we are using (5.10.72) is the same as the one used in the SDK, and we are confident that we could move to another kernel version without issues.

Our issue arises when using a newer version of u-boot (2021.04) with the patches from your SDK. While the camera is detected without problem and the commands/responses with the camera remain the same as when running on the working version of u-boot (2020.04), we experience image corruption. We have conducted an in-depth investigation of the drivers and have not found any obvious issues. To capture the images, we have used the data_collect example from the SDK with the default settings (mode 10: 1Mpixel 1024x1024).

Upon analyzing the data, we have found that whilst every four scanlines of each of the nine subframes should be 8192 bytes long, they typically contain only 6726 bytes of data, with the remainder being zeroes. As each value is two bytes, in pixel values this means that we get about 3363 values instead of the expected 4096 values from four 1024-pixel scanlines. When decoding the raw data, the offset for each pixel relative to the start of each of the 9 subframes is normally calculated as y*1024 + x. However, to generate a fixed version of the image, we use the following formula:

(y / 4 * 4096) + ((((y % 4) * 1024) + (x - 2 * (y % 4))) * 3363) / 4096
Our fixed version of the image looks approximately correct, but it is not perfect.

A possible issue that we have discounted is bit-expansion. A four-scanline block of 1024-pixel lines is 8192 bytes at 16-bit-per-pixel. If it were 11-bit-per-pixel, which we understand the camera provides, then this would be 5632 bytes. We see 6726 bytes. If it were 12-bit it would be 6144 bytes, 13-bit would be 6656 bytes, 14-bit would be 7168 bytes. However in all these cases, it would not be possible to simply duplicate 16-bit values every Nth pixel to get valid data (the bit ordering would cycle around through the scanline, and look totally corrupted). Therefore whatever subsystem is responsible for zero-padding from 11(?) bit to 16 bit is being performed correctly, but apparently pixels are being dropped.

We have placed the images and raw files at the end of this query. The zip contains the raw bin file output (raw_frame_20230411121437_00000.bin), and the ccb.dat file. Below are our converted image which shows the corruption (raw_frame_20230411121437_00000.bin ir.png), the fixed version of the image (image raw_frame_20230411121437_00000.bin ir-fixed.png).

We would greatly appreciate any insights you can provide regarding the issue we are encountering.

Image as obtained, converted to PNG:

Image as obtained, converted to PNG

Image "fixed up" using the formula above, duplicating some pixels:

Image "fixed up", duplicating some pixels

Raw data and CCB:

raw_and_ccb.zip

  •    are you using using tofi_comoute_depth.exe after collecting the raw data? And how are you collecting the raw data?

  • We have written our own image decoding / depth-compute pipeline (it works perfectly well when using the existing version of uboot). To capture raw images we have also written our own pipeline. However exactly the same issue occurs when using the standard data_collect command from github.com/.../data_collect

  • Hi dear customer,

    Thank you for the detailed description of your problem.

    We first started development using kernel 5.4 and at that time the recommended u-boot for it was 2020.4 built from NXP branch imx_v2020.04_5.4.70_2.3.0.

    Soon we upgraded to 5.10.72 because we had a lot of problems with USB 3.0 and also CSI capture. But u-boot imx_v2020.04 was working perfectly fine with 5.10.72 and because we had a set of patches applied over imx_v2020.04_5.4.70_2.3.0 we did not upgrade the u-boot. A few weeks ago, we even tested kernel 6.1 and even that was working perfectly fine with u-boot v2020.04.

    Now the patches we apply on u-boot are related to support for the SOM we use, DDR size detection because we have different DRAM sizes on our SOMs, Ethernet and USB PD.

    The CSI RX subsystem is not touched by u-boot so only the Linux kernel space are configuring it. I don't think the problem is there.

    The only thing that can be problematic from my option is the power delivery stuff.

    We had a quite similar problem in our tests with ADSD3100 + ADSD3500. Exactly as in your case the frame got corrupted and it was filled with null data after a number of correct lines. We concluded that it was a power supply problem and one of the voltage supply rails was dropping too low.

    Can you please check that USB PD is still working with the updated u-boot and you don't use the 5V 900mA default value?

    Also can you please test just by changing the u-boot without touching the rootfs? 5.10 should boot fine with both versions of u-boot.

    Regards,

    Bogdan