Post Go back to editing

Point cloud or depth map export to Matlab

I use AD-96TOF1-EBZ (with dragonboard)

Is it possible to get the point cloud exported to Matlab?

The "aditof_imaq.m" Matlab example and the built-in "image acquisition tool" work well. The depth map image and IR image stream is nice, but I miss data stream/export. In special the point cloud or depth map data.

Thanks for help :-)!

Parents
  • It should be possible to export both the depth map and the IR data to MATLAB by saving the frames in an array, which can be afterwards used for further processing.

    The depth map which comes from the camera is already compensated to take into account the camera's intrinsic values, so it can be used directly to display the point cloud. 

Reply
  • It should be possible to export both the depth map and the IR data to MATLAB by saving the frames in an array, which can be afterwards used for further processing.

    The depth map which comes from the camera is already compensated to take into account the camera's intrinsic values, so it can be used directly to display the point cloud. 

Children
  • Thank you for your answer! I saw, that the depth map image can be exported in RGB, YCbCr or Grayscale format. The number representation is uint8, so for Grayscale I get just 256 values (0-255) in a 2D array. Isn't that a loss of precision?

    And what would be the correct scale value, to convert 0-255 to the correct distance? Is this scale value different for near/medium/far mode?

    In the provided aditof_sdk-2.0.0/bindings/matlab I find a textfile "colormap.txt", should this be used to convert a RGB depth map in to depth map with distance values? Again the scale value is unclear.

  • The depth and IR samples from the ADDI9036 are 12 bit and they are converted to 16 bit in the SDK.

    There will be a loss of precision when converting to certain pixel formats, but if the image is used just for display it might not matter. In MATLAB it's also possible to use 16 bit images: https://www.mathworks.com/help/matlab/creating_plots/working-with-8-bit-and-16-bit-images.html

    When converting to distance it's best to map the RGB pixel back to the corresponding depth value received from the SDK and don't try to compute the distance from the RGB pixel value. It might be useful to take a look at how aditof-demo displays the center point distance on top of the depth colormap: https://github.com/analogdevicesinc/aditof_sdk/blob/master/examples/aditof-demo/aditofdemoview.cpp#L911

    The "colormap.txt" is used to convert the depth map into a RGB image. This maps the 16 bit depth values to RGB colors. The scaling of the depth map to RGB happens here: https://github.com/analogdevicesinc/aditof_sdk/blob/master/bindings/matlab/source_adaptor.cpp#L578

  • Thanks again for the answer!
    1. i cannot see a option to save the depthMap in uint16 with the matlab function "getsnapshot()". a later conversion from uint8 to uint16 is no increase in precision in my opinion. but for now that is not the big issue.

    2. https://github.com/analogdevicesinc/aditof_sdk/blob/master/examples/aditof-demo/aditofdemoview.cpp#L911

    +


    3. https://github.com/analogdevicesinc/aditof_sdk/blob/master/bindings/matlab/source_adaptor.cpp#L578

    Unfortunately I still just use Matlab and for me its not clear how to adapt that for my needs, to get a correct point Cloud with correct axis scaling. My implementations of L911+912 or L578-580 didn't solved my problem.

    I have now two Matlab-scripts for the depthMap convertion. I can recognize in both the object. But it is distorted.

    %% Convertion depth map into point cloud - version A
    close all; clear all; clc
    
    % Range: <6m
    % FoV: 90° x 69.2°
    % Near: 25cm – 80cm
    % Medium: 30cm – 4.5m
    % Far: 3m – 6m
    % Resolution: 640 x 480 pixels
    
    s = load('depthImageBoxGrayscale_MatFile.mat');
    p = s.p1; % uint8
    depthMap = double(p)+1; % uint8 to double
    
    figure();
    imshow(p);
    
    FOVhDeg = 90;
    FOVhRad = deg2rad(FOVhDeg);
    n_c = 640;
    
    FOVvDeg = 69.2;
    FOVvRad = deg2rad(FOVvDeg);
    n_r = 480;
    
    xyzPointsFromDepthMap = zeros(n_c*n_r,3);
    i=1;
    for yPixel = 1:480
        for xPixel = 1:640     
            r_i = round(((i-1) / n_c),0);
            c_i = round(mod((i-1),n_c),0);
            
            % Normalize depth
            maxDepth = 800; % ???
            d_i = depthMap(yPixel, xPixel)/maxDepth;
            
            % Calculate x-coordinate
            alphahRad = (pi() - FOVhRad)/2;
            gamma_h = alphahRad + (c_i/n_c)*FOVhRad;
            xyzPointsFromDepthMap(i,1) = d_i/tan(gamma_h); % x value
            
            % Calculate y-coordinate
            alphavRad = 2*pi() - FOVvRad/2;
            gamma_v = alphavRad + (r_i/n_r)*FOVvRad;
            xyzPointsFromDepthMap(i,2) = -d_i*tan(gamma_v); % y value
    
            % Calculate z-coordinate
            xyzPointsFromDepthMap(i,3) = d_i; % z value
            i = i+1;
        end
    end
    
    
    ptCloud = pointCloud(xyzPointsFromDepthMap);
    
    %% Plot point cloud
    figure
    pcshow(ptCloud);
    title('ToF point cloud from depth map')
    xlabel('X')
    ylabel('Y')
    zlabel('Z')
    colorbar
    
    figure
    pcshow(ptCloud);
    colormap('gray');
    title('ToF point cloud from depth map')
    xlabel('X')
    ylabel('Y')
    zlabel('Z')
    colorbar
    

    %% Convertion depth map into point cloud - version B
    close all; clear all; clc
    
    % Range: <6m
    % FoV: 90° x 69.2°
    % Near: 25cm – 80cm
    % Medium: 30cm – 4.5m
    % Far: 3m – 6m
    % Resolution: 640 x 480 pixels
    
    s = load('depthImageBoxGrayscale_MatFile.mat');
    p = s.p1; % uint8
    depthMap = double(p)+1; % uint8 to double
    
    figure();
    imshow(p);
    
    % FOVhDeg = 90;
    % FOVhRad = deg2rad(FOVhDeg);
    n_c = 640;
    
    % FOVvDeg = 69.2;
    % FOVvRad = deg2rad(FOVvDeg);
    n_r = 480;
    
    xyzPointsFromDepthMap = zeros(n_c*n_r,3);
    i=1;
    for yPixel = 1:480
        for xPixel = 1:640     
            % Calculate x-coordinate
            xyzPointsFromDepthMap(i,1) = xPixel; % x value
            
            % Calculate y-coordinate
            xyzPointsFromDepthMap(i,2) = yPixel; % y value
    
            % Calculate z-coordinate
            xyzPointsFromDepthMap(i,3) = depthMap(yPixel,xPixel); % z value
            i = i+1;
        end
    end
    
    ptCloud = pointCloud(xyzPointsFromDepthMap);
    
    %% Plot point cloud
    figure
    pcshow(ptCloud);
    title('ToF point cloud from depth map')
    xlabel('X')
    ylabel('Y')
    zlabel('Z')
    colorbar
    
    figure
    pcshow(ptCloud);
    colormap('gray');
    title('ToF point cloud from depth map')
    xlabel('X')
    ylabel('Y')
    zlabel('Z')
    colorbar

    The measured object is the analog devices box from development kit. It should be a flat surface. As I can already see intensity differences in the grayscale depth map image i would assume that the intensity reflects the distance from the middlepoint of the camera to the object and not the xy-plane. Therefore i guess the Matlab-script version A should be the way to go. But I have problems to get further. Appreciate very much your help.

  • Version B of the script should be good if you are using the MATALB adaptor from SDK 2.0. Depth correction happens here: 

    https://github.com/analogdevicesinc/aditof_sdk/blob/master/sdk/src/cameras/ad-96tof1-ebz/camera_96tof1.cpp#L448

    https://github.com/analogdevicesinc/aditof_sdk/blob/master/sdk/src/cameras/ad-96tof1-ebz/camera_96tof1.cpp#L452 

    We are trying to reproduce the issue and will provide feedback in the next couple of days. 

  • Thank you for the help! Yes, I am using SDK 2.0.

    When I look at L448 and L452, I can already see there is a depth and geometry correction function... I will try to understand and implement that.

    But if you could reproduce the issue and provide the needed Matlab code, that would be really great!

  • Hello,

    Using the second matlab script (version B) we were able to obtain the following results without any modifications:

  • that looks promising! how did you made the depth map image? also with the matlab example "aditof-imaq.m"? and how did you exactly exported it (RGB, Grayscale, YPbBr, as Mat-File, Image-File,..)? Thank you very much!
    I have used aditof-imaq.m and exported depthMap as Grayscale in Mat-File format.

  • Using aditof-imaq.m and export depthMap as RGB (it is the default mode) in MAT-File.

  • That wonders me. I wrote the conversion for Grayscale depthMaps. It handels MxNx1 arrays. The depth is read out in line 37 as the grayscale value directly. But as you exported it as RGB (MxNx3), how can you use this Matlab-Code without any modifications, because I expect then just the first color (red) will be interpreted as depth?

  • May you please repeat my measurement (near-mode) of a box in ~40cm distance and present your RGB and Grayscale depth map and converted point cloud? Because I see circles (RGB and Grayscale), when I measure a flat surface. As I understand, this means the distance is from the middlepoint of the camera and a conversion is therefore necessary...