Point cloud or depth map export to Matlab

It should be possible to export both the depth map and the IR data to MATLAB by saving the frames in an array, which can be afterwards used for further processing.

The depth map which comes from the camera is already compensated to take into account the camera's intrinsic values, so it can be used directly to display the point cloud. 

Thank you for your answer! I saw, that the depth map image can be exported in RGB, YCbCr or Grayscale format. The number representation is uint8, so for Grayscale I get just 256 values (0-255) in a 2D array. Isn't that a loss of precision?

And what would be the correct scale value, to convert 0-255 to the correct distance? Is this scale value different for near/medium/far mode?

In the provided aditof_sdk-2.0.0/bindings/matlab I find a textfile "colormap.txt", should this be used to convert a RGB depth map in to depth map with distance values? Again the scale value is unclear.

The depth and IR samples from the ADDI9036 are 12 bit and they are converted to 16 bit in the SDK.

There will be a loss of precision when converting to certain pixel formats, but if the image is used just for display it might not matter. In MATLAB it's also possible to use 16 bit images: https://www.mathworks.com/help/matlab/creating_plots/working-with-8-bit-and-16-bit-images.html

When converting to distance it's best to map the RGB pixel back to the corresponding depth value received from the SDK and don't try to compute the distance from the RGB pixel value. It might be useful to take a look at how aditof-demo displays the center point distance on top of the depth colormap: https://github.com/analogdevicesinc/aditof_sdk/blob/master/examples/aditof-demo/aditofdemoview.cpp#L911

The "colormap.txt" is used to convert the depth map into a RGB image. This maps the 16 bit depth values to RGB colors. The scaling of the depth map to RGB happens here: https://github.com/analogdevicesinc/aditof_sdk/blob/master/bindings/matlab/source_adaptor.cpp#L578

Thanks again for the answer!
1. i cannot see a option to save the depthMap in uint16 with the matlab function "getsnapshot()". a later conversion from uint8 to uint16 is no increase in precision in my opinion. but for now that is not the big issue.

2. https://github.com/analogdevicesinc/aditof_sdk/blob/master/examples/aditof-demo/aditofdemoview.cpp#L911

+

3. https://github.com/analogdevicesinc/aditof_sdk/blob/master/bindings/matlab/source_adaptor.cpp#L578

Unfortunately I still just use Matlab and for me its not clear how to adapt that for my needs, to get a correct point Cloud with correct axis scaling. My implementations of L911+912 or L578-580 didn't solved my problem.

I have now two Matlab-scripts for the depthMap convertion. I can recognize in both the object. But it is distorted.

%% Convertion depth map into point cloud - version A
close all; clear all; clc

% Range: <6m
% FoV: 90° x 69.2°
% Near: 25cm – 80cm
% Medium: 30cm – 4.5m
% Far: 3m – 6m
% Resolution: 640 x 480 pixels

s = load('depthImageBoxGrayscale_MatFile.mat');
p = s.p1; % uint8
depthMap = double(p)+1; % uint8 to double

figure();
imshow(p);

FOVhDeg = 90;
FOVhRad = deg2rad(FOVhDeg);
n_c = 640;

FOVvDeg = 69.2;
FOVvRad = deg2rad(FOVvDeg);
n_r = 480;

xyzPointsFromDepthMap = zeros(n_c*n_r,3);
i=1;
for yPixel = 1:480
    for xPixel = 1:640     
        r_i = round(((i-1) / n_c),0);
        c_i = round(mod((i-1),n_c),0);
        
        % Normalize depth
        maxDepth = 800; % ???
        d_i = depthMap(yPixel, xPixel)/maxDepth;
        
        % Calculate x-coordinate
        alphahRad = (pi() - FOVhRad)/2;
        gamma_h = alphahRad + (c_i/n_c)*FOVhRad;
        xyzPointsFromDepthMap(i,1) = d_i/tan(gamma_h); % x value
        
        % Calculate y-coordinate
        alphavRad = 2*pi() - FOVvRad/2;
        gamma_v = alphavRad + (r_i/n_r)*FOVvRad;
        xyzPointsFromDepthMap(i,2) = -d_i*tan(gamma_v); % y value

        % Calculate z-coordinate
        xyzPointsFromDepthMap(i,3) = d_i; % z value
        i = i+1;
    end
end


ptCloud = pointCloud(xyzPointsFromDepthMap);

%% Plot point cloud
figure
pcshow(ptCloud);
title('ToF point cloud from depth map')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

figure
pcshow(ptCloud);
colormap('gray');
title('ToF point cloud from depth map')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

%% Convertion depth map into point cloud - version B
close all; clear all; clc

% Range: <6m
% FoV: 90° x 69.2°
% Near: 25cm – 80cm
% Medium: 30cm – 4.5m
% Far: 3m – 6m
% Resolution: 640 x 480 pixels

s = load('depthImageBoxGrayscale_MatFile.mat');
p = s.p1; % uint8
depthMap = double(p)+1; % uint8 to double

figure();
imshow(p);

% FOVhDeg = 90;
% FOVhRad = deg2rad(FOVhDeg);
n_c = 640;

% FOVvDeg = 69.2;
% FOVvRad = deg2rad(FOVvDeg);
n_r = 480;

xyzPointsFromDepthMap = zeros(n_c*n_r,3);
i=1;
for yPixel = 1:480
    for xPixel = 1:640     
        % Calculate x-coordinate
        xyzPointsFromDepthMap(i,1) = xPixel; % x value
        
        % Calculate y-coordinate
        xyzPointsFromDepthMap(i,2) = yPixel; % y value

        % Calculate z-coordinate
        xyzPointsFromDepthMap(i,3) = depthMap(yPixel,xPixel); % z value
        i = i+1;
    end
end

ptCloud = pointCloud(xyzPointsFromDepthMap);

%% Plot point cloud
figure
pcshow(ptCloud);
title('ToF point cloud from depth map')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

figure
pcshow(ptCloud);
colormap('gray');
title('ToF point cloud from depth map')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

The measured object is the analog devices box from development kit. It should be a flat surface. As I can already see intensity differences in the grayscale depth map image i would assume that the intensity reflects the distance from the middlepoint of the camera to the object and not the xy-plane. Therefore i guess the Matlab-script version A should be the way to go. But I have problems to get further. Appreciate very much your help.

Thanks again for the answer!
1. i cannot see a option to save the depthMap in uint16 with the matlab function "getsnapshot()". a later conversion from uint8 to uint16 is no increase in precision in my opinion. but for now that is not the big issue.

2. https://github.com/analogdevicesinc/aditof_sdk/blob/master/examples/aditof-demo/aditofdemoview.cpp#L911

+

3. https://github.com/analogdevicesinc/aditof_sdk/blob/master/bindings/matlab/source_adaptor.cpp#L578

Unfortunately I still just use Matlab and for me its not clear how to adapt that for my needs, to get a correct point Cloud with correct axis scaling. My implementations of L911+912 or L578-580 didn't solved my problem.

I have now two Matlab-scripts for the depthMap convertion. I can recognize in both the object. But it is distorted.

%% Convertion depth map into point cloud - version A
close all; clear all; clc

% Range: <6m
% FoV: 90° x 69.2°
% Near: 25cm – 80cm
% Medium: 30cm – 4.5m
% Far: 3m – 6m
% Resolution: 640 x 480 pixels

s = load('depthImageBoxGrayscale_MatFile.mat');
p = s.p1; % uint8
depthMap = double(p)+1; % uint8 to double

figure();
imshow(p);

FOVhDeg = 90;
FOVhRad = deg2rad(FOVhDeg);
n_c = 640;

FOVvDeg = 69.2;
FOVvRad = deg2rad(FOVvDeg);
n_r = 480;

xyzPointsFromDepthMap = zeros(n_c*n_r,3);
i=1;
for yPixel = 1:480
    for xPixel = 1:640     
        r_i = round(((i-1) / n_c),0);
        c_i = round(mod((i-1),n_c),0);
        
        % Normalize depth
        maxDepth = 800; % ???
        d_i = depthMap(yPixel, xPixel)/maxDepth;
        
        % Calculate x-coordinate
        alphahRad = (pi() - FOVhRad)/2;
        gamma_h = alphahRad + (c_i/n_c)*FOVhRad;
        xyzPointsFromDepthMap(i,1) = d_i/tan(gamma_h); % x value
        
        % Calculate y-coordinate
        alphavRad = 2*pi() - FOVvRad/2;
        gamma_v = alphavRad + (r_i/n_r)*FOVvRad;
        xyzPointsFromDepthMap(i,2) = -d_i*tan(gamma_v); % y value

        % Calculate z-coordinate
        xyzPointsFromDepthMap(i,3) = d_i; % z value
        i = i+1;
    end
end


ptCloud = pointCloud(xyzPointsFromDepthMap);

%% Plot point cloud
figure
pcshow(ptCloud);
title('ToF point cloud from depth map')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

figure
pcshow(ptCloud);
colormap('gray');
title('ToF point cloud from depth map')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

%% Convertion depth map into point cloud - version B
close all; clear all; clc

% Range: <6m
% FoV: 90° x 69.2°
% Near: 25cm – 80cm
% Medium: 30cm – 4.5m
% Far: 3m – 6m
% Resolution: 640 x 480 pixels

s = load('depthImageBoxGrayscale_MatFile.mat');
p = s.p1; % uint8
depthMap = double(p)+1; % uint8 to double

figure();
imshow(p);

% FOVhDeg = 90;
% FOVhRad = deg2rad(FOVhDeg);
n_c = 640;

% FOVvDeg = 69.2;
% FOVvRad = deg2rad(FOVvDeg);
n_r = 480;

xyzPointsFromDepthMap = zeros(n_c*n_r,3);
i=1;
for yPixel = 1:480
    for xPixel = 1:640     
        % Calculate x-coordinate
        xyzPointsFromDepthMap(i,1) = xPixel; % x value
        
        % Calculate y-coordinate
        xyzPointsFromDepthMap(i,2) = yPixel; % y value

        % Calculate z-coordinate
        xyzPointsFromDepthMap(i,3) = depthMap(yPixel,xPixel); % z value
        i = i+1;
    end
end

ptCloud = pointCloud(xyzPointsFromDepthMap);

%% Plot point cloud
figure
pcshow(ptCloud);
title('ToF point cloud from depth map')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

figure
pcshow(ptCloud);
colormap('gray');
title('ToF point cloud from depth map')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

The measured object is the analog devices box from development kit. It should be a flat surface. As I can already see intensity differences in the grayscale depth map image i would assume that the intensity reflects the distance from the middlepoint of the camera to the object and not the xy-plane. Therefore i guess the Matlab-script version A should be the way to go. But I have problems to get further. Appreciate very much your help.

Version B of the script should be good if you are using the MATALB adaptor from SDK 2.0. Depth correction happens here: 

- https://github.com/analogdevicesinc/aditof_sdk/blob/master/sdk/src/cameras/ad-96tof1-ebz/camera_96tof1.cpp#L448

- https://github.com/analogdevicesinc/aditof_sdk/blob/master/sdk/src/cameras/ad-96tof1-ebz/camera_96tof1.cpp#L452 

We are trying to reproduce the issue and will provide feedback in the next couple of days. 

Thank you for the help! Yes, I am using SDK 2.0.

When I look at L448 and L452, I can already see there is a depth and geometry correction function... I will try to understand and implement that.

But if you could reproduce the issue and provide the needed Matlab code, that would be really great!

Hello,

Using the second matlab script (version B) we were able to obtain the following results without any modifications:

that looks promising! how did you made the depth map image? also with the matlab example "aditof-imaq.m"? and how did you exactly exported it (RGB, Grayscale, YPbBr, as Mat-File, Image-File,..)? Thank you very much!
I have used aditof-imaq.m and exported depthMap as Grayscale in Mat-File format.

Using aditof-imaq.m and export depthMap as RGB (it is the default mode) in MAT-File.

That wonders me. I wrote the conversion for Grayscale depthMaps. It handels MxNx1 arrays. The depth is read out in line 37 as the grayscale value directly. But as you exported it as RGB (MxNx3), how can you use this Matlab-Code without any modifications, because I expect then just the first color (red) will be interpreted as depth?

May you please repeat my measurement (near-mode) of a box in ~40cm distance and present your RGB and Grayscale depth map and converted point cloud? Because I see circles (RGB and Grayscale), when I measure a flat surface. As I understand, this means the distance is from the middlepoint of the camera and a conversion is therefore necessary...

Hello, we will update the script and come back with an answer as soon as possible.

Perfect, thank you very much!

Hello, to begin with, please checkout at the latest master.

After you run aditof_imaq.m please export the data as MONO16 and select RawDepth format.
After that you ca run the following script and the data should look alright.

%% Convertion depth map into point cloud - version B
close all; clear all; clc

% Range: <6m
% FoV: 90° x 69.2°
% Near: 25cm – 80cm
% Medium: 30cm – 4.5m
% Far: 3m – 6m
% Resolution: 640 x 480 pixels

s = load('depthMat.mat');
p = s.depth; % uint8
depthMap = double(p)+1; % uint8 to double

figure();
imshow(p);

% FOVhDeg = 90;
% FOVhRad = deg2rad(FOVhDeg);
n_c = 640;

% FOVvDeg = 69.2;
% FOVvRad = deg2rad(FOVvDeg);
n_r = 480;

xyzPointsFromDepthMap = zeros(n_c*n_r,3);
i=1;
for yPixel = 1:480
    for xPixel = 1:640     
        % Calculate x-coordinate
        xyzPointsFromDepthMap(i,1) = xPixel; % x value
        
        % Calculate y-coordinate
        xyzPointsFromDepthMap(i,2) = yPixel; % y value

        % Calculate z-coordinate
        xyzPointsFromDepthMap(i,3) = depthMap(yPixel,xPixel) / 10; % z value
        i = i+1;
    end
end

ptCloud = pointCloud(xyzPointsFromDepthMap);

%% Plot point cloud
figure
pcshow(ptCloud);
title('ToF point cloud from depth map')
set(gca, 'Zdir', 'reverse')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

figure
pcshow(ptCloud);
colormap('gray');
title('ToF point cloud from depth map')
set(gca, 'Zdir', 'reverse')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

Best regards,
Septimiu.