Point cloud or depth map export to Matlab

It should be possible to export both the depth map and the IR data to MATLAB by saving the frames in an array, which can be afterwards used for further processing.

The depth map which comes from the camera is already compensated to take into account the camera's intrinsic values, so it can be used directly to display the point cloud. 

Thank you for your answer! I saw, that the depth map image can be exported in RGB, YCbCr or Grayscale format. The number representation is uint8, so for Grayscale I get just 256 values (0-255) in a 2D array. Isn't that a loss of precision?

And what would be the correct scale value, to convert 0-255 to the correct distance? Is this scale value different for near/medium/far mode?

In the provided aditof_sdk-2.0.0/bindings/matlab I find a textfile "colormap.txt", should this be used to convert a RGB depth map in to depth map with distance values? Again the scale value is unclear.

Version B of the script should be good if you are using the MATALB adaptor from SDK 2.0. Depth correction happens here: 

- https://github.com/analogdevicesinc/aditof_sdk/blob/master/sdk/src/cameras/ad-96tof1-ebz/camera_96tof1.cpp#L448

- https://github.com/analogdevicesinc/aditof_sdk/blob/master/sdk/src/cameras/ad-96tof1-ebz/camera_96tof1.cpp#L452 

We are trying to reproduce the issue and will provide feedback in the next couple of days. 

Thank you for the help! Yes, I am using SDK 2.0.

When I look at L448 and L452, I can already see there is a depth and geometry correction function... I will try to understand and implement that.

But if you could reproduce the issue and provide the needed Matlab code, that would be really great!

Hello,

Using the second matlab script (version B) we were able to obtain the following results without any modifications:

that looks promising! how did you made the depth map image? also with the matlab example "aditof-imaq.m"? and how did you exactly exported it (RGB, Grayscale, YPbBr, as Mat-File, Image-File,..)? Thank you very much!
I have used aditof-imaq.m and exported depthMap as Grayscale in Mat-File format.

Using aditof-imaq.m and export depthMap as RGB (it is the default mode) in MAT-File.

That wonders me. I wrote the conversion for Grayscale depthMaps. It handels MxNx1 arrays. The depth is read out in line 37 as the grayscale value directly. But as you exported it as RGB (MxNx3), how can you use this Matlab-Code without any modifications, because I expect then just the first color (red) will be interpreted as depth?

May you please repeat my measurement (near-mode) of a box in ~40cm distance and present your RGB and Grayscale depth map and converted point cloud? Because I see circles (RGB and Grayscale), when I measure a flat surface. As I understand, this means the distance is from the middlepoint of the camera and a conversion is therefore necessary...

Hello, we will update the script and come back with an answer as soon as possible.

Perfect, thank you very much!

Hello, to begin with, please checkout at the latest master.

After you run aditof_imaq.m please export the data as MONO16 and select RawDepth format.
After that you ca run the following script and the data should look alright.

%% Convertion depth map into point cloud - version B
close all; clear all; clc

% Range: <6m
% FoV: 90° x 69.2°
% Near: 25cm – 80cm
% Medium: 30cm – 4.5m
% Far: 3m – 6m
% Resolution: 640 x 480 pixels

s = load('depthMat.mat');
p = s.depth; % uint8
depthMap = double(p)+1; % uint8 to double

figure();
imshow(p);

% FOVhDeg = 90;
% FOVhRad = deg2rad(FOVhDeg);
n_c = 640;

% FOVvDeg = 69.2;
% FOVvRad = deg2rad(FOVvDeg);
n_r = 480;

xyzPointsFromDepthMap = zeros(n_c*n_r,3);
i=1;
for yPixel = 1:480
    for xPixel = 1:640     
        % Calculate x-coordinate
        xyzPointsFromDepthMap(i,1) = xPixel; % x value
        
        % Calculate y-coordinate
        xyzPointsFromDepthMap(i,2) = yPixel; % y value

        % Calculate z-coordinate
        xyzPointsFromDepthMap(i,3) = depthMap(yPixel,xPixel) / 10; % z value
        i = i+1;
    end
end

ptCloud = pointCloud(xyzPointsFromDepthMap);

%% Plot point cloud
figure
pcshow(ptCloud);
title('ToF point cloud from depth map')
set(gca, 'Zdir', 'reverse')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

figure
pcshow(ptCloud);
colormap('gray');
title('ToF point cloud from depth map')
set(gca, 'Zdir', 'reverse')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

Best regards,
Septimiu.

Hello, to begin with, please checkout at the latest master.

After you run aditof_imaq.m please export the data as MONO16 and select RawDepth format.
After that you ca run the following script and the data should look alright.

%% Convertion depth map into point cloud - version B
close all; clear all; clc

% Range: <6m
% FoV: 90° x 69.2°
% Near: 25cm – 80cm
% Medium: 30cm – 4.5m
% Far: 3m – 6m
% Resolution: 640 x 480 pixels

s = load('depthMat.mat');
p = s.depth; % uint8
depthMap = double(p)+1; % uint8 to double

figure();
imshow(p);

% FOVhDeg = 90;
% FOVhRad = deg2rad(FOVhDeg);
n_c = 640;

% FOVvDeg = 69.2;
% FOVvRad = deg2rad(FOVvDeg);
n_r = 480;

xyzPointsFromDepthMap = zeros(n_c*n_r,3);
i=1;
for yPixel = 1:480
    for xPixel = 1:640     
        % Calculate x-coordinate
        xyzPointsFromDepthMap(i,1) = xPixel; % x value
        
        % Calculate y-coordinate
        xyzPointsFromDepthMap(i,2) = yPixel; % y value

        % Calculate z-coordinate
        xyzPointsFromDepthMap(i,3) = depthMap(yPixel,xPixel) / 10; % z value
        i = i+1;
    end
end

ptCloud = pointCloud(xyzPointsFromDepthMap);

%% Plot point cloud
figure
pcshow(ptCloud);
title('ToF point cloud from depth map')
set(gca, 'Zdir', 'reverse')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

figure
pcshow(ptCloud);
colormap('gray');
title('ToF point cloud from depth map')
set(gca, 'Zdir', 'reverse')
xlabel('X')
ylabel('Y')
zlabel('Z')
colorbar

Best regards,
Septimiu.

• I used from the very beginning the aditof_imaq.m from Release ADI Time of Flight SDK - Release v2.0.0 · analogdevicesinc/aditof_sdk · GitHub
• My Matlab-Version is R2020b Update 5 (9.9.0.1592791) 64-bit
• Image Acquisition Toolbox version: 6.3
• Analog Devices, Inc. Time of Flight Toolbox version: 20.1.1
  
  
  

After I run aditof_imaq.m, the image acquisition toolbox opens automatically. There I see no option to export it as MONO16, neither RawDepth format

The changes you made in the script (1. divide depth by 10, and 2. flip the z-axis), I unfortunately do not understand why that should solve my distortion problem..

I am very thankful for your help! But now I have invested so many hours in that.. and still many open, unanswered questions (like: is the depth shown in the depthMap the distance to the center-point of the camera or to the xy-plane?).

Hello, please use the latest master from https://github.com/analogdevicesinc/aditof_sdk and not the v2.0.0 since there were updates made since that release one of which is the update for matlab.

The distance division was required so that the space would't be altered and the shown distance would be in cm. The inversion of axis was required so that the points at a higher distance would be represented in the background plane. 

First of all, thank you very much for still supporting this feed!
In the github repository it says that the v2.0.0 is the latest release. As you recommend, I used now the version 2.0.1 (pre-release).
But as soon as I use the aditof.dll and the aditofadapter.dll version 2.0.1 (downloaded as zip from: Release ADI Time of Flight SDK - Release v2.0.1 · analogdevicesinc/aditof_sdk · GitHub) my Matlab crashes ("Access violation detected).
When I change the dlls back to version 2.0.0 it runs again, but still with round circles representing a flat wall...

Hello, sorry for the delay, we have updated the files from:
https://github.com/analogdevicesinc/aditof_sdk/releases/tag/v2.0.1

Please download them again.

Best regards,
Septimiu.

Hi Septimiu

The new DLL files help. The image is still not metric scaled after my conversion, but it is undistorted :-)!
Now I would need to have a metric scaled pointCloud (not depth values between 0-255 and also x,y axis should be metric. How can I achieve that in Matlab?

Unfortunately, I couldn't copy an image in this answer (tried different browsers, copy-paste and also the insert tool)

Best regards
Marco