Query on plotCamera and world2img plots

Hi everyone, I am trying to create a simulated environment whereby i have a camera, with given extrinsics parameters, looking at a group of points. I am using plotCamera to visualise such a scene as seen in the 'Perspective View' plot. Additionally, with given intrinsics parameter, I would like to display the output of the pixel coordinates for the points observed from its extrinsics parameters (i.e pose from world coordinates). I have used world2img to obtain the image points to display the points shown in 'Camera View' plot. A sample of my code are given below.

So my question is:

Why is it that when my camera is facing upwards into the sky (default 0 rotation) as seen in the 'Perspective View' plot, my 'Camera View' plot shows a point oberseved? The points_in_sight (from world2img) indicated that 1 point is within boundary of the image. This should be physically impossible!

Do note that both functions uses the same rigidtform3d to represent the geometric pose of the camera.

Thank you in advance!

close all; clear all; clc;

% Points in World Coordinates
P = [0.855157970900440	9.63088539286913	0
    2.62482234698333	5.46805718738968	0
    8.01014622769739	5.21135830804002	0
    0.292202775621463	2.31594386708524	0
    9.28854139478045	4.88897743920167	0
    7.30330862855453	6.24060088173690	0
    4.88608973803579	6.79135540865748	0
    5.78525061023439	3.95515215668593	0
    2.37283579771522	3.67436648544477	0
    4.58848828179931	9.87982003161633	0];

%% Camera Extrinsics
% Translation
tx = 5;
ty = 5;
tz = 25;
t = [tx, ty, tz];
% Rotation
rx = 0;
ry = 0;
rz = 0;
Rx = [1 0 0; 0 cosd(rx) -sind(rx); 0 sind(rx) cosd(rx)];
Ry = [cosd(ry) 0 sind(ry); 0 1 0; -sind(ry) 0 cosd(ry)];
Rz = [cosd(rz) -sind(rz) 0; sind(rz) cosd(rz) 0; 0 0 1];
R = Rz*Ry*Rx;
pose = rigidtform3d(R, t);

%% Camera Intrinsics
focalLength    = [1600 1600];
principalPoint = [960 540];
imageSize      = [1080 1920];
intrinsics = cameraIntrinsics(focalLength, principalPoint, imageSize);

% World to image Coordinates
[imagePoints, validIndex] = world2img(P, pose, intrinsics);

% Number of points seen
points_in_sight = sum(validIndex)


%% Plots
% Plot of world view
cam = plotCamera('AbsolutePose', pose, 'Opacity', 0, 'AxesVisible', 1);
hold on
plot(P(:, 1), P(:, 2), 'ko', 'MarkerFaceColor', 'b', 'MarkerSize', 5)
xlim([-10, 20]); 
ylim([-10, 20]); 
zlim([0, 30]);
grid on; 
axis equal; axis manual; 
title('Perspective View')

% Plot of camera view
figure;
plot(imagePoints(:, 1), imagePoints(:, 2), 'ko', 'MarkerFaceColor', 'b')
xlim([0 1920]);
ylim([0 1080]);
grid on
title('Camera View')

Answer 1

[Ayush Gupta]

Hi @MarshMellow,

Looking at your code, I can see that you have set the camera's translation and rotation values such that in this configuration, the camera is positioned at (5, 5, 25) in world coordinates and has no rotation (rx = ry = rz = 0).

Since the camera is looking upwards into the sky, it should not be able to see any of the points in the scene. However, you are observing a point in the 'Camera View' plot.

One possible reason for this unexpected behavior could be an issue with the coordinate systems or units used. Make sure that the units of your camera's translation parameters (tx, ty, tz) and the world coordinates (P) are consistent.

Additionally, double-check the correctness of the `world2img` function and the `intrinsics` parameters you provided. Ensure that the focal length, principal point, and image size accurately represent the camera's properties.

By carefully reviewing these aspects and ensuring consistency in units and coordinate systems, you should be able to resolve the issue.

Thanks.