How to find relative camera pose from homography matrix

Hello,

I have a sequence of images, and i need to know the relative pose of my camera in regard to the first frame. I have found the homography matrix and I have all the camera params. How can i get the relative pose using this information?

Thank you in advance.

[Matt J]

Does the image sequence come from a moving camera in different poses? Which two images does the homography relate? Why does it help to have only one homography matrix when you have a whole sequence of images?

[Nuno Marques]

Hi. Yes, it's from a moving drone. I calculate the Homography matrix in every frame using the current frame and the previous. In other words, im constantly re-calculating the homography matrix.

Answer 1

[Matt J]

Yes, it's from a moving drone.

This means that the camera center is moving. Two images from cameras with different centers cannot be related by a homography, unfortunately. You would have to calculate relative pose by doing single-view geometric calibration of each camera position. If there are 5 or more trackable points in the scene, you may be able to do this.

[Nuno Marques]

Do you have any idea how to calculate the relative camera pose then? The landing pad has some leds that I can use. I try to use SURF but the matches arent of good quality.

This is the landing pad, and that flare is from the lights that ate placed below the drone.

[Nuno Marques]

And how should i do single view geometric calibration? And when the drone has landed i can only see up to 3 leds. Is that a problem?

Here is how the it looks before the drone has taken flight. I can only see 4 leds.

[Matt J]

Do you know the 3D world coordinates of the LEDs?

[Nuno Marques]

No I dont.

[Matt J]

You will probably have to use lsqnonlin() to implement your own iterative calibration algorithm. You already know the camera intrinsics, so it is just the 6 extrinsic parameters that you will have to estimate, plus thte LED positions.

[Matt J]

There are also some published algorithms for estimating extrinsics plus 3D point feature locations, like this one,

https://ieeexplore.ieee.org/document/4774416

However, that paper is applied to a very different applications from yours.

[Nuno Marques]

I also know the extrinsic parameters. And im confused on how to use lsqnonlin(). I've seen the explanation inhttps://www.mathworks.com/help/optim/ug/lsqnonlin.html but i have no ideia how to use it.

Answer 2

[Matt J]

You might also have a look at the relativeCameraPose() command. According to its documentation,

https://www.mathworks.com/help/vision/ref/relativecamerapose.html

you have all the required inputs, however, I do not understand how a homography matrix could be enough to get it to work in light of the fact that the camera center is moving.

[Nuno Marques]

I've been trying to use relativeCameraPose. My problem is that i don't have a way to reliably match the points in the 2 frames. SURF and ORB dont really work that well

[Matt J]

Is the LED configuration fixed? Maybe you can write your own matching algorithm. It doesn't look like a very complicated LED geometry. You might also be able to use the different LED colors to distinguish them.

[Nuno Marques]

Yes. I've applied 2 masks. 1 green and 1 red and then combined them. After that i did a blob analisys. However, the blob analisys makes a search from the top left corner to the bottom right corner, and saves the blob coordinates in an array in the order that finds them. That means, when the attitute of the drone changes, the order of the blobs changes aswell. That makes poor matches beetween some frames. Also there's the problem when a new led enter's the scene.

[Matt J]

There are orientation-invariant features that can be used to distinguish them. For example, the separation distances between two of the orange LEDs is smaller than the 3rd in the outer square, and similarly in the inner square.