Calibrate two different cameras jointly

I have two cameras, located in a different position. One is a RGB camera. The other one is a thermal camera. I have their pose, respect to each others. I have taken several pictures of a chessboard from both cameras, and I have computed the intrinsic and extrinsic parameters of both cameras.

Now, using this data, I would like to calibrate the two cameras in order to be able to overlay each image taken from camera1 (RGB) with each image taken with camera2 (thermal), using the same setup clearly. In other words, I would like to find the correspondence pixel to pixel of each cameras, in order to warp images of one of the two camera in order to match the other one.

How can I do that using Matlab? Any hint is appreciated.

Thanks.

[Andrea Alberti]

I realized I need to use a depth sensor also, because it is the only way to have the disparity map for the warping stage. In other words, I will use a RGB-D camera (e.g. Kinect), rather than a RGB camera. But the first step is still the same: how can I calculate using Matlab the relative transform between the two cameras, in order to use such information to warp the thermal camera to the RGB camera?