Short Answer:
You can avoid data loss in images of type ‘float’. You most likely can’t do it with images of type ‘uint8’ or ‘uint16’ and the like. To convert your image to float types, have a look at im2single and im2double. To convert it back, have a look at im2uint8, im2uint16, etc.
Long answer:
rgb2ycbcr and ycbcr2rgb are essentially linear combinations of the different channels in each representation. Say x1, x2, x3 are the three channels of your RGB image and y1, y2, y3 are the three channels of the equivalent YCbCr image then:
y1 = a0 + a1*x1 + a2*x2 + a3*x3
y2 = b0 + b1*x1 + b2*x2 + b3*x3
y3 = c0 + c1*x1 + c2*x2 + c3*x3
where the ai-s bi-s and ci-s are all some pre-defined decimal (float) constants. There exist similar constants for conversion from YCbCr to RGB too. If your input image is of type ‘int8’, your yi-s will be restricted to be of the same type. This introduces inaccuracies because the linear combination of input channels with decimal (float) coefficients will also be decimal which you are trying to store in an integral container. Internally, these decimal values get rounded off to the nearest integer. That should explain why most of the differences/errors you notice are either 0/1.
More inaccuracies add up when you do the reverse operation by calling ycbcr2rgb. None of this would occur if the input image is of type float because they can hold other intermediate float values. If you tried the same comparison which you’ve mentioned here, your difference would be of the order of 1e-16 which can be attributed to the numerical accuracy of that datatype.
Foot note:
