This is a very generic question and there are many ways to answer. It depends on what information you have about the two coordinate systems, or points in the two systems.
You can use planar Euler rotations, quaternions, rotation vectors, etc. to describe the orientation of one frame relative to another. In general, a "transformation matrix" is defined which can multiply a vector to convert it from one frame to the other. this matrix is also called a "direction cosine matrix" because it can be derived, by inspection, from using vector dot products (vector dot products of unit vectors represent the cosine of the angle between the vectors)
So, this is physically what the transformation matrix represents - a mapping of unit vectors from one frame to another.
in this format, it is easy to see that the inverse transformation (from B to A) is equal to the transpose of the A-to-B transformation matrix. But, in order to cunstruct the actual matrix, we need to know more about the two systems.