How to get transform between two fixed frames using a camera

Hello everyone,

In my task, I have two manipulators: one 6 dof robot (Manipulator 2 as shown in attached link picture), and one passive arm (Manipulator 1 as shown in attached link picture). Both the frames 1 and 2 are fixed. I would like to transfer the movement that I have with respect to the frame 1 to the frame 2, for this I need to compute the transform between the two fixed frames. To this end I would like to use a camera for defining the frames, how can I get the frames axes using the camera?

Thank you in advance for any helpful insights!

Answer 1

[Akshat Dalal]

Hi Lara,

I understand you want to use MATLAB to define fixed frames using 2 cameras to compute transforms between them. This is posssible using MATLAB and to get the frames' axes and compute the transformation between two fixed frames using a camera, you can follow these steps:

1. Camera Calibration: Before you start, calibrate your camera to determine its intrinsic parameters (focal length, principal point, distortion coefficients, etc.). This will allow you to accurately map from 2D image coordinates to 3D world coordinates.
2. Marker Placement: Place 'ArUco' markers on known locations in both frames 1 and 2. These markers should be easily recognizable by the camera and have known dimensions. To read more about 'ArUco' markers, you could refer the following documentation: https://docs.opencv.org/4.x/d5/dae/tutorial_aruco_detection.html
3. Marker Detection: Capture images of the markers using the camera. Use computer vision techniques to detect the markers in the images. To read about how to achieve this using MATLAB, you could refer the following MATLAB Answer: https://www.mathworks.com/matlabcentral/answers/372403
4. Pose Estimation: Once the markers are detected, use the camera calibration data to estimate the pose (position and orientation) of each marker relative to the camera. This will give you the transformation matrices from the camera frame to each marker's frame.
5. Frame Definition: Define the fixed frames 1 and 2 based on the poses of the markers. If you have multiple markers for each frame, you may need to compute an average or otherwise combine the pose data to define the frame.

Hope this helps!