bundleAdjustment

Refine 3-D points and camera poses

Syntax

[xyzRefinedPoints,refinedPoses] = bundleAdjustment(xyzPoints,pointTracks,cameraPoses,intrinsics)

[___,reprojectionErrors] = bundleAdjustment(___)

[___] = bundleAdjustment(___,Name,Value)

Description

[xyzRefinedPoints,refinedPoses] = bundleAdjustment(xyzPoints,pointTracks,cameraPoses,intrinsics) refines 3-D points and camera poses to minimize reprojection errors. The refinement procedure is a variant of the Levenberg-Marquardt algorithm. The function uses the same global reference coordinate system to return both the 3-D points and camera poses.

[___,reprojectionErrors] = bundleAdjustment(___) returns the mean reprojection error for each 3-D world point, in addition to the arguments from the previous syntax.

[___] = bundleAdjustment(___,Name,Value) specifies options using one or more name-value arguments. Unspecified arguments have default values.

Examples

Refine 3-D points and camera poses From Single Camera

Load data for initialization.

data = load('sfmGlobe');

Refine the camera poses and points.

[xyzRefinedPoints,refinedPoses] = ...
    bundleAdjustment(data.xyzPoints,data.pointTracks,data.cameraPoses,data.intrinsics);

Display the refined 3-D points and camera poses.

pcshow(xyzRefinedPoints,'VerticalAxis','y','VerticalAxisDir', ...
    'down','MarkerSize',45)
hold on
plotCamera(refinedPoses,'Size',0.1)
hold off
grid on

Input Arguments

xyzPoints — Unrefined 3-D points

M-by-3 matrix

Unrefined 3-D points, specified as an M-by-3 matrix of [x y z] locations.

pointTracks — Matching points across multiple images

N-element array of pointTrack objects

Matching points across multiple images, specified as an N-element array of pointTrack objects. Each element contains two or more matching points across multiple images.

cameraPoses — Camera pose information

three-column table

Camera pose ViewId, Orientation, and Location information, specified as a three-column table. The view IDs relate to the IDs of the objects in the pointTracks argument. Specify orientations a 3-by-3 rotation matrices. Specify locations as three-element vectors

intrinsics — Camera intrinsics

cameraIntrinsics object | N-element array of cameraIntrinsics objects

Camera intrinsics, specified as a cameraIntrinsics object or an N-element array of cameraIntrinsics objects. N is the number of camera poses. Use a single cameraIntrinsics object when images are captured using the same camera. Use a vector cameraIntrinsics objects when images are captured by different cameras.

Name-Value Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Example: 'MaxIterations', 50

MaxIterations — Maximum number of iterations

50 (default) | positive integer

Maximum number of iterations before the Levenberg-Marquardt algorithm stops, specified as a positive integer.

AbsoluteTolerance — Absolute termination tolerance

1.0 (default) | positive scalar

Absolute termination tolerance of the mean squared reprojection error in pixels, specified as positive scalar.

RelativeTolerance — Relative termination tolerance

1e-5 (default) | positive scalar

Relative termination tolerance of the reduction in reprojection error between iterations, specified as positive scalar.

PointsUndistorted — Flag to indicate lens distortion

false (default) | true

Flag to indicate lens distortion, specified as false or true. When you set PointsUndistorted to false, the 2-D points in pointTracks must be from images with lens distortion. To use undistorted points, first use the undistortImage function to remove distortions from the images, then set PointsUndistorted.

FixedViewIDs — View IDs for fixed camera pose

[] (default) | vector of nonnegative integers

View IDs for fixed camera pose, specified as a vector of nonnegative integers. Each ID corresponds to the ViewId of a fixed camera pose in cameraPoses. An empty value for FixedViewIDs means that all camera poses are optimized.

Solver — Solver

'sparse-linear-algebra' (default) | 'preconditioned-conjugate-gradient'

Solver, specified as 'sparse-linear-algebra' or 'preconditioned-conjugate-gradient'. Use the 'sparse-linear-algebra' solver for low sparsity images. Low sparsity indicates that many camera views observe some of the same world points. Use the 'preconditioned-conjugate-gradient' (PCG) solver, from the general graphic optimization (g2o) library, for high sparsity images. High sparsity indicates that each camera view observes, only a small portion of the world points, specified by xyzPoints.

Verbose — Display progress information

false (default) | true

Display progress information, specified as false or true.

Output Arguments

xyzRefinedPoints — 3-D locations of refined world points

M-by-3 matrix

3-D locations of refined world points, returned as an M-by-3 matrix of [x y z] locations.

Data Types: single | double

refinedPoses — Refined camera poses

three-column table

Refined camera poses, returned as a three-column table. The table contains columns for ViewId, Orientation, and Location.

reprojectionErrors — Reprojection errors

M-element vector

Reprojection errors, returned as an M-element vector. The function projects each world point back into each camera. Then, in each image, the function calculates the reprojection error as the distance between the detected and the reprojected point. The reprojectionErrors vector contains the average reprojection error for each world point.

References

[1] Lourakis, Manolis I. A., and Antonis A. Argyros. "SBA: A Software Package for Generic Sparse Bundle Adjustment." ACM Transactions on Mathematical Software 36, no. 1 (March 2009): 2:1–2:30.

[2] Hartley, Richard, and Andrew Zisserman. Multiple View Geometry in Computer Vision. 2nd ed. Cambridge, UK ; New York: Cambridge University Press, 2003.

[3] Triggs, Bill, Philip F. McLauchlan, Richard I. Hartley, and Andrew W. Fitzgibbon. "Bundle Adjustment — A Modern Synthesis." In Proceedings of the International Workshop on Vision Algorithms, 298–372. Springer-Verlag, 1999.