Geometric Image Transformations

The functions in this section perform various geometrical transformations of 2D images. They do not change the image content but deform the pixel grid and map this deformed grid to the destination image. In fact, to avoid sampling artifacts, the mapping is done in the reverse order, from destination to the source. That is, for each pixel (x, y) of the destination image, the functions compute coordinates of the corresponding "donor" pixel in the source image and copy the pixel value:

\texttt{dst} (x,y)= \texttt{src} (f_x(x,y), f_y(x,y))

In case when you specify the forward mapping \left<g_x, g_y\right>: \texttt{src} \rightarrow \texttt{dst} , the OpenCV functions first compute the corresponding inverse mapping \left<f_x, f_y\right>: \texttt{dst} \rightarrow \texttt{src} and then use the above formula.

The actual implementations of the geometrical transformations, from the most generic :ocv:func:`remap` and to the simplest and the fastest :ocv:func:`resize` , need to solve two main problems with the above formula:

• Extrapolation of non-existing pixels. Similarly to the filtering functions described in the previous section, for some (x,y) , either one of f_x(x,y) , or f_y(x,y) , or both of them may fall outside of the image. In this case, an extrapolation method needs to be used. OpenCV provides the same selection of extrapolation methods as in the filtering functions. In addition, it provides the method BORDER_TRANSPARENT . This means that the corresponding pixels in the destination image will not be modified at all.
• Interpolation of pixel values. Usually f_x(x,y) and f_y(x,y) are floating-point numbers. This means that \left<f_x, f_y\right> can be either an affine or perspective transformation, or radial lens distortion correction, and so on. So, a pixel value at fractional coordinates needs to be retrieved. In the simplest case, the coordinates can be just rounded to the nearest integer coordinates and the corresponding pixel can be used. This is called a nearest-neighbor interpolation. However, a better result can be achieved by using more sophisticated interpolation methods , where a polynomial function is fit into some neighborhood of the computed pixel (f_x(x,y), f_y(x,y)) , and then the value of the polynomial at (f_x(x,y), f_y(x,y)) is taken as the interpolated pixel value. In OpenCV, you can choose between several interpolation methods. See :ocv:func:`resize` for details.

convertMaps

Converts image transformation maps from one representation to another.

The function converts a pair of maps for :ocv:func:`remap` from one representation to another. The following options ( (map1.type(), map2.type()) \rightarrow (dstmap1.type(), dstmap2.type()) ) are supported:

• \texttt{(CV\_32FC1, CV\_32FC1)} \rightarrow \texttt{(CV\_16SC2, CV\_16UC1)} . This is the most frequently used conversion operation, in which the original floating-point maps (see :ocv:func:`remap` ) are converted to a more compact and much faster fixed-point representation. The first output array contains the rounded coordinates and the second array (created only when nninterpolation=false ) contains indices in the interpolation tables.
• \texttt{(CV\_32FC2)} \rightarrow \texttt{(CV\_16SC2, CV\_16UC1)} . The same as above but the original maps are stored in one 2-channel matrix.
• Reverse conversion. Obviously, the reconstructed floating-point maps will not be exactly the same as the originals.

getAffineTransform

Calculates an affine transform from three pairs of the corresponding points.

The function calculates the 2 \times 3 matrix of an affine transform so that:

\begin{bmatrix} x'_i \\ y'_i \end{bmatrix} = \texttt{map\_matrix} \cdot \begin{bmatrix} x_i \\ y_i \\ 1 \end{bmatrix}

where

dst(i)=(x'_i,y'_i),
src(i)=(x_i, y_i),
i=0,1,2

getPerspectiveTransform

Calculates a perspective transform from four pairs of the corresponding points.

The function calculates the 3 \times 3 matrix of a perspective transform so that:

\begin{bmatrix} t_i x'_i \\ t_i y'_i \\ t_i \end{bmatrix} = \texttt{map\_matrix} \cdot \begin{bmatrix} x_i \\ y_i \\ 1 \end{bmatrix}

where

dst(i)=(x'_i,y'_i),
src(i)=(x_i, y_i),
i=0,1,2,3

getRectSubPix

Retrieves a pixel rectangle from an image with sub-pixel accuracy.

The function getRectSubPix extracts pixels from src :

dst(x, y) = src(x +  \texttt{center.x} - ( \texttt{dst.cols} -1)*0.5, y +  \texttt{center.y} - ( \texttt{dst.rows} -1)*0.5)

where the values of the pixels at non-integer coordinates are retrieved using bilinear interpolation. Every channel of multi-channel images is processed independently. While the center of the rectangle must be inside the image, parts of the rectangle may be outside. In this case, the replication border mode (see :ocv:func:`borderInterpolate` ) is used to extrapolate the pixel values outside of the image.

getRotationMatrix2D

Calculates an affine matrix of 2D rotation.

The function calculates the following matrix:

\begin{bmatrix} \alpha &  \beta & (1- \alpha )  \cdot \texttt{center.x} -  \beta \cdot \texttt{center.y} \\ - \beta &  \alpha &  \beta \cdot \texttt{center.x} + (1- \alpha )  \cdot \texttt{center.y} \end{bmatrix}

where

\begin{array}{l} \alpha =  \texttt{scale} \cdot \cos \texttt{angle} , \\ \beta =  \texttt{scale} \cdot \sin \texttt{angle} \end{array}

The transformation maps the rotation center to itself. If this is not the target, adjust the shift.

invertAffineTransform

Inverts an affine transformation.

The function computes an inverse affine transformation represented by 2 \times 3 matrix M :

\begin{bmatrix} a_{11} & a_{12} & b_1  \\ a_{21} & a_{22} & b_2 \end{bmatrix}

The result is also a 2 \times 3 matrix of the same type as M .

LogPolar

Remaps an image to log-polar space.

The function cvLogPolar transforms the source image using the following transformation:

• Forward transformation (CV_WARP_INVERSE_MAP is not set):

  dst( \phi , \rho ) = src(x,y)
  

• Inverse transformation (CV_WARP_INVERSE_MAP is set):

  dst(x,y) = src( \phi , \rho )
  

where

\rho = M  \cdot \log{\sqrt{x^2 + y^2}} , \phi =atan(y/x)

The function emulates the human "foveal" vision and can be used for fast scale and rotation-invariant template matching, for object tracking and so forth. The function can not operate in-place.

remap

Applies a generic geometrical transformation to an image.

The function remap transforms the source image using the specified map:

\texttt{dst} (x,y) =  \texttt{src} (map_x(x,y),map_y(x,y))

where values of pixels with non-integer coordinates are computed using one of available interpolation methods. map_x and map_y can be encoded as separate floating-point maps in map_1 and map_2 respectively, or interleaved floating-point maps of (x,y) in map_1 , or fixed-point maps created by using :ocv:func:`convertMaps` . The reason you might want to convert from floating to fixed-point representations of a map is that they can yield much faster (~2x) remapping operations. In the converted case, map_1 contains pairs (cvFloor(x), cvFloor(y)) and map_2 contains indices in a table of interpolation coefficients.

This function cannot operate in-place.

resize

Resizes an image.

The function resize resizes the image src down to or up to the specified size. Note that the initial dst type or size are not taken into account. Instead, the size and type are derived from the src,``dsize``,``fx`` , and fy . If you want to resize src so that it fits the pre-created dst , you may call the function as follows:

// explicitly specify dsize=dst.size(); fx and fy will be computed from that.
resize(src, dst, dst.size(), 0, 0, interpolation);

If you want to decimate the image by factor of 2 in each direction, you can call the function this way:

// specify fx and fy and let the function compute the destination image size.
resize(src, dst, Size(), 0.5, 0.5, interpolation);

To shrink an image, it will generally look best with CV_INTER_AREA interpolation, whereas to enlarge an image, it will generally look best with CV_INTER_CUBIC (slow) or CV_INTER_LINEAR (faster but still looks OK).

warpAffine

Applies an affine transformation to an image.

The function warpAffine transforms the source image using the specified matrix:

\texttt{dst} (x,y) =  \texttt{src} ( \texttt{M} _{11} x +  \texttt{M} _{12} y +  \texttt{M} _{13}, \texttt{M} _{21} x +  \texttt{M} _{22} y +  \texttt{M} _{23})

when the flag WARP_INVERSE_MAP is set. Otherwise, the transformation is first inverted with :ocv:func:`invertAffineTransform` and then put in the formula above instead of M . The function cannot operate in-place.

warpPerspective

Applies a perspective transformation to an image.

The function warpPerspective transforms the source image using the specified matrix:

\texttt{dst} (x,y) =  \texttt{src} \left ( \frac{M_{11} x + M_{12} y + M_{13}}{M_{31} x + M_{32} y + M_{33}} ,
     \frac{M_{21} x + M_{22} y + M_{23}}{M_{31} x + M_{32} y + M_{33}} \right )

when the flag WARP_INVERSE_MAP is set. Otherwise, the transformation is first inverted with :ocv:func:`invert` and then put in the formula above instead of M . The function cannot operate in-place.

initUndistortRectifyMap

Computes the undistortion and rectification transformation map.

The function computes the joint undistortion and rectification transformation and represents the result in the form of maps for :ocv:func:`remap` . The undistorted image looks like original, as if it is captured with a camera using the camera matrix =newCameraMatrix and zero distortion. In case of a monocular camera, newCameraMatrix is usually equal to cameraMatrix , or it can be computed by :ocv:func:`getOptimalNewCameraMatrix` for a better control over scaling. In case of a stereo camera, newCameraMatrix is normally set to P1 or P2 computed by :ocv:func:`stereoRectify` .

Also, this new camera is oriented differently in the coordinate space, according to R . That, for example, helps to align two heads of a stereo camera so that the epipolar lines on both images become horizontal and have the same y- coordinate (in case of a horizontally aligned stereo camera).

The function actually builds the maps for the inverse mapping algorithm that is used by :ocv:func:`remap` . That is, for each pixel (u, v) in the destination (corrected and rectified) image, the function computes the corresponding coordinates in the source image (that is, in the original image from camera). The following process is applied:

\begin{array}{l} x  \leftarrow (u - {c'}_x)/{f'}_x  \\ y  \leftarrow (v - {c'}_y)/{f'}_y  \\{[X\,Y\,W]} ^T  \leftarrow R^{-1}*[x \, y \, 1]^T  \\ x'  \leftarrow X/W  \\ y'  \leftarrow Y/W  \\ x"  \leftarrow x' (1 + k_1 r^2 + k_2 r^4 + k_3 r^6) + 2p_1 x' y' + p_2(r^2 + 2 x'^2)  \\ y"  \leftarrow y' (1 + k_1 r^2 + k_2 r^4 + k_3 r^6) + p_1 (r^2 + 2 y'^2) + 2 p_2 x' y'  \\ map_x(u,v)  \leftarrow x" f_x + c_x  \\ map_y(u,v)  \leftarrow y" f_y + c_y \end{array}

where (k_1, k_2, p_1, p_2[, k_3]) are the distortion coefficients.

In case of a stereo camera, this function is called twice: once for each camera head, after :ocv:func:`stereoRectify` , which in its turn is called after :ocv:func:`stereoCalibrate` . But if the stereo camera was not calibrated, it is still possible to compute the rectification transformations directly from the fundamental matrix using :ocv:func:`stereoRectifyUncalibrated` . For each camera, the function computes homography H as the rectification transformation in a pixel domain, not a rotation matrix R in 3D space. R can be computed from H as

\texttt{R} =  \texttt{cameraMatrix} ^{-1}  \cdot \texttt{H} \cdot \texttt{cameraMatrix}

where cameraMatrix can be chosen arbitrarily.

getDefaultNewCameraMatrix

Returns the default new camera matrix.

The function returns the camera matrix that is either an exact copy of the input cameraMatrix (when centerPrinicipalPoint=false ), or the modified one (when centerPrincipalPoint=true).

In the latter case, the new camera matrix will be:

\begin{bmatrix} f_x && 0 && ( \texttt{imgSize.width} -1)*0.5  \\ 0 && f_y && ( \texttt{imgSize.height} -1)*0.5  \\ 0 && 0 && 1 \end{bmatrix} ,

where f_x and f_y are (0,0) and (1,1) elements of cameraMatrix , respectively.

By default, the undistortion functions in OpenCV (see :ocv:func:`initUndistortRectifyMap`, :ocv:func:`undistort`) do not move the principal point. However, when you work with stereo, it is important to move the principal points in both views to the same y-coordinate (which is required by most of stereo correspondence algorithms), and may be to the same x-coordinate too. So, you can form the new camera matrix for each view where the principal points are located at the center.

undistort

Transforms an image to compensate for lens distortion.

The function transforms an image to compensate radial and tangential lens distortion.

The function is simply a combination of :ocv:func:`initUndistortRectifyMap` (with unity R ) and :ocv:func:`remap` (with bilinear interpolation). See the former function for details of the transformation being performed.

Those pixels in the destination image, for which there is no correspondent pixels in the source image, are filled with zeros (black color).

A particular subset of the source image that will be visible in the corrected image can be regulated by newCameraMatrix . You can use :ocv:func:`getOptimalNewCameraMatrix` to compute the appropriate newCameraMatrix depending on your requirements.

The camera matrix and the distortion parameters can be determined using :ocv:func:`calibrateCamera` . If the resolution of images is different from the resolution used at the calibration stage, f_x, f_y, c_x and c_y need to be scaled accordingly, while the distortion coefficients remain the same.

undistortPoints

Computes the ideal point coordinates from the observed point coordinates.

The function is similar to :ocv:func:`undistort` and :ocv:func:`initUndistortRectifyMap` but it operates on a sparse set of points instead of a raster image. Also the function performs a reverse transformation to :ocv:func:`projectPoints` . In case of a 3D object, it does not reconstruct its 3D coordinates, but for a planar object, it does, up to a translation vector, if the proper R is specified.

// (u,v) is the input point, (u', v') is the output point
// camera_matrix=[fx 0 cx; 0 fy cy; 0 0 1]
// P=[fx' 0 cx' tx; 0 fy' cy' ty; 0 0 1 tz]
x" = (u - cx)/fx
y" = (v - cy)/fy
(x',y') = undistort(x",y",dist_coeffs)
[X,Y,W]T = R*[x' y' 1]T
x = X/W, y = Y/W
// only performed if P=[fx' 0 cx' [tx]; 0 fy' cy' [ty]; 0 0 1 [tz]] is specified
u' = x*fx' + cx'
v' = y*fy' + cy',

where undistort() is an approximate iterative algorithm that estimates the normalized original point coordinates out of the normalized distorted point coordinates ("normalized" means that the coordinates do not depend on the camera matrix).

The function can be used for both a stereo camera head or a monocular camera (when R is empty).