pycvcam.compute_rays#

compute_rays(image_points, intrinsic, distortion, extrinsic, *, transpose=False, inverse_intrinsic_kwargs=None, inverse_distortion_kwargs=None, ray_extrinsic_kwargs=None)[source]#

Compute the rays emitted from the camera to the scene based on the given image points, intrinsic parameters, distortion model, and extrinsic parameters.

The process to compute the rays is as follows:

  1. The image_points (\(\vec{x}_i\)) are normalized by the inverse intrinsic matrix transformation to obtain the distorted_points (\(\vec{x}_d\)).

  2. The distorted_points (\(\vec{x}_d\)) are undistorted by the distortion model using the coefficients \(\{\lambda_1, \lambda_2, \lambda_3, \ldots\}\) to obtain the normalized_points (\(\vec{x}_n\)).

  3. The normalized_points (\(\vec{x}_n\)) are used to compute the rays in the world coordinate system using the extrinsic parameters to obtain the rays.

The ray structure is as follows:

  • The first 3 elements are the origin of the ray in the world coordinate system.

  • The last 3 elements are the direction of the ray in the world coordinate system. The direction vector is normalized.

See also

Note

The expected image_points can be extracted from the pixel_points by swaping the axes.

Warning

The points are converting to float before applying the inverse transformation. See pycvcam.core.Package for more details on the default data types used in the package.

Parameters:

  • image_points (ArrayLike) – The 2D image points in the camera normalized coordinate system. Shape (…, 2)

  • intrinsic (Optional[Intrinsic]) – The intrinsic transformation to be applied to the image points. If None, a no intrinsic transformation is applied (i.e., identity transformation).

  • distortion (Optional[Distortion]) – The distortion model to be applied to the normalized points. If None, a no distortion transformation is applied (i.e., identity transformation).

  • extrinsic (Optional[Extrinsic]) – The extrinsic transformation (rotation and translation) to be applied to the normalized points. If None, a no extrinsic transformation is applied (i.e., identity transformation).

  • transpose (bool, optional) – If True, the input image points are transposed before processing, the input shape is expected to be (2, …) instead of (…, 2) and the output shape will be (6, …). Default is False.

  • inverse_intrinsic_kwargs (Optional[Dict], optional) – Additional keyword arguments to be passed to the intrinsic transformation’s inverse method (intrinsic._inverse_transform). Default is None.

  • inverse_distortion_kwargs (Optional[Dict], optional) – Additional keyword arguments to be passed to the distortion transformation’s inverse method (distortion._inverse_transform). Default is None.

  • ray_extrinsic_kwargs (Optional[Dict], optional) – Additional keyword arguments to be passed to the extrinsic transformation’s method to compute the rays (extrinsic._compute_rays). Default is None.

Returns:

The rays in the world coordinate system.

Return type:

Rays

Example

Create a simple example to construct the rays from an image to the scene:

import numpy
import cv2
from pycvcam import compute_rays, Cv2Extrinsic, Cv2Intrinsic, Cv2Distortion

# Read the image :
image = cv2.imread('image.jpg')
image_height, image_width = image.shape[:2]

# Construct the intrinsic transformation :
intrinsic = Cv2Intrinsic.from_matrix(numpy.array([[1000, 0, image_width / 2],
                                                  [0, 1000, image_height / 2],
                                                  [0, 0, 1]]))

# Construct the distortion transformation:
distortion = Cv2Distortion(parameters=numpy.array([0.1, -0.05, 0, 0, 0]))

# Construct the extrinsic transformation:
extrinsic = Cv2Extrinsic.from_rt(rvec=[0.1, 0.2, 0.3], tvec=[0, 0, 5])

# Define the image points (e.g., pixels in the image):
pixel_points = numpy.indices((image_height, image_width)) # shape (2, H, W)
pixel_points = pixel_points.reshape(2, -1).T  # shape (H*W, 2) WARNING: [H, W -> Y, X]
image_points = pixel_points[:, [1, 0]]  # Swap to [X, Y] format

# Throw rays from the image points to the scene:
rays = compute_rays(image_points, intrinsic, distortion, extrinsic, transpose=False).rays

# Here `rays` will contain the origin and direction of the rays in the world coordinate system with shape (..., 6).
# rays[i, :] = [origin_x, origin_y, origin_z, direction_x, direction_y, direction_z]
On this page

This Page