Camera#

Camera class #

class Camera(sensor_height, sensor_width, intrinsic=None, distortion=None, extrinsic=None)[source]#

A camera is a structure with :

a sensor size (height and width).
an Extrinsic transformation.
an Distortion transformation.
an Intrinsic transformation.

See also

Package pycvcam (Artezaru/pycvcam) to learn more about the camera model.

As described in pycvcam, the height of the image is along the y-axis and the width of the image is along the x-axis. As described in the figure below, the package pycvcam uses the following notation:

world_points: The 3-D points \(\vec{X}_w\) with shape (…,3) expressed in the world coordinate system \((\vec{E}_x, \vec{E}_y, \vec{E}_z)\).
normalized_points: The 2-D points \(\vec{x}_n\) with shape (…,2) expressed in the normalized camera coordinate system \((\vec{I}, \vec{J})\) with a unit distance along the optical axis \((\vec{K})\).
distorted_points: The distorted 2-D points \(\vec{x}_d\) with shape (…,2) expressed in the normalized camera coordinate system \((\vec{I}, \vec{J})\) with a unit distance along the optical axis \((\vec{K})\).
image_points: The 2-D points \(\vec{x}_i\) with shape (…,2) expressed in the image coordinate system \((\vec{e}_x, \vec{e}_y)\) in the sensor plane.
pixel_points: The 2-D points \(\vec{x}_p\) with shape (…,2) expressed in the pixel coordinate system \((u, v)\) in the matrix of pixels.

../_images/definition_pycvcam.png — Definition of quantities in `pycvcam`.#

To convert the image_points to the pixel_points, a simple switch of coordinate system can be performed. You can use the methods pixel_points_to_image_points and image_points_to_pixel_points to perform these conversions.

Parameters:

sensor_height (Integral) – The height of the camera sensor in pixels.
sensor_width (Integral) – The width of the camera sensor in pixels.
intrinsic (Optional[pycvcam.core.Intrinsic]) – The intrinsic transformation of the camera. Default is None meaning no intrinsic transformation (identity transformation).
distortion (Optional[pycvcam.core.Distortion]) – The distortion transformation of the camera. Default is None meaning no distortion transformation (identity transformation).
extrinsic (Optional[pycvcam.core.Extrinsic]) – The extrinsic transformation of the camera. Default is None meaning no extrinsic transformation (identity transformation).

Instantiate a Camera object #

To instantiate a Camera object, you need to provide the sensor dimensions (height and width in pixels), the intrinsic parameters (as a pycvcam.Cv2Intrinsic object), the distortion parameters (as a pycvcam.Cv2Distortion object) and the extrinsic parameters (as a pycvcam.Cv2Extrinsic object).

Lets consider a simple OpenCV pinhole camera model without distortion. The intrinsic parameters can be defined from the camera matrix, and the extrinsic parameters from the rotation and translation vectors.

from pycvcam import Cv2Extrinsic, Cv2Intrinsic, Cv2Distortion
from pysdic import Camera
import numpy

rotation_vector = numpy.array([0.1, 0.2, 0.3])
translation_vector = numpy.array([12.0, 34.0, 56.0])

extrinsic = Cv2Extrinsic.from_rt(rotation_vector, translation_vector)

intrinsic = Cv2Intrinsic.from_matrix(
    numpy.array([[1000, 0, 320],
                [0, 1000, 240],
                [0, 0, 1]])
)

camera = Camera(
    sensor_height=480,
    sensor_width=640,
    intrinsic=intrinsic,
    distortion=None,
    extrinsic=extrinsic,
)

To load the extrinsic, intrinsic and distortion parameters from .json files, you can use the methods write_transform and read_transform from the package pycvcam (Artezaru/pycvcam).

Accessing Camera attributes #

You can access the camera attributes such as sensor dimensions, intrinsic, distortion and extrinsic parameters.

`Camera.sensor_height`	[Get or set] The height of the camera sensor in pixels (\(y\)-axis).
`Camera.sensor_width`	[Get or set] The width of the camera sensor in pixels (\(x\)-axis).
`Camera.distortion`	[Get or set] The distortion transformation of the camera.
`Camera.extrinsic`	[Get or set] The extrinsic transformation of the camera.
`Camera.intrinsic`	[Get or set] The intrinsic transformation of the camera.
`Camera.internal_bypass`	Get and set the internal bypass mode status.

Several cameras can have the same transformations (Intrinsic, Distortion, Extrinsic). Any modification of these transformations will be reflected in all the cameras sharing them.

Warning

If you use the internal_bypass = True feature to avoid checking the consistency of the parameters, the updates methods below must be called manually after any modification of the parameters on a transformations to propagate the changes to the camera.

`Camera.update`()	Called when the camera parameters have changed.
`Camera.intrinsic_update`()	Called when the intrinsic transformation has changed.
`Camera.distortion_update`()	Called when the distortion transformation has changed.
`Camera.extrinsic_update`()	Called when the extrinsic transformation has changed.
`Camera.size_update`()	Called when the sensor size has changed.

Manipulating Camera objects #

The camera can be used to project 3D points into the 2D image plane and to compute the rays emmited by the camera for each pixel.

`Camera.image_points_to_pixel_points`(image_points)	Convert image points to pixel points.
`Camera.get_camera_normalized_points`([mask])	Get the `normalized_points` of the camera by computing the inverse intrinsic and distortion transformations on the pixel points.
`Camera.get_camera_pixel_points`([mask])	Get the `pixel_points` of the camera for each pixel.
`Camera.get_camera_rays`([mask])	Get the camera rays emitted from the camera sensor to scene in the world coordinate system for each pixel.
`Camera.pixel_points_to_image_points`(pixel_points)	Convert pixel points to image points.
`Camera.project`(world_points[, dx, ...])	Project 3D world points to 2D images points using the camera's intrinsic, extrinsic, and distortion parameters.
`Camera.project_points`(world_points[, dx, ...])	Project 3D world points to 2D images points using the camera's intrinsic, extrinsic, and distortion parameters from an `PointCloud` instance.

Visualize 2D Projections #

The Camera class provides a method to visualize the 2D projection of 3D points onto the image plane.

`Camera.visualize_projected_point_cloud`(...)	Visualize the projected 2D points of a `PointCloud` on a 2D plot using matplotlib.
`Camera.visualize_projected_mesh`(mesh[, ...])	Visualize the projected 2D mesh of a `Mesh` on a 2D plot using matplotlib.

Usage #

Creating a camera with only intrinsic and extrinsic transformations:

import numpy
from pysdic import Camera
from pycvcam import Cv2Extrinsic, Cv2Intrinsic

rotation_vector = numpy.array([0.1, 0.2, 0.3])
translation_vector = numpy.array([12.0, 34.0, 56.0])

extrinsic = Cv2Extrinsic.from_rt(rotation_vector, translation_vector)

intrinsic = Cv2Intrinsic.from_matrix(
    numpy.array([[1000, 0, 320],
                [0, 1000, 240],
                [0, 0, 1]])
)

camera = Camera(
    sensor_height=480,
    sensor_width=640,
    intrinsic=intrinsic,
    extrinsic=extrinsic,
)

To project 3D points into the 2D image plane, you can use the project() method.

points_3d = numpy.array([[0, 0, 0],
                        [1, 1, 1],
                        [2, 2, 2]]) # world_points with shape (N, 3)

result = camera.project(points_3d)
image_points = result.image_points

Warning

image_points are expressed in the (x,y) coordinate system and not as pixel_points in the (u,v) coordinate system.

You can also access the jacobian of the projection (dx, dintrinsic, ddistortion, dextrinsic).

result = camera.project(points_3d, dintrinsic=True) # Compute also the jacobian of the projection with respect to the intrinsic parameters

image_points = result.image_points
jacobian = result.jacobian_dintrinsic

For reverse transformation, you can construct the rays emmited by the camera for each pixel by using the get_camera_rays() method. A mask with shape (height, width) can be used to filter the rays. To construct the rays for any 2D image points, you can use the package pycvcam (Artezaru/pycvcam) and the method compute_rays.

rays = camera.get_camera_rays(mask=mask)
origins = rays[0][:, :3]  # shape (N, 3)
directions = rays[1][:, 3:]  # shape (N, 3)

Camera#

Camera class#

Instantiate a Camera object#

Accessing Camera attributes#

Manipulating Camera objects#

Visualize 2D Projections#

Usage#

This Page