Computational Photography

Aim

After completing the course, the student should be able to construct and use systems for computational photography, such as ordinary digital cameras and other systems where calculations are integrated with image generation. This gives the following four objectives:

• The student should be able to explain how the image is generated geometrically and then measured electronically with an ordinary light-measuring camera.
• The student should be able to explain how camera calibration works and apply this to the image stitching/panorama application.
• The student should be able to explain the basic structure and areas of use for different types of specialized cameras, such as IR cameras, range cameras, event cameras, high dynamic range (HDR) cameras, and cameras with coded aperture.
• The student should be able to apply the theory from objectives 1, 2, and 3 in several laboratory exercises and implement parts of the code in a high-level language such as Python and/or Matlab.

Content

The course describes the function of modern digital cameras (for still and video images). We analyze the process that from a luminous flux into the camera forms pixel values that can be combined and corrected through integrated calculations and finally read out in the form of an image. The collective name for this process is computational photography, and this includes optics, geometry, electronics, and the calculations that improve the result. Special types of cameras such as infrared cameras and distance cameras are also covered in the course. The course consists of 3 parts corresponding to objectives 1, 2, and 3.

1) How an image is generated geometrically and then measured electronically with a standard light-measuring camera.

• The digital camera: sensor structure, bayer patterns, noise sources, lens effects, etc.
• 3D visualization: simulation of the interaction of the light source with surfaces and volumes

2) How camera calibration works and an application: image stitching/panorama.

• Camera geometry: the pinhole camera, projective geometry, homogeneous coordinates, homography
• Camera calibration and an application: image stitching/panorama

3) Specialized cameras.

• Infrared and multi-spectral cameras
• Range cameras and point clouds
• Specialized cameras with high dynamic range (HDR), coded aperture, event cameras, etc.