Questions¶
What is exposure stacking?
Computational Photography: Epsilon to Coded Photography¶
Computational photography includes use of “Smart Lights” to escape the limitations of traditional camera.
Epsilon photography is concerned with synthesizing omnipictures and proceeds by multiple capture single image paradigm (MCSI).
The scene properties are encoded in few photographs.
Fourier transform Fourier optics Optical heterodyning.
Light Field
The light field is a function that describes the amount of light traveling in every direction through every point in space.
plenoptic function.
The plenoptic illumination function is an idealized function used in computer vision and computer graphics to express the image of a scene from any possible viewing position at any viewing angle at any point in time.
Since rays in space can be parametrized by three spatial coordinates, x, y and z and two angles \(\theta\) and \(\sigma\) it is a five dimensional function.
Show the lighting and effect¶
Capturing the visually meaningful changes from the light leaving the scene.
Single Object Change.
Richer Visual Experience.
Field of View.
Close Depth of Field for all objects.
Records the moment.
Interactive Display.
Fraction of the Light field of the scene.
Fusion of time-lapsed events.
Not a fluttering shutter.
Image Stabilizing.
Computational illumination¶
What sorts of spatio-temporal modulations for light might better reveal the visually important contents of a scene?
https://en.wikipedia.org/wiki/Harold_Eugene_Edgerton
Spatio-temporal modulations for light might better reveal the visually important
spatio-temporal modulators and optics
https://en.wikipedia.org/wiki/Helmholtz_reciprocity
changing color filters in front of the camera, using tunable wavelength filters or using diffraction gratings
better lighting control during cap- ture allows one to build richer representations of photographed scenes.
Camera Parameter - Instant of Capture - Change.
Each setting allows recording of partial information about the scene and the final image is reconstructed from these multiple observations.
multiple film-style photos computationally merged for a scene description.
The merged photo contains the best features from all of them.
Novel illumuniation
Capturing a rubik’s cube solution, with changes of instance of capture. We merge the photos to create rich time-lapse events of the solution. To add some more color, the time-lapse is presented with focus on two toys near the Rubiks cube.
Digital Image Formats¶
GIF, JPG, PPM, TIF, BMP
Raw Format.
OpenCV / Python
opencv.org, python.org
Matlab / Octave
Processing.org
Exercises: Read and write images.
Image compression information.
Metadata about pictures (Exchangeable Image file format - EXIF “etc”).
Digital Image - Processing and Filtering¶
Point-process computations.
Add/Subtract Images.
alpha-blending and its applications.
Image Histograms.
I(x, y)
Point-process: Pixel / Point Arithmetic¶
0-255
0.34 * CD + 0.34 * AE + 0.34 * LD =
Transparency (Conversely, Opacity).
Usually represented as \(\alpha\)
\(\alpha\) varies from 0 to 1. (0 = invisible, 1 = fully visible).
RGB \(\righarrow\) \(\alpha\) RGB
Point process computations on Images to Add and Subtract images.
Showed an example of \(\alpha\) blending commonly used in Image processing.
Image Histogram in Image processing.
Image Processing, Filtering via Convolution and Correlation¶
A Real life example of point arithmetic
Variety of Blending modes built on concept of point processes.
average = fblend(a, b) = (a + b) / 2
normal = fblend(a, b) = b
Arithmetic Blend modes¶
Divide (brightens photos)
Addition (too many whites)
Subtract (too many blacks)
Difference (subtract with scaling)
Darken fblend(a, b) = min(a, b) for RGB
Lighten: blend(a, b) = max(a, b) for RGB
Advanced Modes¶
- Multiply
Darker fblend(a, b) = ab
- Screen
brighter fblend(a, b) = 1 - (1 - a)(1 - b)
Summary¶
Introduced Pixel / Layer Blending
Explained variety of blending approaches
Showed why some of the videos look ODD
Lesson Objectives¶
Smooth an image over a neighborhood of pixels.
Media filtering as a special non-linear filtering and smoothing approach.
How to Smooth a Signal?
A mathematical representation for smoothing¶
Cross-correlation or x-correlation.
Box Filter (Averaging) for Smoothing¶
Box / Average Filter - 21 x 21
Special Case: Median Filtering¶
Median filtering is a non-linear operation often used in image processing.
Reduce noise, but
Preserves Edges (Sharp Lines!)
Main Idea: Use median of all pixels in the kernel area, instead of mean.
Average filter (11 x 11) Median filter (11 x 11)
Image smoothing
Applying a kernel to smooth an image.
Averaging and median filtering.
Next Class¶
Cross Correlation and Convolution.
Cross-Correlation.
Convolution
Difference between Cross-Correlation and Convolution.
Properties of these methods.
In signal processing, cross-correlation is a measure of similarity of two waveforms as a function of a time-lag applied to one of them.
Sliding dot product or sliding inner-product of two different matrices.
Cross-Correlation Method¶
G[i, j] = :math:sum_{u=-k}^{k}sum_{v=-k}^k h[u, v] F[i + u, j + v]
Denoted by
G = h :math:bigotimes F
Filtering an image.
Replace each pixel with a linear combination of its neighbors.
Filter “kernel” or “mask”
h[u, v] is a prescription for weights in the linear combination.
Example: Box Filter¶
Box/ Average Filter
Size 21 x 21
Values: Uniform
Example: Gaussian Filter¶
Gaussian Filter
Size: 21 x 21
Values: Gaussian or Normal Distribution.
Using Gaussian Filters for Smoothing¶
Convolution Method¶
Convolution is a mathematical operation on two functions F and h
Linear Filters¶
Original 64 x 64
Cross-Correlation
Convolution
Differences between Cross-correlation and convolution.
Image Analysis and Edge Detection¶
Image Variance
Convolution and Cross Correlation.
Using Filters to Find Features
Map raw pixels to an intermediate representation.
Reduce amount of data, preserve useful information.
Good features to match between images¶
Features
Water bottle
Parts of an image that encode it in a compact form.
Edges
What kind of discontinuities are in a scene?
Surface Normal
Depth
Surface Color
Illumination.
Images as Functions F(x, y)¶
Edges appear as ridges in the 3d height map of an image.
Edge Detection¶
Basic Idea
Look for a neighborhood with strong signs of change.
Issues to consider.
Size of the neighborhood? K=1
What metrics represent a “change”?
Threshold
Derivatives of F(x, y) to get Edges¶
Test Image
An Edge where there is a rapid change in the image intensity function.
Edge Detection¶
Feature Detection.
Reliable way to extract information from images.
Compute Edges.
Derivatives using kernels and neighborhood operations.
Three (3) methods for computing edges using kernels
Image noise can complicate the computation of gradients.
The Canny Edge Detector.
Computing Discrete Gradients¶
An “operator” (mask / kernel) that effectively computes discrete derivative values with cross-correlation (using finite differences).
Various Kernels for Computing Gradients¶
Prewitt
Sobel
Roberts