Energy-Efficient Illumination and Imaging Techniques for Structured Light Applications

Homogeneous Codes for Energy-
Efficient Illumination and Imaging
Presented by Utkarsh Sinha and Jenna Lake
A
C
M
 
S
I
G
G
R
A
P
H
 
2
0
1
5
Motivation
G
o
a
l
How can we use energy most effectively in structured light applications?
A
p
p
l
i
c
a
t
i
o
n
s
Distinguish between translucency and inter-reflections
Remove artifacts caused by structured light
Reconstruct 3D objects in challenging conditions (smoke, strong lights)
Record live video from the projector’s point of view
Capture structured light video of very bright scenes
Outdoor gesture recognition
Challenges
Masks are inefficient (blocked photons are wasted)
Required for light-field displays and indirect-only photography
Live imaging requires short exposures
Devices used are low-powered
Photons emitted can cause unwanted artifacts in the image
Setup
Digital Micromirror Device (DMD) Projector
A Laser Impulse Projector
Rolling Shutter Camera
VSYNC chip
Imaging with Rolling Shutter Cameras
A single exposure is taken over hundreds
of microseconds
Each row at a slightly different time
Rolling shutter
 
Illumination patterns and masks
Projector sends out an illumination pattern
Camera sensor records single rows (masked)
Exploit these two phenomena to efficiently capture photons
Spectrum of projectors
 
Steerable mirror
Laser source
Illuminates a single
Pixel at any given time
𝞼=1
𝞼=N
2D multi-pixel mirror
Projector Lamp
Constantly uses all energy
Wastes energy
Energy efficient, complex
Energy Efficiency
Total Energy generated by Projector
Redistribution Factor
T
 
i
s
 
t
h
e
 
t
o
t
a
l
 
t
i
m
e
𝛟 is the constant wattage of the light source
l
 
i
s
 
t
h
e
 
i
l
l
u
m
i
n
a
t
i
o
n
 
p
a
t
t
e
r
n
 
(
e
n
e
r
g
y
 
e
m
i
t
t
e
d
 
b
y
 
e
a
c
h
p
r
o
j
e
c
t
o
r
 
p
i
x
e
l
)
𝛔
 is the redistribution factor.  This can range from 1 to N,
where 1 means the projector is able to redirect all power
from turned off pixels to one that are on.
Masks
Masks attenuation
Probing matrix & energy efficiency
m 
is a vector with each element is 0 to 1 that
describes the mask attenuation on the sensor
𝛾
: the scalar energy efficiency in Joules
𝜫:
 unit-less probing matrix (examples below)
High-Rank Probing Matrices
High-rank probing matrices require changing the illumination K > 1 over the
exposure time
Homogeneous Factorization
We want to maximize the energy efficiency
Homogeneous Factorization Part 2
We relax the previous equation to solve more easily.
ƛ 
is the regularization parameter which balances energy efficiency and the
reproduction of the probing matrix
Details of how this equation is derived can be found in the appendix
Homogeneous Factorization Part 3
By dropping the non-negativity constraints and leaving the sequence K
unconstrained, we get:
What does this homogeneous factorization mean?
Impulse illumination is globally optimal
For DMDs we need to create a code which will be energy efficient
Epipolar illumination is globally optimal for epipolar and non-epipolar imaging
Epipolar illumination and epipolar masking confer robustness to ambient light
Producing probing matrices
Illumination
(5x5)
Camera Mask
(5x5)
Illumination
(25x1)
Camera Mask
(1x25)
Probing Matrix
(25x25)
Producing probing matrices
Epipolar probing
 
 
Illumination
(5x5)
Camera mask
(5x5)
Probing Matrix
(25x25)
Producing probing matrices
Non-epipolar probing
 
 
Illumination
(5x5)
Camera mask
(5x5)
Probing Matrix
(25x25)
Producing probing matrices
Structured light
 
 
Illumination
(5x5)
Camera mask
(5x5)
Probing Matrix
(25x25)
Producing probing matrices
Different probing matrices capture different characteristics of the scene
Can we go in the other direction?
Factorization
Maximize efficiency while doing this
Producing probing matrices
Imaging with DMDs
Distributes power over the entire screen (𝝈=N)
Requires solving the factorization to produce the illumination pattern and mask
Imaging with DMDs
Epipolar imaging
Results from 2014
This paper
Mirror
Candle
Original scene
Imaging with DMDs
Non-epipolar imaging
Original scene
Results
Imaging with DMDs
Allows probing matrices with high ranks too!
Non-epipolar / Indirect lighting
Short-range indirect
Long-range indirect
Spectrum of projectors
 
Steerable mirror
Laser source
Illuminates a single
Pixel at any given time
𝞼=1
𝞼=N
2D multi-pixel mirror
Projector Lamp
Constantly uses all energy
Wastes energy
Energy efficient, complex
Hybrid systems (don’t exist yet)
Solving the optimization is mathematically challenging
Imaging with Laser Projectors
Epipolar and Non-epipolar imaging
Can be achieved in real-time
Epipolar imaging
Non-epipolar imaging
High specularity
Translucency
Inter-reflections
Imaging with Laser Projectors
Problems
The projector’s scanlines aren’t perfect
Thicken the region to accommodate errors
Imaging with Laser Projectors
Epipolar structured light
Lamp off
Lamp on
Reducing the iris size
Reducing the iris size
 
Imaging with Laser Projectors
Epipolar structured light
 
Imaging with Laser Projectors
Disparity Gating
Allows triangulating a point
The illumination pattern and camera mask allow us
to locate the position of a pixel
Rotate the camera 90 degrees
Set a depth plane
Capture images
Projector
Camera
Camera
Projector
Imaging with Laser Projectors
Thin smoke
Captured image
Regular disparity
Depth slices
Disparity (gating)
Thick smoke
Imaging with Laser Projectors
Dual Videography
Similar to Disparity Gating but with the plane at infinity
Capturing the physical transport matrix is still challenging
This method captures an approximate epipolar image from the projector’s view
 
C
o
n
s
The math can be hard to understand
Conclusion
Energy efficient codes product sharp, artifact-
free images
R
a
t
i
n
g
:
 
1
.
0
P
r
o
s
Uses physical limitations of the sensor
Produces sharp images
Uses off-the-shelf components
Several applications
Poses problem statement for potential future
projectors
Questions?
Slide Note
Embed
Share

This presentation at ACM SIGGRAPH 2015 by Utkarsh Sinha and Jenna Lake explores the use of homogeneous codes to optimize energy efficiency in structured light applications. The focus is on distinguishing between translucency and inter-reflections, removing artifacts, reconstructing 3D objects in challenging conditions, capturing structured light video of bright scenes, and more. Challenges include inefficient masks, the need for short exposures in live imaging, and the potential for unwanted artifacts. Various devices and techniques like VSYNC chips, rolling shutter cameras, and illumination patterns are discussed for efficient photon capture in imaging. The concept of energy efficiency is also detailed, considering factors like total time, energy generated by projectors, and the redistribution factor.

  • Energy efficiency
  • Structured light
  • Illumination patterns
  • Homogeneous codes
  • Imaging techniques

Uploaded on Sep 29, 2024 | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. Homogeneous Codes for Energy- Efficient Illumination and Imaging ACM SIGGRAPH 2015 Presented by Utkarsh Sinha and Jenna Lake

  2. Motivation Goal How can we use energy most effectively in structured light applications? Applications Distinguish between translucency and inter-reflections Remove artifacts caused by structured light Reconstruct 3D objects in challenging conditions (smoke, strong lights) Record live video from the projector s point of view Capture structured light video of very bright scenes

  3. Challenges Masks are inefficient (blocked photons are wasted) Required for light-field displays and indirect-only photography Live imaging requires short exposures Devices used are low-powered Photons emitted can cause unwanted artifacts in the image

  4. VSYNC chip Setup Rolling Shutter Camera Digital Micromirror Device (DMD) Projector A Laser Impulse Projector

  5. Imaging with Rolling Shutter Cameras A single exposure is taken over hundreds of microseconds Each row at a slightly different time Rolling shutter

  6. Illumination patterns and masks Projector sends out an illumination pattern Camera sensor records single rows (masked) Exploit these two phenomena to efficiently capture photons

  7. Spectrum of projectors Steerable mirror 2D multi-pixel mirror Wastes energy Energy efficient, complex Projector Lamp Constantly uses all energy ?=N ?=1 Laser source Illuminates a single Pixel at any given time

  8. Energy Efficiency T is the total time Total Energy generated by Projector ? is the constant wattage of the light source l is the illumination pattern (energy emitted by each projector pixel) Redistribution Factor ? is the redistribution factor. This can range from 1 to N, where 1 means the projector is able to redirect all power from turned off pixels to one that are on.

  9. Masks m is a vector with each element is 0 to 1 that describes the mask attenuation on the sensor Masks attenuation ?: the scalar energy efficiency in Joules ?: unit-less probing matrix (examples below) Probing matrix & energy efficiency

  10. High-Rank Probing Matrices High-rank probing matrices require changing the illumination K > 1 over the exposure time

  11. Homogeneous Factorization We want to maximize the energy efficiency

  12. Homogeneous Factorization Part 2 We relax the previous equation to solve more easily. is the regularization parameter which balances energy efficiency and the reproduction of the probing matrix Details of how this equation is derived can be found in the appendix

  13. Homogeneous Factorization Part 3 By dropping the non-negativity constraints and leaving the sequence K unconstrained, we get:

  14. What does this homogeneous factorization mean? Impulse illumination is globally optimal For DMDs we need to create a code which will be energy efficient Epipolar illumination is globally optimal for epipolar and non-epipolar imaging Epipolar illumination and epipolar masking confer robustness to ambient light

  15. Producing probing matrices Camera Mask (1x25) Illumination (5x5) Camera Mask (5x5) Illumination (25x1) Probing Matrix (25x25)

  16. Producing probing matrices Epipolar probing Illumination (5x5) Camera mask (5x5) Probing Matrix (25x25)

  17. Producing probing matrices Non-epipolar probing Illumination (5x5) Camera mask (5x5) Probing Matrix (25x25)

  18. Producing probing matrices Structured light Illumination (5x5) Camera mask (5x5) Probing Matrix (25x25)

  19. Producing probing matrices Different probing matrices capture different characteristics of the scene

  20. Producing probing matrices Can we go in the other direction? Factorization Maximize efficiency while doing this

  21. Imaging with DMDs Distributes power over the entire screen (?=N) Requires solving the factorization to produce the illumination pattern and mask

  22. Imaging with DMDs Mirror Epipolar imaging Candle Original scene Results from 2014 This paper

  23. Imaging with DMDs Non-epipolar imaging Original scene Results

  24. Imaging with DMDs Allows probing matrices with high ranks too! Non-epipolar / Indirect lighting Short-range indirect Long-range indirect

  25. Spectrum of projectors Steerable mirror 2D multi-pixel mirror Wastes energy Energy efficient, complex Projector Lamp Constantly uses all energy ?=N ?=1 Laser source Illuminates a single Pixel at any given time Hybrid systems (don t exist yet) Solving the optimization is mathematically challenging

  26. Imaging with Laser Projectors Epipolar and Non-epipolar imaging High specularity Can be achieved in real-time Epipolar imaging Non-epipolar imaging Translucency Inter-reflections

  27. Imaging with Laser Projectors Problems The projector s scanlines aren t perfect Thicken the region to accommodate errors

  28. Imaging with Laser Projectors Epipolar structured light Reducing the iris size Reducing the iris size Lamp off Lamp on

  29. Imaging with Laser Projectors Epipolar structured light

  30. Imaging with Laser Projectors Disparity Gating Allows triangulating a point The illumination pattern and camera mask allow us to locate the position of a pixel Projector Camera Projector Camera Rotate the camera 90 degrees Set a depth plane Capture images

  31. Imaging with Laser Projectors Thin smoke Captured image Regular disparity Depth slices Disparity (gating) Thick smoke

  32. Imaging with Laser Projectors Dual Videography Similar to Disparity Gating but with the plane at infinity Capturing the physical transport matrix is still challenging This method captures an approximate epipolar image from the projector s view

  33. Conclusion Energy efficient codes product sharp, artifact- free images Rating: 1.0 Cons Pros The math can be hard to understand Uses physical limitations of the sensor Produces sharp images Uses off-the-shelf components Several applications Poses problem statement for potential future

  34. Questions?

More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#