Advanced Techniques in Shape and Illumination Analysis
Exploring cutting-edge methods such as shape recovery from varying illumination and viewpoint, dynamic shape refinement from multi-view video, and depth estimation through photometric stereo constraints. The process involves factorization, compute depth maps, normal field calculations, and final surface recovery. The goal is to achieve high-quality 3D shape details under general illumination conditions without controlled lighting. Normal recovery and geometry refinement play a crucial role in enhancing the accuracy of the initial geometry. Results demonstrate the effectiveness of shading-based dynamic shape refinement from multi-view video data.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Illumination: Shape and Reflectance Chenglei Wu Graphics, Vision and Video Group 1/19
Agenda Shape from Varying Illumination and Viewpoint, ICCV 2007 Static Reconstruction Shading-based Dynamic Shape Refinement from Multi-view Video under General Illumination, ICCV 2011 Dynamic Reconstruction 2/19
Shape from Varying Illumination and Viewpoint Goal: given views of an object rotating in front of a light and a camera, where the lighting and object rotation is unknown, this paper recovers the 3D geometry of the object. Notice: if assuming the object is static, then the camera and the light are rotating around the objects. 3/19
Pipeline Tomasi-Kanade Factorization to get projection matrices for each image Compute dense depth map by minimizing the multi-view photometric constraint using graph cuts Compute normal field by photometric stereo Recover the final surface using the dense depth map and normal field. 4/19
Depth estimation by photometric stereo constraint Rank-3 Approximation for pixel neighborhood: ? = ?0,?1, ,?? ? ? ? = ???(?), if depth is correct, the rank of ? should be 3. photometric stereo constraint (rank 3 constraint): ? = ? ? 2 ? = ? ? ? ,? = ? :,1:3 ,? = ? 1:3,: ,? = ?(1:3,1:3) 5/19
Normal recovery and geometry refinement Photometric stereo ? = ??? ?? Solving the ambiguity matrix A by registering to the normal to that from the initial depth map. With normal map recovered, which contains the high frequency of the geometry, the initial geometry is further refined by the normal constraint. 6/19
Results 7/19
Shading-based Dynamic Shape Refinement from Multi-view Video under General Illumination Target: High-quality 3D details for dynamic shapes without any controlled illumination Embossed static detail from initial pose Previous methods Dynamic shape details from shading cues Our method Multi-view Video 8/19
Workflow (a)estimated lighting, (b)albedo map, (c)detailed surface geometry, (d)coarse tracked model, (e)multi-view images 9/19 September 30, 2024
LightingandAlbedoEstimation Assuming k different albedo segments, the albedo values and the lighting coefficients in the Spherical Harmonic basis are simultaneously estimated by finding a MAP solution: t t t t t t t t ( , ) ( , ) ( ) ( P l P ) P l I P I l priors for lighting and albedos from previous frame shading error for the current frame I l t t t , , represent images, lighting coefficients, albedos for the current frame. 10/19 September 30, 2024
Shading-based Shape Refinement Based on the estimated lighting and albedos, the coarse shape at the current frame is refined by solving a MAP problem: 1 1 t t t t t t t ( , ) ( ) ( ) P g I g P I g P g g prior: similarity to shape prediction from the previous frame likelihood: shading error 1 t t t , , I g g represent images, geometry for the current frame and the previous frame.
