Modeling from Reality: Representing and Integration

Heung-Yeung Shum
doctoral dissertation, tech. report CMU-RI-TR-96-36, Robotics Institute, Carnegie Mellon University, July, 1996

  • Adobe portable document format (pdf) (4MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Traditional virtual reality systems rely on manual construction of the virtual environment, which is labor-intensiveand usually not very realistic. This thesis describes a "modeling-from-reality " system which observes from multiple viewpoints, analyzes the geometrical shape of a scene or an object, and subsequently integrates the multiple views to build a complete scene or object model form the existing environment. Our system produces statistically optimal objects by adopting a new approach to modeling from reality -- an integral approach to object modeling. The integral approach consists of two parts: how to integrate and what to integrate.

Using a polyhedral objecty as an example, this thesis shows how to integrate multiple views in a statistically optimal fashion. It is illustrated that multiple view integration can be formulated as a problem of principle component analysis with missing data (PCAMD). In spite of the noisy input and missing data at each view, the PCAMD algorithm makes use of input redundancy among different views and guarantees that the recovered object model is statistically optimal. It is shown that the problem of PCAMD can be generalized as a weighted least-squares problem, which is solved using an efficient bilinear iterative algorithm.

Matching multiple views of a polyhedral object can be accomplished by tracking its planar surface patches. It is, however, difficult to match multiple views of a free-form (i.e., smooth) object. This mesh representation provides a one-to-one mapping between a resampled convex/concave mesh to its curvature distribution on a unit sphere. The correspondence between two meshes can then be established by minimizing the difference between two curvature distributions. The same mesh representation also enables one to compare 3D shapes and to synthesize new shapes.

In summary, this thesis shows that PCAMD can be used to integrate multiple views to obtain a statistically optimal object model provided that those vies can be resampled and matched under appropriate representations. Based on the integral approach, our "modeling-from-reality" system has been successfully applied for modeling both polyhedral and free-form object.

Sponsor: ARPA (Dept. of the Army), ONR, NSF
Grant ID: DAAH04-94-G-006 N00014-95-1-0591 INI-9224521
Associated Center(s) / Consortia: Vision and Autonomous Systems Center
Number of pages: 135

Text Reference
Heung-Yeung Shum, "Modeling from Reality: Representing and Integration," doctoral dissertation, tech. report CMU-RI-TR-96-36, Robotics Institute, Carnegie Mellon University, July, 1996

BibTeX Reference
   author = "Heung-Yeung Shum",
   title = "Modeling from Reality: Representing and Integration",
   booktitle = "",
   school = "Robotics Institute, Carnegie Mellon University",
   month = "July",
   year = "1996",
   number= "CMU-RI-TR-96-36",
   address= "Pittsburgh, PA",