Space/Frequency Shape Inference and Segmentation of Textured 3D Surfaces

John Krumm
doctoral dissertation, tech. report CMU-RI-TR-93-32, Robotics Institute, Carnegie Mellon University, December, 1993

  • Adobe portable document format (pdf) (8MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Image texture is useful for segmentation and for computing surface orientations of uniformly textured objects. If texture is ignored, it can cause failure for stereo and gray-scale segmentation algorithms. In the past, mathematical representations of image texture have been applied to only specific texture problems, and no consideration has been given to the models' generality across different computer vision tasks and different image phenomea. We advocate the space/frequency representation, which shows the local spatial frequency content of every point in the image. From several different methods of computing the representation, we pick the spectrogram. The spectrogram elucidates many disparate image phenomena including texture boundaries, texture in perspective, aliasing, zoom, and blur.

Many past shape-from-texture algorithms require potentially unreliable feature detections and "magic numbers" (arbitrary parameters), and none of them were developed in the context of a more general texture-understanding system. Toward this end, we show that the spatial frequency shifts caused by perspective can be approximated by an affine transformation which is a function of the textured surface's surface normal. We use this relationship in three different shape-from-texture algorithms. Two of them require no feature-finding and work on the raw spectrogram, giving a high-level scene parameter directly from low-level image data. The first algorithm includes an analytical sensitivity analysis. The third algorithm works with just the peak frequencies and gives a fast way of computing local surface normals from periodic texture. The algorithms need only a few magic numbers. On real textures, the average error in computed surface normal is only about four degrees.

We use the third algorithm to solve a long-standing problem in image texture analysis: segmenting images of textured, 3D surfaces. Past work in texture segmentation and shape-from-texture is based on assumptions that make this problem impossible to solve. 3D perspective effects warp the otherwise uniform textures so segmentation fails. We develop a region-growing algorithm that accounts for the surface normals by unwarping the frequency distribution. It uses a minimum description length merge criterion. Our algorithm successfully segments images of real texture. We conclude by showing how the space/frequency representation has the potential for unifying several different computer vision algorithms.

Sponsor: Avionics Laboratory, Wright Research and Development Center, AFSC, U.S. Air Force, Wright-Patterson AFB, OH 45433-6543, ARPA, NASA
Grant ID: F33615-90-C-1465, Order No.7597, NGT-50423
Associated Center(s) / Consortia: Vision and Autonomous Systems Center
Associated Lab(s) / Group(s): Calibrated Imaging Lab
Number of pages: 206

Text Reference
John Krumm, "Space/Frequency Shape Inference and Segmentation of Textured 3D Surfaces," doctoral dissertation, tech. report CMU-RI-TR-93-32, Robotics Institute, Carnegie Mellon University, December, 1993

BibTeX Reference
   author = "John Krumm",
   title = "Space/Frequency Shape Inference and Segmentation of Textured 3D Surfaces",
   booktitle = "",
   school = "Robotics Institute, Carnegie Mellon University",
   month = "December",
   year = "1993",
   number= "CMU-RI-TR-93-32",
   address= "Pittsburgh, PA",