Supervised Descent Method

Xuehan Xiong
doctoral dissertation, tech. report CMU-RI-TR-15-28, Robotics Institute, Carnegie Mellon University, September, 2015


Abstract
In this dissertation, we focus on solving Nonlinear Least Squares problems using a supervised approach. In particular, we developed a Supervised Descent Method (SDM), performed thorough theoretical analysis, and demonstrated its effectiveness on optimizing analytic functions, and four other real-world applications: Inverse Kinematics, Rigid Tracking, Face Alignment (frontal and multi-view), and 3D Object Pose Estimation. In Rigid Tracking, SDM was able to take advantage of more robust features, such as, HoG and SIFT. Those non-differentiable image features were out of consideration of previous work because they relied on gradient-based methods for optimization. In Inverse Kinematics where we minimize a non-convex function, SDM achieved significantly better convergence than gradient-based approaches. In Face Alignment, SDM achieved state-of-the-arts results. Moreover, it was extremely computationally efficient, which makes it applicable for many mobile applications. In addition, we provided a unified view of several popular methods including SDM on sequential prediction, and reformulated them as a sequence of function compositions. Finally, we suggested some future research directions on SDM and sequential prediction.

Keywords
nonlinear optimization, global optimization, non-convex optimization, nonlinear least squares, face alignment, facial feature tracking, pose estimation, inverse kinematics, imitation learning, learning from demonstration, policy derivation, neural network, stacking, gradient boosting, sequential prediction

Notes

Text Reference
Xuehan Xiong, "Supervised Descent Method," doctoral dissertation, tech. report CMU-RI-TR-15-28, Robotics Institute, Carnegie Mellon University, September, 2015

BibTeX Reference
@phdthesis{Xiong_2015_8016,
   author = "Xuehan Xiong",
   title = "Supervised Descent Method",
   booktitle = "",
   school = "Robotics Institute, Carnegie Mellon University",
   month = "September",
   year = "2015",
   number= "CMU-RI-TR-15-28",
   address= "Pittsburgh, PA",
}