Distance fields help matching surfaces

We investigate in this project the possibility to infer the dense correspondences between a surface model and a complete 3D visual hull obtained in multiple camera environments. With frame-wise data-model associations, our method is able to detect human shapes in a clutter scene, which we term tracking-by-detection of 3D human shapes.

We convert surface meshes into volumetric truncated signed distance fields (TSDF), where one can design features more straightforwardly compared to working with surface manifolds. We then apply random regression forests to learn the associations. The results on the FAUST dataset are shown in the image above. Check the following video and the papers below for more details!

1. C. H. Huang, B. Allain, E. Boyer, J.-S. Franco, F. Tombari, N. Navab, and S. Ilic, “Tracking-by-Detection of 3D Human Shapes: from Surfaces to Volumes,” TPAMI, 2017. (pdf, video)

2. C. H. Huang, F. Tombari, N. Navab, “Repeatable Local Coordinate Frame for 3D Human Motion Tracking: from Rigid to Non-Rigid,” 3D Vision conference, 2015. (pdf)

3. C. H. Huang, E. Boyer, B. do Canto Angonese, N. Navab, and S. Ilic, “Toward User-specific Tracking by Detection of Human Shapes in Multi-Camera,” CVPR, 2015. (pdf, video)

#randomforest