Geometry-based representations

Much of our work  is cross-cutting across different areas. Here we delve a little deeper into the more theoretical side of our work.

We have been spearheading the development and broad application of many representations rooted in differential geometry and topology in areas where the need for geometry arises quite naturally — either due to constraints on the core features themselves, or due to certain invariance criteria needed to be respected during high-level inference. In this research thrust, we draw fuse mathematical theory with effective algorithmic and approximation techniques to deliver improved performance (accuracy, stability, scalability etc.) in a wide array of applications: including in human activity recognition, video-based scene analysis, activity analysis from wearable devices, as well as in systems for physical rehabilitation. Our sampling of our publications in this area is given below.

TTN
In this paper, we propose “Temporal Transformer Network”, capable generating input-dependent time-warping functions which lead to rate-robust representations. It is an interpretable differentiable module, which can be easily integrated at the front end of a classification network.
  • Suhas Lohit, Qiao Wang, Pavan Turaga, “Temporal Transformer Networks: Joint Learning of Invariant and Discriminative Time Warping”, The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 12426-1243.
ECCV_TopoGeometry
We develop efficient representations of topological persistence diagrams, and stability proofs, in a manner that we anticipate will be more amenable to fusion with contemporary machine learning pipelines. These tools can be used to enforce invariances of many kinds in many applications.
  • Som et. al. “Perturbation Robust Representations of Topological Persistence Diagrams”, at the European Conference on Computer Vision (ECCV) 2018. [pdf]
  • Lohit et. al., “Predicting Dynamical Evolution of Human Activities from a Single Image”, at the 4th workshop on Differential Geometry in Computer Vision and Machine Learning (in conjunction with IEEE CVPR), 2018.
MangaImpliedMotion
We use neuroscience insights about how Hokusai Manga art depicts “implied motion” to develop a predictive computational model which is shown useful for pose-stability analysis, action completion, as well as action recognition. We using Riemannian geometry of linear dynamical systems and statistics on manifolds to develop the model.
CVPR2015
Elastic low-dimensional coding of trajectories on Riemannian manifolds. Anirudh et al PAMI 2016.

 

  • R. Anirudh, P. Turaga, J. Su, A. Srivastava, “Elastic functional coding of Riemannian Trajectories“, accepted at the IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2016. [pdf]

 

  • R. Anirudh, P. Turaga, J. Su, A. Srivastava, “Elastic functional coding human actions: From vector-fields to latent variables”, at IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. [pdf]

 

HilbertTopo
Non-linear dynamics, and topological properties of dynamical attractors for studying human-movement. See below for related papers.
  • Rushil Anirudh, Vinay Venkataraman, Karthikeyan Natesan Ramamurthy, Pavan Turaga, “A Riemannian Framework for Statistical Analysis of Topological Persistence Diagrams”, at the Second International Workshop on Differential Geometry in Computer Vision and Machine Learning (Diff-CVML) held in conjunction with CVPR 2016. [pdf]
  • V. Venkataraman, P. Turaga, “Shape distributions of  non-linear dynamical systems for video-based inference”, at the IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) Dec 2016. [pdf]
  • Vinay Venkataraman, Pavan Turaga, “Dynamical Regularity for Action Analysis”, at the British Machine Vision Conference (BMVC), September 2015. [pdf]
  • Qiao Wang, Rushil Anirudh, Pavan Turaga, “Temporal Reflection Symmetry of Human Actions: A Riemannian Analysis”, at the First International Workshop on Differential Geometry in Computer Vision (Diff-CV) held in conjunction with BMVC 2015. [pdf]
RCCV
Learn more about the broader applications of Riemannian geometry in Computer Vision and Machine Learning.