Pose Tracking and Augmented Reality for 3D-Printed Objects

Abstract

Digital and physical data visualizations have complementary strengths and weaknesses. For example, digital versions better-facilitate complex visual effects or data retrieval but physical ones offer intuitive spatial navigation. Applying augmented reality (AR) overlays onto physical models (AR+physicalization), one can realize such benefits simultaneously. In this thesis, we apply AR to affordable 3D prints. These monochrome prints lack texture, so we utilize region-based pose tracking methods, well-suited for such objects, to accurately position the objects' digital overlays. We adapt such algorithms for mobile device deployment (which imposes strict resource constraints) and demonstrate how to robustly handle close-up viewing of small visual details via techniques such as subobject tracking or separate ``side-platform'' prints. For demonstration, we construct an example mobile application that renders satellite imagery, historical data, and 3D reconstructions onto (or beside) 3D prints of historical locations. This thesis also covers ongoing efforts to improve pose tracking via predictive, image-free motion extrapolation using neural networks. These predictions can supplement the computer vision estimates via use of an extended Kalman filter.