Dense 3D Reconstruction of Dynamic Actors in Natural Environments
Utilizing Multiple Flying Cameras for Enhanced 3D Human Pose Estimation in Dynamic, Natural Settings
Time: Sep 2022 – Present
Introduction
This project sets out to accomplish dense 3D reconstruction of dynamic actors in natural environments using uncalibrated, multiple flying cameras. It is inspired by the advancements in multi-view 3D pose estimation, particularly the innovative work by Xu and Kitani. This field has seen significant progress, with notable contributions from Joo et al. (2015), Reddy et al. (2021), and Tu et al. (2020), focusing on calibrated camera networks and controlled settings. Our project extends this technology into more dynamic, natural environments, using adaptive aerial formations for filming groups, field testing with GPS-equipped aerial robots, and a comprehensive 3D pose reconstruction pipeline that adapts Xu and Kitani’s methods for uncalibrated cameras.
Design
The design leverages Isaac Sim for simulating natural environments and OpenPose for 2D human pose reconstruction, setting the stage for advanced 3D pose estimation.
Evaluation
Extensive testing in simulated environments assures accuracy and robustness under diverse conditions.
Methods
Methods include extracting datasets from Isaac Sim, applying OpenPose, and utilizing techniques from Yan Xu and Kris Kitani’s research for 3D reconstruction.
Results
The results demonstrate significant advancements in precise and reliable 3D human pose estimations in uncontrolled settings.
Conclusion
This project exemplifies the effective use of uncalibrated camera networks in dense 3D human pose reconstruction, promising new applications in dynamic, natural environments.