Towards embodied smartphone AR: Improving avatar tracking accuracy via face tracking and hand IK on smartphones
-
Aims: Smartphone Augmented Reality (AR) enables social applications, but single-point Inverse Kinematics (IK) avatar tracking limits pose expressiveness and introduces positional errors. We introduce DPIK, a dual-point tracking system combining ...
MoreAims: Smartphone Augmented Reality (AR) enables social applications, but single-point Inverse Kinematics (IK) avatar tracking limits pose expressiveness and introduces positional errors. We introduce DPIK, a dual-point tracking system combining hand IK with face tracking, and evaluated whether it improves tracking accuracy, embodiment, and user experience over single-point IK.
Methods: DPIK uses the front and back cameras of ARKit to track the face of the user and hand/device pose, driving avatar hand IK and hip/head positioning. In a within-subjects study (N=20), we compared DPIK against our prior single-point IK system (ARIKA) and full-body motion capture (ground truth) across positional/rotational accuracy, embodiment, and user experience. We also examined temporal drift in tracking error and Spearman correlations between error and embodiment/user experience scores.
Results: DPIK significantly outperformed ARIKA in positional accuracy for 18 of 19 joints and reduced head rotational error by over . A temporal drift analysis found that neither ARIKA nor DPIK exhibited statistically significant growth in positional error over time. DPIK received significantly higher overall user experience ratings than ARIKA. Embodiment differences were not statistically significant, though effect sizes were moderate-to-large. Exploratory correlations suggested lower DPIK embodiment scores for Acceptance and Control with higher positional error, but these did not survive multiple-comparison correction; positional error did not predict user experience ratings for either technique.
Conclusion: DPIK improves tracking accuracy and user experience over ARIKA, with tracking fidelity selectively shaping embodiment rather than overall user experience, highlighting a distinction between perceptual ownership and holistic evaluation of smartphone AR avatar systems.
Less -
Elhassan Makled, Wolfgang Broll
-
DOI: https://doi.org/10.70401/ec.2026.0025 - September 29, 2026