EgoPoints: Advancing Point Tracking for Egocentric Videos

Ahmad A K Dar Khalil; Rhodri G L Guerrier; Adam Harley; Dima  Damen

doi:10.1109/WACV61041.2025.00829

EgoPoints: Advancing Point Tracking for Egocentric Videos

Ahmad A K Dar Khalil, Rhodri G L Guerrier, Adam Harley, Dima Damen

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

6 Downloads (Pure)

Abstract

We introduce EgoPoints, a benchmark for point tracking in egocentric videos. We annotate 4.7K challenging tracks in egocentric sequences. Compared to the popular TAP-Vid-DAVIS evaluation benchmark, we include 9x more points that go out-of-view and 59x more points that require re-identification (ReID) after returning to view. To measure the performance of models on these challenging points, we introduce evaluation metrics that specifically monitor tracking performance on points in-view, out-of-view, and points that require re-identification. We then propose a pipeline to create semi-real sequences, with automatic ground truth. We generate 11K such sequences by combining dynamic Kubric objects with scene points from EPIC Fields. When fine-tuning point tracking methods on these sequences and evaluating on our annotated EgoPoints sequences, we improve CoTracker across all metrics, including the tracking accuracy δ⋆avg by 2.7 percentage points and accuracy on ReID sequences (ReIDδavg) by 2.4 points. We also improve δ⋆avg and ReIDδavg of PIPs++ by 0.3 and 2.8 respectively.

Original language	English
Title of host publication	2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	8556-8565
Number of pages	10
ISBN (Electronic)	9798331510831
ISBN (Print)	9798331510848
DOIs	https://doi.org/10.1109/WACV61041.2025.00829
Publication status	Published - 8 Apr 2025
Event	IEEE/CVF Winter Conference on Applications of Computer Vision: WACV - Tuscon, Arizona, United States Duration: 28 Feb 2025 → 5 Mar 2025 https://wacv2025.thecvf.com/

Publication series

Name	IEEE Workshop on Applications of Computer Vision (WACV)
Publisher	IEEE
ISSN (Print)	2472-6737
ISSN (Electronic)	2642-9381

Conference

Conference	IEEE/CVF Winter Conference on Applications of Computer Vision
Country/Territory	United States
City	Tuscon, Arizona
Period	28/02/25 → 5/03/25
Internet address	https://wacv2025.thecvf.com/

Bibliographical note

Publisher Copyright:
© 2025 IEEE.

Access to Document

10.1109/WACV61041.2025.00829

Full-text PDF (accepted manuscript)
This is the accepted author manuscript (AAM) of the article which has been made Open Access under the University of Bristol's Scholarly Works Policy. The final published version (Version of Record) can be found on the publisher's website. The copyright of any third-party content, such as images, remains with the copyright holder.
Accepted author manuscript, 14.9 MBLicence: CC BY

Handle.net

Persistent link

Cite this

Dar Khalil, A. A. K., Guerrier, R. G. L., Harley, A., & Damen, D. (2025). EgoPoints: Advancing Point Tracking for Egocentric Videos. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (pp. 8556-8565). (IEEE Workshop on Applications of Computer Vision (WACV)). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/WACV61041.2025.00829

@inproceedings{db255aa73a694acd8bec24abce78eae9,

title = "EgoPoints: Advancing Point Tracking for Egocentric Videos",

abstract = "We introduce EgoPoints, a benchmark for point tracking in egocentric videos. We annotate 4.7K challenging tracks in egocentric sequences. Compared to the popular TAP-Vid-DAVIS evaluation benchmark, we include 9x more points that go out-of-view and 59x more points that require re-identification (ReID) after returning to view. To measure the performance of models on these challenging points, we introduce evaluation metrics that specifically monitor tracking performance on points in-view, out-of-view, and points that require re-identification. We then propose a pipeline to create semi-real sequences, with automatic ground truth. We generate 11K such sequences by combining dynamic Kubric objects with scene points from EPIC Fields. When fine-tuning point tracking methods on these sequences and evaluating on our annotated EgoPoints sequences, we improve CoTracker across all metrics, including the tracking accuracy δ⋆avg by 2.7 percentage points and accuracy on ReID sequences (ReIDδavg) by 2.4 points. We also improve δ⋆avg and ReIDδavg of PIPs++ by 0.3 and 2.8 respectively.",

author = "{Dar Khalil}, {Ahmad A K} and Guerrier, {Rhodri G L} and Adam Harley and Dima Damen",

note = "Publisher Copyright: {\textcopyright} 2025 IEEE.; IEEE/CVF Winter Conference on Applications of Computer Vision : WACV ; Conference date: 28-02-2025 Through 05-03-2025",

year = "2025",

month = apr,

day = "8",

doi = "10.1109/WACV61041.2025.00829",

language = "English",

isbn = "9798331510848",

series = "IEEE Workshop on Applications of Computer Vision (WACV)",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

pages = "8556--8565",

booktitle = "2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)",

address = "United States",

url = "https://wacv2025.thecvf.com/",

}

Dar Khalil, AAK , Guerrier, RGL, Harley, A & Damen, D 2025, EgoPoints: Advancing Point Tracking for Egocentric Videos. in 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE Workshop on Applications of Computer Vision (WACV), Institute of Electrical and Electronics Engineers (IEEE), pp. 8556-8565, IEEE/CVF Winter Conference on Applications of Computer Vision, Tuscon, Arizona, United States, 28/02/25. https://doi.org/10.1109/WACV61041.2025.00829

EgoPoints: Advancing Point Tracking for Egocentric Videos. / Dar Khalil, Ahmad A K ; Guerrier, Rhodri G L; Harley, Adam et al.
2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Institute of Electrical and Electronics Engineers (IEEE), 2025. p. 8556-8565 (IEEE Workshop on Applications of Computer Vision (WACV)).

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - EgoPoints

T2 - IEEE/CVF Winter Conference on Applications of Computer Vision

AU - Dar Khalil, Ahmad A K

AU - Guerrier, Rhodri G L

AU - Harley, Adam

AU - Damen, Dima

PY - 2025/4/8

Y1 - 2025/4/8

N2 - We introduce EgoPoints, a benchmark for point tracking in egocentric videos. We annotate 4.7K challenging tracks in egocentric sequences. Compared to the popular TAP-Vid-DAVIS evaluation benchmark, we include 9x more points that go out-of-view and 59x more points that require re-identification (ReID) after returning to view. To measure the performance of models on these challenging points, we introduce evaluation metrics that specifically monitor tracking performance on points in-view, out-of-view, and points that require re-identification. We then propose a pipeline to create semi-real sequences, with automatic ground truth. We generate 11K such sequences by combining dynamic Kubric objects with scene points from EPIC Fields. When fine-tuning point tracking methods on these sequences and evaluating on our annotated EgoPoints sequences, we improve CoTracker across all metrics, including the tracking accuracy δ⋆avg by 2.7 percentage points and accuracy on ReID sequences (ReIDδavg) by 2.4 points. We also improve δ⋆avg and ReIDδavg of PIPs++ by 0.3 and 2.8 respectively.

AB - We introduce EgoPoints, a benchmark for point tracking in egocentric videos. We annotate 4.7K challenging tracks in egocentric sequences. Compared to the popular TAP-Vid-DAVIS evaluation benchmark, we include 9x more points that go out-of-view and 59x more points that require re-identification (ReID) after returning to view. To measure the performance of models on these challenging points, we introduce evaluation metrics that specifically monitor tracking performance on points in-view, out-of-view, and points that require re-identification. We then propose a pipeline to create semi-real sequences, with automatic ground truth. We generate 11K such sequences by combining dynamic Kubric objects with scene points from EPIC Fields. When fine-tuning point tracking methods on these sequences and evaluating on our annotated EgoPoints sequences, we improve CoTracker across all metrics, including the tracking accuracy δ⋆avg by 2.7 percentage points and accuracy on ReID sequences (ReIDδavg) by 2.4 points. We also improve δ⋆avg and ReIDδavg of PIPs++ by 0.3 and 2.8 respectively.

UR - https://wacv2025.thecvf.com/

U2 - 10.1109/WACV61041.2025.00829

DO - 10.1109/WACV61041.2025.00829

M3 - Conference Contribution (Conference Proceeding)

SN - 9798331510848

T3 - IEEE Workshop on Applications of Computer Vision (WACV)

SP - 8556

EP - 8565

BT - 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

PB - Institute of Electrical and Electronics Engineers (IEEE)

Y2 - 28 February 2025 through 5 March 2025

ER -

EgoPoints: Advancing Point Tracking for Egocentric Videos

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Handle.net

Other files and links

Fingerprint

8030 EPSRC via Oxford EP/T028572/1 Visual AI

8459 EPSRC EP/T004991/1 UMPIRE - Dima Aldamen Fellowship

EPIC-KITCHENS-DIFF

EPIC-KITCHENS-100

Cite this

EgoPoints: Advancing Point Tracking for Egocentric Videos

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Handle.net

Other files and links

Fingerprint

Projects

8030 EPSRC via Oxford EP/T028572/1 Visual AI

8459 EPSRC EP/T004991/1 UMPIRE - Dima Aldamen Fellowship

Datasets

EPIC-KITCHENS-DIFF

EPIC-KITCHENS-100

Cite this