Abstract
Animals in groups touch each other, move in paths that cross, and interact in complex ways. Current video tracking methods sometimes switch identities of unmarked individuals during these interactions. These errors propagate and result in random assignments after a few minutes unless manually corrected. We present idTracker, a multitracking algorithm that extracts a characteristic fingerprint from each animal in a video recording of a group. It then uses these fingerprints to identify every individual throughout the video. Tracking by identification prevents propagation of errors, and the correct identities can be maintained indefinitely. idTracker distinguishes animals even when humans cannot, such as for size-matched siblings, and reidentifies animals after they temporarily disappear from view or across different videos. It is robust, easy to use and general. We tested it on fish (Danio rerio and Oryzias latipes), flies (Drosophila melanogaster), ants (Messor structor) and mice (Mus musculus).
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$259.00 per year
only $21.58 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Ohayon, S., Avni, O., Taylor, A.L., Perona, P. & Egnor, S.E.R. Automated multi-day tracking of marked mice for the analysis of social behavior. J. Neurosci. Methods 219, 10–19 (2013).
Shemesh, Y. et al. High-order social interactions in groups of mice. eLife 2, e00759 (2013).
Dennis, R.L., Newberry, R.C., Cheng, H.-W. & Estevez, I. Appearance matters: artificial marking alters aggression and stress. Poult. Sci. 87, 1939–1946 (2008).
Dahlbom, S.J., Lagman, D., Lundstedt-Enkel, K., Sundström, L.F. & Winberg, S. Boldness predicts social status in zebrafish (Danio rerio). PLoS ONE 6, e23565 (2011).
Delcourt, J., Becco, C., Vandewalle, N. & Poncin, P. A video multitracking system for quantification of individual behavior in a large fish shoal: advantages and limits. Behav. Res. Methods 41, 228–235 (2009).
Kato, S. et al. A computer image processing system for quantification of zebrafish behavior. J. Neurosci. Methods 134, 1–7 (2004).
Mirat, O., Sternberg, J.R., Severi, K.E. & Wyart, C. ZebraZoom: an automated program for high-throughput behavioral analysis and categorization. Front. Neural Circuits 7, 107 (2013).
Branson, K., Robie, A.A., Bender, J., Perona, P. & Dickinson, M.H. High-throughput ethomics in large groups of Drosophila. Nat. Methods 6, 451–457 (2009).
Branson, K. & Belongie, S. in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 1, 1039–1046 (IEEE, 2005).
Fontaine, E. et al. Automated visual tracking for studying the ontogeny of zebrafish swimming. J. Exp. Biol. 211, 1305–1316 (2008).
de Chaumont, F. et al. Computerized video analysis of social interactions in mice. Nat. Methods 9, 410–417 (2012).
Butail, S. & Paley, D.A. Three-dimensional reconstruction of the fast-start swimming kinematics of densely schooling fish. J. R. Soc. Interface 9, 77–88 (2012).
Attanasi, A. et al. Tracking in three dimensions via multi-path branching. Preprint http://arXiv.org/abs/1305.1495 (2013).
Straw, A.D., Branson, K., Neumann, T.R. & Dickinson, M.H. Multi-camera real-time three-dimensional tracking of multiple flying animals. J. R. Soc. Interface 8, 395–409 (2011).
Herbert-Read, J.E. et al. Inferring the rules of interaction of shoaling fish. Proc. Natl. Acad. Sci. USA 108, 18726–18731 (2011).
Huang, J., Kumar, S.R., Mitra, M., Zhu, W.-J. & Zabih, R. Spatial color indexing and applications. Int. J. Comput. Vis. 35, 245–268 (1999).
Lister, J.A., Robertson, C.P., Lepage, T., Johnson, S.L. & Raible, D.W. nacre encodes a zebrafish microphtalmia-related protein that regulates neural-crest-derived pigment cell fate. Development 126, 3757–3767 (1999).
Sumpter, D.J.T. Collective Animal Behavior (Princeton University Press, 2010).
Cavagna, A. et al. Scale-free correlations in starting flocks. Proc. Natl. Acad. Sci. USA 107, 11865–11870 (2010).
Gallup, A.C. et al. Visual attention and the acquisition of information in human crowds. Proc. Natl. Acad. Sci. USA 109, 7245–7250 (2012).
Gautrais, J. et al. Deciphering interactions in moving animal groups. PLoS Comput. Biol. 8, e1002678 (2012).
Vicsek, T. & Zafeiris, A. Collective motion. Phys. Rep. 517, 71–140 (2012).
King, A.J. et al. Selfish-herd behavior of sheep under threat. Curr. Biol. 22, R561–R562 (2012).
Arganda, S., Pérez-Escudero, A. & de Polavieja, G.G. A common rule for decision-making in animal collectives across species. Proc. Natl. Acad. Sci. USA 109, 20508–20513 (2012).
Strandburg-Peshkin, A. et al. Visual sensory networks and effective information transfer in animal groups. Curr. Biol. 23, R709–R711 (2013).
Nagy, M., Akos, Z., Biro, D. & Vicsek, T. Hierarchical group dynamics in pigeon flocks. Nature 464, 890–893 (2010).
Lowe, D.G. in Proc. IEEE Int. Conf. Comput. Vis. 2, 1150–1157 (1999).
Turk, M.A. & Pentland, A.P. in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 586–591 (1991).
Khotanzad, A. & Hong, Y.H. Rotation invariant image recognition using features selected via a systematic method. Pattern Recognit. 23, 1089–1101 (1990).
Wu, X., Zhang, D. & Wang, K. Fisherpalms based palmprint recognition. Pattern Recognit. Lett. 24, 2829–2838 (2003).
Crall, J.P., Stewart, C.V., Berger-Wolf, T.Y., Rubenstein, D.I. & Sundaresan, S.R. in Proc. IEEE Workshop Appl. Comput. Vis. 230–237 (2013).
Bhatkar, A. & Whitcomb, W.H. Artificial diet for rearing various species of ants. Fla. Entomol. 53, 229–232 (1970).
Acknowledgements
We thank P. Bovolenta (Centro de Biología Molecular Severo Ochoa, Consejo Superior de Investigaciones Científicas (CSIC)), A. Dussutour (Centre de Recherches sur la Cognition Animale, Centre National de la Recherche Scientifique, Toulouse University), A. Ferrús (Instituto Cajal, CSIC), A. Sorribes (Instituto Cajal, CSIC), G. Sumbre (Ecole Normale Supérieure) and J.L. Trejo (Instituto Cajal, CSIC) for providing animals. We also thank I. Arganda-Carreras, K. Branson, I. Couzin, J. Gautrais, R. Ottenhoff, H. Roosken, R. Tegelenbosch, W. van Dommelen and H. Wu for feedback; A. Bruce, A. King, E. Pope, G. Thelauraz, R. Goulard, U. Lopez and D. Calovi for testing in their labs; and R. Gil de Sagredo, G. Madirolas, A.C. Román and F. Romero-Ferrero for a critical reading of the paper. We acknowledge an FPU fellowship from Ministerio de Economía y Competitividad, Spain (to A.P.-E.), an FPI fellowship from Ministerio de Economía y Competitividad (to R.C.H.), a JAE fellowship from CSIC (to J.V.-P.), funding from Spanish Plan Nacional BFU2009-09967 and BFU2012-33448 from Ministerio de Economía y Competitividad (to G.G.d.P.) and funding from ERASysBio+ initiative supported under the European Union European Research Area Networks (ERA-NET) Plus scheme in the FP7 Seventh Framework Programme as EUI2009-04090 (to G.G.d.P., including contracts to S.A. and A.P.-E.).
Author information
Authors and Affiliations
Contributions
A.P.-E. designed the project, performed experiments, analyzed data, wrote and tested the software and wrote the paper; R.C.H. performed experiments, tested the software and analyzed data; J.V.-P. performed experiments, tested the software and analyzed data; S.A. performed experiments and tested the software, G.G.d.P. designed the project, advised and directed the project including experiments, software and data analysis and wrote the paper.
Corresponding authors
Ethics declarations
Competing interests
Consejo Superior de Investigaciones Cientificas (CSIC) owns a patent describing this method (patent no. PCT/ES2013/070585). A.P.-E., G.G.d.P. and S.A. are listed as inventors of the patent and will receive a part of the revenues that it may generate. Noncommercial use of the method and the associated software is allowed free of charge
Supplementary information
Supplementary Text and Figures
Supplementary Figures 1–7, Supplementary Table 1 and Supplementary Notes 1–4 (PDF 16844 kb)
Supplementary Software
Source code of idTracker (Matlab and C) (ZIP 130 kb)
Supplementary Data
Raw data and scripts to reproduce the results in Figure 4 (ZIP 45931 kb)
idTracker: Visual Summary
This movie explains the main features and advantages of idTracker, outlines the method, and shows examples of multitracking of several species: zebrafish (Danio rerio), medaka fish (Oryzias latipes), fruitflies (Drosophila melanogaster), mice (Mus musculus) and ants. (MP4 20213 kb)
Rights and permissions
About this article
Cite this article
Pérez-Escudero, A., Vicente-Page, J., Hinz, R. et al. idTracker: tracking individuals in a group by automatic identification of unmarked animals. Nat Methods 11, 743–748 (2014). https://doi.org/10.1038/nmeth.2994
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nmeth.2994
This article is cited by
-
High-resolution ethograms, accelerometer recordings, and behavioral time series of Japanese quail
Scientific Data (2024)
-
Labidochromis caeruleus cichlid preference for background colour varied between individuals and groups but did not vary for body colour of other fish
Journal of Ethology (2024)
-
An automated, low-latency environment for studying the neural basis of behavior in freely moving rats
BMC Biology (2023)
-
Metabolic shift toward ketosis in asocial cavefish increases social-like affinity
BMC Biology (2023)
-
DVT: a high-throughput analysis pipeline for locomotion and social behavior in adult Drosophila melanogaster
Cell & Bioscience (2023)