The authors have declared that no competing interests exist.
Wrote the paper: FA NH RF LE RP CHLS RSS NB GE JC MT TR. Contributed to design of classification according to their relevant expertise: FA NH RF RP CHLS RSS NB GE JC MT AJ KGH. Incorporated classification into CAAB: TR KGH.
‡ These authors made smaller but essential contributions to this work.
Imagery collected by still and video cameras is an increasingly important tool for minimal impact, repeatable observations in the marine environment. Data generated from imagery includes identification, annotation and quantification of biological subjects and environmental features within an image. To be long-lived and useful beyond their project-specific initial purpose, and to maximize their utility across studies and disciplines, marine imagery data should use a standardised vocabulary of defined terms. This would enable the compilation of regional, national and/or global data sets from multiple sources, contributing to broad-scale management studies and development of automated annotation algorithms. The classification scheme developed under the Collaborative and Automated Tools for Analysis of Marine Imagery (CATAMI) project provides such a vocabulary. The CATAMI classification scheme introduces Australian-wide acknowledged, standardised terminology for annotating benthic substrates and biota in marine imagery. It combines coarse-level taxonomy and morphology, and is a flexible, hierarchical classification that bridges the gap between habitat/biotope characterisation and taxonomy, acknowledging limitations when describing biological taxa through imagery. It is fully described, documented, and maintained through curated online databases, and can be applied across benthic image collection methods, annotation platforms and scoring methods. Following release in 2013, the CATAMI classification scheme was taken up by a wide variety of users, including government, academia and industry. This rapid acceptance highlights the scheme’s utility and the potential to facilitate broad-scale multidisciplinary studies of marine ecosystems when applied globally. Here we present the CATAMI classification scheme, describe its conception and features, and discuss its utility and the opportunities as well as challenges arising from its use.
Imagery collected by still and video cameras is an effective tool for minimal impact, repeatable observations in the marine environment. Imagery has been used in the marine environment in a scientific context since at least the 1950s [
Because imagery archives represent a permanent record of the environment at a particular point in time and space they will become increasingly valuable given the nature and scale of contemporary issues facing marine systems. While studies that collect and use marine imagery are often local or regional in scale, and annotate imagery with a specific question in mind (e.g. [
Standardised vocabularies of defined terms or ‘labels’ are necessary to enable the amalgamation of local and regional datasets and to realise the full potential of image databases in providing broad-scale and long-term outcomes [
Nevertheless, marine imagery is used for a wide range of purposes and often more detailed information than habitat or biotope type is required. At the finest level of identification, a standardised taxonomic classification exists for marine species through the World Register of Marine Species (WoRMS) [
We propose that a standardised annotation vocabulary (classification) for identifying taxa, shape and growth forms, and substrates in images would streamline data management, facilitate data sharing and collation for future projects; in addition, it could make historical data more accessible for other users through translations from existing classifications. Furthermore, imagery annotated with consistent, standardised labels could be used as training sets to facilitate the advancement of automated machine-learning approaches to image annotation (e.g. [
To address the issues and needs identified above we developed a flexible, hierarchical classification scheme for annotating physical and biological components observed in imagery through the Collaborative and Automated Tools for Analysis of Marine Imagery (CATAMI) project [
The need for a standard for classifying substrates and biota in marine imagery beyond broad habitat types was identified at an initial stakeholder workshop of the CATAMI Project in March 2012 (
The CATAMI Technical Working Group developed the CCS through video-conference discussions, workshops and e-mails, with refinements based on feedback from interested parties and the wider community during scientific conferences (
Continued discussions between the members of the CATAMI Technical Working Group and interested parties ensure endurance and longevity of the classification scheme. We welcome feedback regarding the use of the CCS, as well as suggestions for additions to and further refinements of the classification tree. Presently, readers can direct comments and communication to the primary authors (FA, NH, RF and LE).
Ideally, a classification for benthic substrates and biota in marine imagery should be: (i) applicable across benthic image collection methods (e.g. [
Existing classifications [
Clear documentation and description of each branch in the hierarchy is key to wide uptake and longevity of any classification. This was achieved for the CCS through the CATAMI web-site [
The CCS annotates habitats and biota; it was primarily directed towards classification of benthic imagery, but adaptation to pelagic systems is possible through further development of some of the classification branches. The CCS has two main branches, one that describes the physical components of benthic images (36 categories), the other describes the biological components (251 categories) [
The biological classification at coarsest level is into phyla or broad groups, which are then divided using either taxonomy or morphology (as shown here), depending on what can be more consistently determined from imagery. The number of categories (C) and levels (L) defined under each branch are shown. The full classification scheme can be viewed at
Additional descriptors such as health status (bleached/unbleached; damaged; etc.), colour or other interpretations can be added to each group by the use of standardised ‘modifiers’[
The
The
With the exceptions of relief and bedforms, the CCS classes can be applied to individuals or scoring points within images (
The figure illustrates the level of detail achieved within each classification system. Image 1: taken by IMAS with the ACFR AUV off the east coast of Tasmania, Australia at 24 m depth; Image 2 taken by CSIRO with the ‘Deep Camera Platform’ at Hill U Seamount south of Tasmania, Australia at 1167 m depth.
Detailed descriptions of all levels of the CCS are accessible at
The CATAMI Classification scheme (CCS) is now in use since 2013 and has been taken up by numerous local, national and international users. It has been adopted across Australia’s marine community involved in ongoing processing of marine imagery, including government organisations, academic institutions and private industry. As of April 2015, 784 copies of the visual guide, 503 copies of the technical document and 358 copies of the code file have been downloaded from the CATAMI CCS website [
The CCS is also the classification scheme underlying two web applications being developed for image scoring—CATAMI of the CATAMI Project [
The rapid acceptance and the success of the CCS are likely based on the wide scope of application paired with its versatility, as well as its documentation and curation [
The hierarchical structure is particularly useful when data from studies with different foci are combined for 'higher' level comparison of data across broad regions. Similar to global analyses of collated specimen records at family- or class-level (e.g. [
The CCS is not region specific, and thus it has the potential to be adopted for image annotation worldwide. Taxonomic data for well-known species from collections are necessary to identify bioregions (e.g. [
The CCS provides a framework for marine image annotation that fills a critical gap between coarse, habitat-level classifications and purely taxonomic classifications. Because it is standardised it represents a significant improvement on
Combining datasets increases the quantity and coverage of data that can be used for answering ecological and/or management questions at the broad scales that will increasingly be needed to tackle contemporary issues. Many regional or global models draw on data collated from collection records and published surveys with comparable taxonomic groups classified to a common level such as species, or class ([
Funding bodies increasingly demand that data generated through public funding become publicly available so they will be discoverable and available for wider use (e.g. Integrated Marine Ocean Observing System [
Indicator species or taxa have been proposed as an effective tool for assessing ecosystem health and monitoring change [
CATAMI Level 3 Category | CATAMI Level 4 Category | Description | Successional Status | Opportu-nistic? | Ecol. State Group | Examples of CATAMI group |
---|---|---|---|---|---|---|
Filamentous / filiform | Green, Red, Brown | Appears very fine and thread- or hair-like but may not necessarily be technically a filament | Early successional | Yes | II | |
Sheet-like / membraneous | Red, Brown | Thin, delicate and often translucent. A flattened and sheet-like structure | Early successional | Some | II | |
Sheet-like / membraneous | Green | As above | Early successional | Yes | II | |
Globose / saccate | Green, Red, Brown | Spherical shape or balloon-like form. | Early successional | No | II | |
Laminate | Green, Red, Brown | Low profile, plate-like and lobed forms | Mid- successional | No | I/II | |
Erect fine branching | Green, Red, Brown | Distinct branching form with a vertical growth habit. Branches are small or narrow | Mid- successional | No | II | |
Erect coarse branching | Green, Red, Brown | Distinct branching form with a vertical growth habit. Branches are robust or have broader blades than fine-branching | Late- successional | No | I | |
Large canopy-forming | Brown | Large (>>50 cm when mature) and robust, habitat- forming species. Generally large and distinctive fucoids and kelps | Late- successional | No | I | |
Articulated calcareous | Green, Red, Brown | Jointed or segmented, calcified algae | Late- successional | No | I | |
Encrusting | Red, Brown | Crust-like; thin form growing flattened and closely adhering to the substratum. | Late- successional | No | I | crustose coralline reds, |
A range of macroalgal indices have been proposed for subtidal environments that utilise morphological and biological traits of species or groups. Table 1 shows how CATAMI macroalgal classifications aligned with life-history characteristics often used in the derivation of indicators, and with the ecological status groups proposed by Orfandis et al. for monitoring the health of marine systems [
Other branches of the CCS hierarchy have similar application potential. The reporting of tropical reef health status is generally based on the composition of morpho-functional groups of stony corals (e.g. [
The CCS provides a framework for identifying and labelling structures in marine images, without being prescriptive regarding methods for the collection and scoring of marine imagery. It can, nevertheless, guide and streamline the process of developing protocols when setting up new studies. It can also facilitate discussions between research providers and clients regarding the level of detail and outcomes required or achievable, when planning observational studies. In addition, enabling assessment across a wide spectrum of taxa through the CCS can bring certain groups into focus that otherwise may be overlooked. For example, sponges are often only scored generically as ‘sponges’ or ‘filter feeders’, despite being important and diverse components of benthic ecosystems (e.g. [
With a standard vocabulary, the foundation is laid for developing protocols or ‘standards’ for image processing. National or international standards document
Annotating large volumes of marine imagery collected by still and video platforms is a time consuming process and a standardised classification scheme opens exciting opportunities to increase processing efficiency. Automatic image classification and annotation of marine imagery using computer vision algorithms has rapidly advanced in recent years [
The engagement of citizen scientists, enthusiastic non-experts who complete tasks that contribute to scientific programs, presents another opportunity to increase the efficiency of both capturing
The CCS facilitates collation and combination of datasets across large geographical and bathymetric scales, maximizing the potential value of the data. However, combining datasets collected for a range of different purposes presents a number of pragmatic, logistic and analytical challenges (e.g. [
Imagery and associated derived data are increasingly important tools for minimal impact, repeatable observations in the marine environment. In addition, an increasing need exists to publish data to ensure their longevity, with increasing reliance on digitally accessible data for large-scale studies, and modelling. The CCS caters to the need for standardised, defined terminology which is fundamental for broad dissemination and uptake of data. The CCS vocabulary can be used to classify physical features and biota in order to describe and quantify (statistically or otherwise) biological communities or habitats in imagery. The scheme is collaboratively designed to be easily accessible, adaptable, and agile, with the potential to translate existing data into the scheme. The strength of the CCS lies in its ability to encompass multiple scales and resolutions, with flexibility that allows its use with most scoring methods and annotation platform types for underwater imagery and video. Longevity of the CCS is enhanced by continual maintenance and curation through the CAAB coding system by CSIRO [
(DOCX)
(DOCX)
(DOCX)
We would like to thank Stefan Williams (ACFR) for his work initiating the CATAMI project and organising the first stakeholder workshop, and Keith Hayes (CSIRO) for supporting the development of the CCS through the Australian National Environmental Research Program Marine Biodiversity Hub and the Australian Research Council. Our thanks also go to all the members of the CATAMI Technical Working Group listed in