Table 1.
Summary of datasets included in the benchmark dataset.
All sensor data has been cropped to the extent of NEON field sampling plots.
Table 2.
Annotations for each data type for each of the NEON sites.
Fig 1.
A 40m x 40m evaluation plot of RGB data from the Teakettle Canyon (TEAK) NEON site (left) and Bartlett Experimental Forest, New Hampshire (BART) (right).
Fig 2.
Normalized LIDAR point cloud for evaluation plot SJER_064 from the San Joaquin Experimental Range, California (left) and MLBS_071 from Mountain Lake Biological Station, Virginia.
Points are colored by height above ground.
Fig 3.
Composite hyperspectral image (left) and corresponding RGB image (right) for the MLBS site.
The composite image contains near infrared (940nm), red (650nm), and blue (430nm) channels. Trees that are difficult to segment in RGB imagery may be more separable in hyperspectral imagery due to the differing foliar chemical and structural properties of co-occurring trees.
Fig 4.
Screenshot of the program RectLabel used for tree annotation for the image-annotated crowns for NEON plot MLBS_071.
For each visible tree crown, we created a four point bounding box.
Fig 5.
Image-annotated tree crowns for the evaluation data set for two sites in the National Ecological Observation Network.
Using the RGB, LiDAR and hyperspectral products together contributes to more careful crown annotation. For some sites, such as MLBS (top row), the RGB and hyperspectral data are useful for differentiating overlapping crowns. For other sites, such as OSBS (bottom row) the LiDAR point cloud, shown as a rasterized height image, is most useful in capturing crown extent. The RGB-stretch image was produced by transforming the RGB data in the three principal components space. To create a three-band hyperspectral image, we used channels from the red, blue and infrared spectrum to capture changes in reflectance not apparent in the RGB imagery.
Fig 6.
Example evaluation from the NeonTreeEvaluation R package.
Predicted boxes (see below) in red and ground truth boxes are in black. In this image there are 10 image-annotated boxes, and 9 predictions. Each prediction matches an image-annotated box with an intersection-over-union score of greater than 0.4. This leads to a recall score of 0.9 and a precision score of 1.
Fig 7.
Intersection-over-union scores (top left), as well as plot-level inferences, between the primary annotator and a 2nd annotator.
For the IoU scores, we plotted precision and recall for 7 different intersection-over-union thresholds. As the overlap threshold decreases, the two annotators tend to agree on ground truth tree crowns. Analysis is based on 71 evaluation images (n = 1172 trees) that were separately annotated by two different annotators.
Fig 8.
Comparison of field-annotated crowns made by one author (SG) in blue (n = 16) and image-annotated crowns made by another author (BW) in red at Mountain Lake Biological Station, Virginia.
Intersection-over-union scores are shown in white. Only the image-annotated crowns associated with the field crowns are shown (out of the 206 image-annotated crowns in this image). From this and similar visualizations we determined that a threshold of 0.4 was a reasonable choice for eliminating crowns that are not sufficiently overlapping to be used for ecological analysis.
Fig 9.
Example predictions using the DeepForest algorithm.
Left) DeepForest predictions in red and compared to image-annotated crowns in black from Teakettle Canyon, California. Middle) DeepForest predictions in red are compared to field-collected stems, with matching stems in yellow and missing stems in blue, from Jones Ecological Research Center, Georgia. Right) DeepForest predictions in red with the field-annotated crown in black from Mountain Lake Biological Station, Virginia. The matching prediction is shown in bold while the other predictions are faded for visibility.
Table 3.
Benchmark evaluation scores for the DeepForest python package.