Fig 1.
A time-series of 3D point clouds of two plants (maize (top) and tomato (bottom)) captured during its growth.
Our goal is to develop techniques for automatically registering such 3D scans captured under challenging conditions of changing topology and anisotropic growth of the plant.
Fig 2.
Extracting skeletal structure for using semantics of the plant.
The figure illustrates the skeletonization pipeline for a maize (top) and tomato (bottom) plant scan. Note that for the tomato plant, we classify individual leaflets (green + yellow + light-blue) as separate instances rather than as an individual leaf. The leaflets can be combined into a single leaf in case this distinction is not desired/required for the application.
Fig 3.
Left: Skeletal matching for an example pair of plant point clouds with the variables involved. Right: Hidden Markov model (HMM) used for correspondence estimation. We only show a subset of the hidden variables, i.e. the potential correspondences, in the HMM. The red line depicts the sequence of best correspondence estimated by the Viterbi algorithm. This produces the correspondences between and
visualized with the dash-lined arrows on the left.
Fig 4.
Left: Registering the skeleton pair involves estimating the deformation parameters attached to the nodes of the source skeleton . Right: Transferring the deformation results to the entire point cloud.
Fig 5.
Semantic classification of maize (top) and tomato (bottom) point clouds.
Each stem and leaf (or leaflet) instance is visualized with a different color. Note that the colors of same leaf instances do not correspond over time, as data associations have not been computed at this stage.
Table 1.
Precision values for class-wise and instance segmentation on our datasets.
Table 2.
Recall values for class-wise and instance segmentation on our datasets.
Table 3.
Intersection over union (IoU) score for our datasets.
Fig 6.
Example ground truth labels used in the evaluation of semantic classification of maize (left) and tomato (right) point clouds.
The instance wise labels for each plant organ have been manually annotated by a human user.
Fig 7.
4D registration of a point cloud pair scanned on consecutive days for maize (top) and tomato (bottom) plant.
The left column shows the two input point clouds () along with their skeletons, with the estimated correspondences between the skeleton nodes shown by dashed lines, and the right column shows the deformed point cloud
(in pink) overlaid on
.
Fig 8.
Visualizing registration error.
We visualize the registration error as a heatmap for two pairs of tomato plant scans, Day 1 vs. Day 2 and Day 6 vs. Day 10. Blue represents low registration error whereas yellow represents a larger error.
Fig 9.
Tracking phenotypic traits for individual organs of the plant.
Our registration procedures allows us to track the growth of the stem and different leave lengths over time and detect topological events such as the emergence of new leaves. Different shades of blue and green in these plots represent individual leaf instances in the first two columns. The orange and red represent the length and diameter of the stem respectively.
Fig 10.
Interpolation of point clouds at intermediate time intervals.
Point clouds (gray) at time t − 1 and t come from actual scan measurements whereas the points clouds (pink) at time instants visualize the three interpolated scans.