Fig 1.
The outline of unsupervised concept discovery framework.
The conceptual examples are provided: convectional, frontal, cyclonic, and orographic precipitations.
Fig 2.
Radar images often contain multiple rainfall systems developed from different mechanisms.
An example image with two precipitation cases: (1) frontal and (2) convective system.
Fig 3.
The heatmap of cosine similarities of the pairs of human-annotated concept labels [11].
Fig 4.
An overview of the proposed framework for unsupervised concept activation vector localization.
(1) the input data is preprocessed by applying the watershed instance segmentation method on the activation vector space of the bottleneck layer of the given trained precipitation forecast model. (2) A masked autoencoder is trained for the self-supervised learning on the previous processed activations to refine a meaningful representation space () from the suboptimal vector space. (3) Pseudo-labels to represent weather patterns are created by performing the proposed multi-label deep clustering on top of the refined representational space, finally obtaining the revised representational space
. (4) Concept vectors (
) are extracted from the final vector space.
Fig 5.
Example instances of 24 clusters.
The date time is in UTC. A. Example instances of Cluster 0: upper-low-jet-coupling. B. Example instances of Cluster 1: upper-low-jet-coupling. C. Example instances of Cluster 2: upper-low-jet-coupling. D. Example instances of Cluster 3: changma. E. Example instances of Cluster 4: upper-low-jet-coupling and east-coast-rainfall. F. Example instances of Cluster 6: changma, typhoon, and convectional. G. Example instances of Cluster 7: typhoon and low-level-jet. H. Example instances of Cluster 8: convectional. I. Example instances of Cluster 10: stationary-front. J. Example instances of Cluster 11: upper-low-jet-coupling.
Fig 6.
Example instances of 24 clusters.
The date time is in UTC. (Cont.). A. Example instances of Cluster 12: typhoon. B. Example instances of Cluster 14: convectional. C. Example instances of Cluster 15: drizzle. D. Example instances of Cluster 16: changma. E. Example instances of Cluster 18: typhoon and north-pacific-high-edge. F. Example instances of Cluster 19: typhoon. G. Example instances of Cluster 20: convectional and stationary-front. H. Example instances of Cluster 21: typhoon. I. Example instances of Cluster 22: changma. J. Example instances of Cluster 23: lake-effect-snowfall.
Table 1.
The Soft Silhouette Coefficient scores for different unsupervised concept extraction methods and vector spaces.
Fig 7.
The t-Distributed Stochastic Neighbor Embedding (t-SNE) [45] representation of the manifold of the embedding vector spaces at each stage of the proposed representation learning procedure: (a) ACE [14] implementation on segmented embedding vectors z, (b) initial clustering of our framework on z.
(c) initial clustering on the self-supervised refined embedding vectors pre. (d) multi-label clustering on its embedding vectors
soft. The same color represents samples within the same cluster. The black markers represent the centroids of individual clusters. The final result showed redundancy in rainfall concepts. We therefore tested for statistically significant clusters, then removed clusters 5, 14, 21, 23, and 28, and merged clusters 1 and 15. Detailed experimental settings are provided in Section C.2 in S1 Appendix for replicability.
Fig 8.
Examples of top three nearest neighbors by Euclidean distance in z and soft: Each example indicates its date and time and corresponding human-annotated labels from relevant open-source data [11]. ‘others’ refers to unclassified labels.
Fig 9.
A comparison of polar low and typhoon cases.
The score refers to the probabilistic score from probe 19. The middle image is from the advanced, very high-resolution radiometer (AVHRR) CH 01, observed by the meteorological operational satellite (METOP-1) on 2021-05-21 at 01:58 (UTC). disentanglement in soft, we use the clusters constructed in this space, which represents the vector space in step (4) of Fig 4, as pseudo labels in subsequent procedures.
Fig 10.
An example of questionnaires and the survey result of the accuracy of detecting homogeneous concepts on annotated labels vs. concepts extracted from the target model.