Obtaining raw data

We collected circRNA data of Oryza sativa (rice) and Zea mays (maize) from two public databases: CropCircDB (http://deepbiology.cn/crop/index.php/Home/Html/Download) and PlantcircBase (http://ibi.zju.edu.cn/plantcircbase/download.php). The distribution of environmental condition labels among the 63,048 rice circRNAs obtained from CropCircDB was as follows: 50,187 circRNAs with control label, 5,724 with cold stress label and 824 with drought stress label. For the maize circRNA data, 20,809 circRNAs were labeled as control and 11,206 had the drought stress label. For this work, only circRNA sequences associated with control conditions, cold stress and drought stress were used. Regarding the PlantcircBase data, genomic sequences of 10,381 maize circRNAs and 43,883 rice circRNAs were used. In summary, circRNA sequence data obtained by CropCircDB were used to train and test the models (since this data had environmental condition labels) and the PlantcircBase data were used as the corpus to build the Word2Vec vocabulary.

Feature engineering using Word2Vec approach

The Word2Vec approach was implemented for feature engineering. Word2Vec is a neural network-based word embedding technique known for its ability to map words into a high-dimensional vector space. This method assigns a fixed-length real-value vector V(m)∈ R^m to any word (w) in a dictionary, where V(m) represents the word vector of w and m is the length of the word vector. This technique ensures that words sharing similar semantics are located closer within the vector space. The method takes a text corpus as input and produces vector representations as output. For this work, the Word2Vec technique was applied using the Gensim library (https://pypi.org/project/gensim/). The embeddings were generated using the Continuous Bag-of-Words (CBOW) architecture, with a window size of 5, minimum word frequency of 5, and vector dimensions of 64 and 100. The model was trained for one epoch, and other parameters were kept as default, including negative sampling (negative = 5). The text corpus used to construct the vocabulary consisted of circRNA sequence data from rice and maize obtained from the PlantCircBase database. The Biopython library was used to read the files in FASTA format.

In addition, different k-mer sizes (3-mers, 4-mers and 5-mers) and vector dimensions (64 and 100) were evaluated. The circRNA sequences obtained from CropCircDB were segmented into substrings (k-mers) and then transformed into vectors through trained Word2Vec models. The vector representation process with Word2Vec generates a vector for each identified k-mer with a predefined vector size (for this work, sizes of 64 and 100 were tested). This means that for each k-mer, the Word2Vec model generates a vector of 64 or 100 dimensions. Finaly, the features representation corresponds to the sum of the vectors for all k-mers found in the sequence. This process of generating the datasets was implemented in Python.

Since the data across environmental conditions were imbalanced, the imbalanced-learn library (https://imbalanced-learn.org/stable/) was used. For the development of machine learning models focused on predicting the potential involvement of circRNAs with the cold stress condition, only rice circRNA data were used to train the models. In this scenario, an approach based on RandomUnderSampler was used in control-condition samples, where the final amount of circRNA data from the control condition decreased from 50187 to 8177, to achieve a more balanced dataset with the number of samples labeled with “cold stress” (5724 circRNAs). For the development of machine learning models focused on predicting the potential involvement with the drought stress, maize and rice circRNA data were used to generate features and train the models. Since the number of circRNAs belonging to the control group was much larger than the number of circRNAs labeled with the drought stress condition in this scenario too, it was necessary to use a combined approach of RandomUnderSampler and RandomOverSampler techniques to balance the data. After using this approach, the total amount of circRNA data for training was 34886 (17443 were for the control condition and 17443 for the drought stress condition).

For binary classification (supervised machine learning approach), the target variable for training the classification models was assigned to the “stress” column, where the name of the environmental condition was converted to a numeric variable: 0 for control situation (without any abiotic stress) and 1 for a specific abiotic stress condition (cold stress or drought stress depending on the model).

Eight feature sets were generated for training and testing the models: feature set 1A (containing vector data of size 64 based on k-mers of length 3-mers as input variables and cold stress column as the target variable); feature set 2A (containing vector data of size 100 based on k-mers of length 3-mers as input variables and cold stress column as the target variable); feature set 3A (containing vector data of size 64 based on k-mers of length 4-mers as input variables and cold stress column as the target variable); feature set 4A (containing vector data of size 100 based on k-mers of length 4-mers as input variables and cold stress column as the target variable); feature set 1B (containing vector data of size 64 based on k-mers of length 3-mers as input variables and drought stress column as the target variable); feature set 2B (containing vector data of size 100 based on k-mers of length 3-mers as input variables and drought stress column as the target variable); feature set 3B (containing vector data of size 64 based on k-mers of length 4-mers as input variables and drought stress column as the target variable); feature set 4B (containing vector data of size 100 based on k-mers of length 4-mers as input variables and drought stress column as the target variable).

Machine learning models

Python was used for the development of machine learning models and scikit-learn library (https://scikit-learn.org/) was the main library used for training, testing and evaluating the supervised machine learning models. In addition, for clustering techniques, algorithms implemented in scikit-learn were used, such as: t-SNE (https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html). T-SNE (T-distributed Stochastic Neighbor Embedding) is a statistical method for visualizing high-dimensional data, providing each data point with a location on a two or three-dimensional map, converting similarities between data points into joint probabilities.

For supervised machine learning-based models, the data were processed in the feature engineering stage and subsequently randomly split into training and test sets using the “train_test_split” module of the scikit-learn library. For testing and model evaluation, 80% of the data were used to train the models and 20% of the data were allocated to the test set. In addition, different supervised classification machine learning algorithms were tested.

The metrics used to evaluate the different machine learning algorithms were: accuracy, precision, recall, f1-score, AUC – ROC and AUPRC (area under the precision-recall curve). Accuracy represents the ratio between the number of correct predictions and the total number of predictions. The f1-score metric corresponds to the harmonic mean of precision (a measure of the number of correctly identified positive cases out of all predicted positive cases) and recall (a measure of the number of correctly identified positive cases out of all real positive cases). The AUC – ROC curve is a performance measure for classification problems at various threshold settings. ROC is a probability curve, and AUC represents the degree or measure of separability. It indicates how well the model can distinguish between classes. The higher the AUC, the better the model is at predicting classes 0 from class 1. AUPRC summarizes the trade-off between Precision (Positive Predictive Value) and Recall (Sensitivity) by calculating the integral of the Precision-Recall curve, providing a more robust evaluation in scenarios involving significant class imbalance. DummyClassifier algorithm was used as a comparison algorithm, because this algorithm makes predictions that ignore the input features. In this sense, it serves as a simple baseline to compare with other more complex classifiers. The other models should outperform DummyClassifier, to conclude that the model was learning the underlying patterns, and not just making random predictions. Finally, the best models (the models that presented the best performance) were saved using the joblib library (https://joblib.readthedocs.io/en/stable/generated/joblib.dump.html). To ensure a fair and reproducible benchmark, all machine learning models, including the Random Forest classifier, were implemented using their default hyperparameter configurations as provided by the Scikit-Learn (v1.3.2) library. Specifically, the Random Forest was grown with 100 decision trees based on the Gini impurity criterion.

Approximate nearest neighbors analysis

To better understand the similarity between circRNA sequences in different biological conditions and which patterns the algorithms were learning, a nearest neighbor search analysis was performed using Python and the Annoy (Approximate Nearest Neighbors Oh Yeah) technique. Annoy (https://github.com/spotify/annoy) is a library that allows efficient identification of nearest neighbors in large datasets, particularly in applications involving vectors, such as machine learning and information retrieval. With this technique, multiple search trees are constructed, where each tree is based on a random sample of data. The idea is to partition the space into regions, allowing for faster neighbor searches. In this sense, an analysis using Annoy library was performed to search for the 5 nearest neighbors of circRNA sequences from the group linked to the abiotic stress situation. To perform this analysis, the following datasets were generated:

• Word2vec vector data of 22412 maize circRNAs were divided equally to generate two groups: a “control” group containing 11206 circRNAs linked to the control condition and 11206 circRNAs linked to the drought stress condition.
• Word2vec vector data of 11448 rice circRNAs were equally divided to generate two groups: “control” group containing 5724 circRNAs linked to the control condition and 5724 circRNAs linked to the cold stress condition.

After that, a Python script was developed using the Annoy library in which each circRNA belonging to a specific stress condition was sequentially analyzed and compared to sequences from control and stress conditions. The aim was to compare circRNA sequences belonging to a specific abiotic stress condition and determine whether greater similarity existed with sequences from the stress group or if there were more similarities with sequences from the control group. For each stress sequence, the script searched for the 5 closest neighbors. Self-matches were ignored, ensuring that only unique neighbors were counted. To generate this analysis, maize and rice circRNA vector data generated by Word2Vec were added to an index, where each vector represents a numerical representation of an item. The Annoy library then builds several search trees, where each tree is built based on a random sample of the data. When querying the data, Annoy analyzes the trees to find the closest neighbors using approximation.

k-mer density analysis

To complement the Approximate Nearest Neighbors analysis, further characterize the sequence signature of stress-responsive circRNAs, and improve the interpretability of the machine learning models, a k-mer density analysis was performed. Genomic sequences of all circRNAs available in the dataset (control, cold stress, and drought stress groups) were decomposed into 3-mer units.

To mitigate bias caused by variation in sequence length, k-mer counts were normalized by their density per 1,000 nucleotides. Density was calculated according to the following formula:

Where:

• C_i,j represents the absolute count of the 3-mer i in sequence j;
• L_j represents the length of sequence j in nucleotides;
• 1000 is a scaling factor used to express the density as occurrences per kilobase (kb).

To identify changes in genomic signatures between experimental conditions, the density delta (ΔD) was calculated. This value represents the arithmetic difference between the mean densities of the stress and control groups. A positive delta (Δ) indicates that the motif is enriched under stress conditions, while a negative value indicates enrichment in the control condition.

Additionally, the proportional shift in motif density between stress and control groups was evaluated using Fold-change (FC) and Log2 Fold-change (Log2FC) analyses. A small constant (ϵ = 0.01) was added as a pseudocount to both the numerator and denominator to ensure numerical stability and avoid division by zero in motifs with very low density.

The formulas used were:

Where and represent the mean density of motif iii in the stress and control experimental groups, respectively.

These analyses allowed the identification of sequence motifs that are preferentially enriched or depleted under abiotic stress conditions, providing additional biological insight into the sequence patterns captured by the machine learning models.

circ-EnviroPredict tool development

The circ-EnviroPredict bioinformatics tool was developed using the Streamlit library (https://streamlit.io/). The Streamlit library is capable of transforming Python scripts into web applications. To develop the application, the best-performing models trained to predict involvement with cold and drought stress were chosen and incorporated into the reference file (app.py). Additionally, checks and tests were implemented, such as, to check if the input sequence is valid and unit tests.

Supervised approach – Evaluation of cold stress models

Using k-mer (k = 3, 4 and 5) vector representations of rice circRNAs sequences, different supervised classification algorithms were evaluated to develop prediction models for circRNAs involvement in cold stress conditions. Although k-mer lengths up to k = 5 were evaluated, no performance improvement was observed beyond k = 4, as shown in S1 Table. In addition, increasing k-mer length resulted in longer processing and training times due to the higher dimensionality of the feature space. Therefore, subsequent analyses were focused on k = 3 and k = 4, with k = 3 selected for the final models due to its favorable balance between predictive performance and computational efficiency. The models were trained with four sets of features: Sets 1A, 2A, 3A and 4A (described in Materials and Methods section). The models were evaluated using an independent test dataset, and the top three models are described in Table 1. A DummyClassifier was used as a baseline model for comparison.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Evaluation of machine learning algorithms used to develop models focused on predicting the involvement of circRNAs and cold stress.

https://doi.org/10.1371/journal.pone.0350943.t001

In general, tree-based algorithms achieved the best performance metrics (Table 1), particularly RandomForestClassifier, LGBMClassifier and ExtraTreesClassifier. When comparing models trained with vectors generated from 3-mers and 4-mers, the metrics were very similar, but in some cases the 3-mers models presented better metrics. Furthermore, increasing the number of features (64 versus 100) does not seem to have any significant impact. To understand the patterns of success in cold stress involvement classification models, a confusion matrix was generated for models trained using LGBMClassifier and Random Forest using the Word2Vec vectors generated from k-mers of 3-mers and vector size 64 (the models that achieved the best performance metrics) (Fig 1).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Confusion matrix generated from cold stress models using Word2Vec data (3-mers and vector size 64).

(A) LGBMClassifier. (B) Random Forest.

https://doi.org/10.1371/journal.pone.0350943.g001

As can be seen in Fig 1, the LGBMClassifier and Random Forest algorithms produced very similar prediction patterns. These models produced errors in approximately 15% of the cases where the model predicted the control condition for a sequence that actually belonged to the cold stress condition. Approximately 10% of errors occurred when the model predicted a cold stress condition for the sequence but in fact that sequence belonged to the control group. One of the differences between these two algorithms is that Random Forest builds multiple decision trees during training and combines their outputs through voting or averaging, while LightGBM is a boosting algorithm that builds decision trees sequentially, where each new tree corrects the errors of the previous one.

To complement the generation of model evaluation metrics, an AU-ROC curve analysis was also performed with Random Forest and LGBM models trained with the same feature set (Fig 2), where ROC curve represents the relationship between the true positive rate and the false positive rate, and AUC represents the degree or measure of separability. It indicates how well the model can distinguish between classes.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. AU-ROC curve for evaluating the performance of cold stress models.

(A) LGBMClassifier. (B) Random Forest.

https://doi.org/10.1371/journal.pone.0350943.g002

In Fig 2, we can see that both models had AU-ROC values equal to or above 0.80, but Random Forest has a slight advantage (0.82) in terms of discrimination between classes. This means that Random Forest has a slightly better rate of identifying positive examples without generating too many false positives. In addition to AU-ROC, we also evaluated model performance using AUPRC, which is particularly informative for imbalanced datasets. The results showed that Random Forest consistently outperformed LightGBM. Random Forest achieved an AUPRC of 0.7574, while LightGBM reached 0.7368. This result further supports the robustness of the Random Forest model, especially in capturing true positive predictions under class imbalance scenarios.

In addition, a cross-validation analysis was also performed to evaluate machine learning models from the Word2Vec 3-mers set. Cross-validation helps assess model generalization, that is, to identify whether a model performs well on a training set but poorly on a validation set, which can lead to overfitting. The 4-fold cross-validation analysis was generated for the Random Forest and LightGBM algorithm models, using the K-Fold Cross-Validation approach, where the scores obtained were:

• LightGBM: Scores – [0.7013809, 0.70589928, 0.70359712, 0.72143885]. Average of 0.71.
• Random Forest: Scores – [0.71375144, 0.70935252, 0.70273381, 0.71856115]. Average of 0.71.

The prediction accuracy of the independent test set was similar to that of the cross-validation, which showed that the developed models were not strongly affected by overfitting or underfitting.

Supervised approach – Evaluation of drought stress models

To improve model performance, the drought stress prediction models were trained using circRNA data from both rice and maize, unlike the approach focused on cold stress, which used only rice circRNA data. The difference here is that there are samples labeled with drought stress condition in both plant species. Thus, new datasets were created by combining rice and maize circRNA data. The advantage of combining data from different plant species, besides providing more data for the model, is the possibility of transferring learned patterns to new plant species and being able to perform the prediction using circRNA data from any species as input. The models obtained from classification algorithms (trained with feature sets 1B, 2B, 3B and 4B – described in Materials and Methods), focus on the prediction of circRNA involvement with drought stress are described in Table 2.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Evaluation of machine learning algorithms used to develop models focused on predicting the involvement of circRNAs and drought stress.

https://doi.org/10.1371/journal.pone.0350943.t002

As can be seen in Table 2, drought stress models combining rice and maize circRNA sequence data in the training stage had a significant increase in metrics when compared to cold stress models (Table 1). In all feature sets, it was possible to obtain accuracy and f1-score values around 80%. This demonstrates that the Word2Vec approach can be a good proposal for a methodology for prediction using only circRNA sequence data and that the increase in the amount of data contributed to a better understanding of the patterns by the algorithms. In addition, once again, the classification algorithms with the best evaluation metrics were LGBMClassifier and RandomForestClassifier, confirming that tree-based models are apparently the best choice for this type of application.

Another point is that models trained with vector data based on 4-mers presented similar metrics compared to models trained with 3-mers, but the training time of the models was much longer when using the feature set of 4-mer and vector size of 100. Therefore, aiming at optimizing machine processing capacity and execution speed of the data preprocessing and model prediction steps, especially if the model receives more data from other plant species in the future, the set of features generated from 3-mer approach with 64 features continues to be the best option, as previously observed in cold stress models.

To understand where models focused on drought stress were making right and wrong predictions, a confusion matrix was generated using the LightGBM and Random Forest algorithms and the 1B feature set, as shown in Fig 3.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Confusion matrix generated from drought stress models using Word2Vec data (3-mers and vector size 64).

(A) LGBMClassifier. (B) Random Forest.

https://doi.org/10.1371/journal.pone.0350943.g003

We can observe in Fig 3 that only ~900 circRNAs out of a total of 4704 in the test sample were incorrectly predicted in both the LGBMClassifier and Random Forest algorithms. This information confirms that data increment can help improve the accuracy and sensitivity of the model. We also repeated the AU-ROC curve analysis to complement model evaluation metrics for the context of drought stress models. In Fig 4, we can see that Random Forest-based models presented relatively high AU-ROC values (0.88) and again were higher than the values found in the LGBM (0.86), as was the case in the evaluation of cold stress models. Similarly, Random Forest consistently outperformed LightGBM in the AUPRC analysis. Random Forest obtained an AUPRC of 0.8151 compared to 0.8039 for LightGBM.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. AU-ROC curve for evaluating the performance of drought stress models.

(A) LGBMClassifier. (B) Random Forest.

https://doi.org/10.1371/journal.pone.0350943.g004

Finally, a cross-validation analysis was performed to evaluate drought stress models using the Word2Vec 3-mers set. The same 4-fold cross-validation approach was used for the Random Forest and LightGBM algorithm models. The scores obtained were:

• LightGBM: Scores – [0.72593204, 0.80840019, 0.80908555, 0.77309735]. Average of 0.78.
• Random Forest: Scores – [0.72581406, 0.79270882, 0.8020059, 0.75846608]. Average of 0.77.

In the drought stress prediction models, the prediction accuracy of the independent test set was also similar to the score values obtained from cross-validation.

Unsupervised approach

In order to obtain a deeper understanding of the sequence patterns of circRNAs for predicting abiotic stress using the Word2Vec vector approach, models were also developed using unsupervised machine learning techniques. In unsupervised machine learning, the learning process aims to identify patterns within the data in order to group or organize them based on their similarities. Since we do not have labeled data in this methodology, there is no need to divide the dataset into training and test sets. In this sense, the Word2Vec vector data (based on k-mers of length 3-mers and vector size of 64) were initially grouped into 3 components using the t-SNE method, the components were colored according to the “drought stress” label and the result generated a 3D representation available in Fig 5.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. 3D representation of t-SNE and components generated from Word2Vec vector compositions of circRNAs.

Color range based on stress column labels (0 – control and 1 – drought stress).

https://doi.org/10.1371/journal.pone.0350943.g005

In Fig 5, we can see that it was possible to segment circRNAs into 2 distinct groups according to their similarity of vector values grouped by t-SNE, even though there was some overlap in colors between the groups. This demonstrates that it is possible to predict potential involvement of circRNAs with drought stress based on their biological sequence structure and vector values. CircRNAs involved in drought stress appear to share common characteristics and patterns, which is why it was possible to cluster these data via t-SNE.

In order to understand whether this behavior also applied to circRNAs belonging to the group linked to cold stress conditions, the same technique of grouping vector values into 3 components was applied using the t-SNE method based on circRNA data from the control and cold stress conditions (Fig 6).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. 3D representation of t-SNE and components generated from Word2Vec vector compositions of circRNAs.

Color range based on stress column labels (0 – control and 1 – cold stress).

https://doi.org/10.1371/journal.pone.0350943.g006

Despite having an isolated group of circRNAs with similar characteristics belonging to the control group in the 3D representation of the components, in Fig 6 we can notice that there is more color overlap in the cold stress group than compared to what was observed in Fig 5 of the analysis of drought stress circRNAs. Perhaps because circRNAs labeled with the cold stress label share more similar patterns with circRNAs from the control condition, we can assume that this may explain the lower metrics in the binary classification models of potential involvement of circRNAs in cold stress conditions compared to the drought stress prediction models.

Approximate nearest neighbors analysis

To understand the similarity between circRNA sequences in different biological conditions and which patterns the algorithms were learning, a nearest neighbor search was performed using Python and the Annoy algorithm. In this sense, 5724 circRNA sequences belonging to the cold stress group were sequentially analyzed and compared with sequences from cold stress and control condition and 12030 circRNA sequences belonging to the drought stress group were also analyzed and compared with sequences from drought stress and control groups to find the 5 nearest neighbors. The result is shown in Table 3.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Five nearest neighbor analysis of stress-related circRNA sequences.

https://doi.org/10.1371/journal.pone.0350943.t003

These results demonstrate that in rice circRNA samples with cold stress involvement, most of the sequences of close neighbors found are also related to the cold stress group (14617 of 28620 neighbors identified – 51.07%). This suggests that the sequences of circRNAs under cold stress have a high similarity to each other, indicating possible common patterns. It is also possible to observe that the same pattern was repeated in the drought stress analysis, where circRNA sequences involved with the drought stress condition also found more similarities and neighbors that were also present in the drought stress group. A total of 73.3% of the total of neighbors were identified in circRNA sequences belonging to the drought stress group, indicating a denser network of similarity in the structure of biological sequences in this condition.

k-mer density analysis

To further investigate whether specific sequence patterns could explain the predictive performance of the machine learning models, a k-mer density analysis was performed comparing circRNAs from control and stress conditions. The results revealed clear differences in the distribution of 3-mer motifs between control and abiotic stress groups (cold and drought), indicating that stress-responsive circRNAs exhibit distinct sequence signatures (Fig 7).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. k-mer density analysis of circRNA sequences under abiotic stress conditions.

Density distribution of 3-mer motifs calculated from circRNA sequences comparing the combined abiotic stress group (cold and drought) with the control condition. k-mer counts were normalized by sequence length and expressed as occurrences per 1,000 nucleotides (kb).

https://doi.org/10.1371/journal.pone.0350943.g007

The density analysis showed that several motifs were consistently enriched in circRNAs associated with abiotic stress conditions when compared to the control group. This enrichment became even more evident when the proportional differences between groups were evaluated using Log2 Fold-Change analysis (Fig 8). In particular, multiple motifs with higher adenine–thymine (AT) content presented positive Log2FC values, indicating that these motifs occur more frequently in circRNAs expressed under stress conditions.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 8. k-mer enrichment analysis based on Log2 Fold-Change.

Log2 Fold-Change analysis of 3-mer motif density between stress (cold + drought) and control groups. Positive Log2FC values indicate motifs enriched in circRNAs associated with abiotic stress, whereas negative values represent motifs more frequent in the control condition.

https://doi.org/10.1371/journal.pone.0350943.g008

A similar pattern was observed when cold and drought stress conditions were analyzed independently (S1-S4 Fig). In both scenarios, stress-associated circRNAs displayed enrichment of AT-rich 3-mer motifs relative to the control group, reinforcing the existence of characteristic sequence signatures associated with abiotic stress responses.

To further explore whether these patterns were conserved across species, additional analyses were performed separately using rice and maize circRNA datasets subjected to drought stress (S5 and S6 Figs). Although the ranking of the most enriched k-mers differed between the two species, indicating a certain degree of species-specific variability, the overall enrichment of AT-rich motifs remained consistent. Interestingly, the occurrence of these motifs was substantially higher in rice circRNAs compared to maize. For example, the 3-mer motif TTT presented a density greater than 10 occurrences per 1,000 nucleotides in rice, whereas in maize its density was slightly above 2 occurrences per 1,000 nucleotides.

Validation using biological sequences from different plant species

To develop the circ-EnviroPredict tool, models that presented the best metrics and trained with features from Word2Vec data generated from k-mers 3-mers and vector size 64 were used. In addition, the models available in the tool were those trained with the approach based on the RandomForestClassifier algorithm. Although the metrics between the LGBMClassifier and RandomForestClassifier algorithms were very similar in the different test sets for the different conditions tested (drought and cold stress), the Random Forest-based models were more accurate in the sequences that were involved in the control condition, as demonstrated in the confusion matrices generated with the data from the test set. As in most cases, a circRNA ends up being identified and not having biological validation that it is involved with some specific stress condition, as it was possible to observe in the differences in quantities between the control set and abiotic stress sets obtained in CropCircDB, the control set being more accurate ends up being closer to the real scenarios. Therefore, leaving only the validation using the test set and validating the models with sequences obtained in other databases and articles, the models based on Random Forest made fewer mistakes, mainly in the control condition, which is why this approach was selected for implementation in the final tool.

Because machine learning models learn patterns from the available data and apply them to unseen data, one of the hypotheses of this work is that machine learning models could be able to help predict the role of circRNAs in abiotic stress conditions for different plant species (in addition to the species used in training), since circRNAs are highly conserved molecules. In this sense, the models were also validated using circRNA sequence data from the following species: Arabidopsis thaliana, Glycine max and Triticum aestivum. To perform this test, genomic sequence data, splice junctions and divergent primers of circRNAs were obtained from PlantcircBase and from four articles on circRNA identification under different environmental conditions [6,11,15,26]. Finally, these sequences were inserted into the circ-EnviroPredict tool for prediction and validation of the models (Table 4).

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Validation of genomic sequences, splice junctions and primers of circRNAs from Arabidpsis thaliana, Glycine max and Triticum aestivum using the circ-EnviroPredict predictor tool.

https://doi.org/10.1371/journal.pone.0350943.t004

The results obtained from all tests using data from different plant species (including mono and dicotyledonous species) are shown in Table 4, demonstrating that the tool was able to transpose the patterns of vector values learned via machine learning with rice and maize circRNAs to other plant species and correctly predict the different environmental conditions in which these circRNAs were in fact involved. The probability values for each sequence are available in the “circ-EnviroPredict probability” column. In addition, it was also possible to make the prediction using different types of circRNA sequences, including genomic sequences, splice junction sequences and even divergent primer sequences used to biologically validate circRNAs in the articles. This ability to transfer patterns learned via machine learning to new species may be related to the fact that circRNAs are widely distributed and conserved molecules among different plant species [4].