Mean Absolute Percentage Error (MAPE).

MAPE is a crucial metric particularly suited for assessing the accuracy of models predicting percentages. It calculates the average percentage difference between predicted and actual values. This metric offers a comprehensive understanding of the average magnitude of percentage errors in the predictions [33]. Since we evaluate the predictions after they have been normalized to provide lean mass percentages, the MAPE calculation can be described as the following: (1) Where is the ith predicted value (in kg), is the ith actual value (in kg), w⁽ⁱ⁾ denotes the total weight of the ith sample (in kg) and N denotes the sample size. The average distance between the values provides the final MAPE value.

In our study, MAPE played a significant role in evaluating the accuracy of predictive models for estimating lean mass percentage. By focusing on percentage errors, MAPE provided insights into the relative accuracy of predictions, enabling a more intuitive assessment of model performance in this context.

Coefficient of determination (R²).

R², or the coefficient of determination, measures the proportion of variance in the dependent variable (lean mass percentage) explained by the independent variables within the model. It ranges from 0 to 1, with one indicating a perfect fit. R² is an essential metric to assess the model’s ability to explain variability in the observed data [34], and is defined as: (2) Where is the ith predicted value (in kg), is the ith actual value (in kg), y_mean is the population mean (in kg), and N denotes the sample size.

In our study, R² was pivotal in determining the goodness of fit of predictive models, indicating the strength of the relationship between predictors and lean mass percentage. A higher R² value signified a more robust model with increased explanatory power.

Utilization of metrics in model assessment.

The incorporation of MAPE and R² facilitated a comprehensive evaluation of predictive models developed using NHANES data for estimating lean mass percentage. These metrics collectively provided a multifaceted assessment of model accuracy, precision, and explanatory power. While MAPE gives an insight on the accuracy of the prediction by targeting the distance between the predicted value and the actual value, R² provides an understanding on how much does the model account for the variance of the prediction tasks. Their use guided the identification of the most effective models, enabling the selection of robust models for further analysis and validation.

Anthropometric equations.

Multiple studies have been conducted, focusing on prediction equations to estimate muscle mass using anthropometric data [35]. A recent and well-validated study using this method is the study by DH Lee et al. [16].

In this NHANES-based study, anthropometric equations were formulated and validated for estimating lean body mass. Adult participants underwent DXA measurements to assess body composition, collecting standardized anthropometric data such as height, weight, BMI, waist circumference, and additional measures. Prediction equation development involved multivariable linear regression, revealing that a basic model with sex, age, height, and weight and waist circumference explained most of lean body mass variation. Polynomial terms and interactions provided minimal benefit. Validation using a separate NHANES group demonstrated the equations’ robust predictive accuracy for lean body mass in both sexes, comparable to DXA-measured values. By comparing with additional biomarkers such as serum creatinine levels, the prediction equations provide a more validated approach than simple linear regression. However, the equations developed were not validated on separate datasets.

As a result, we implemented the following equations:

For males: (3)

For females: (4)

Where eth_hisp, eth_black, eth_other are categorical variables representing Hispanic, Black or African-American, and Other/Mixed ethnicities, respectively.

Linear regression.

This model serves as a fundamental tool in predictive analytics, aiming to establish a linear relationship between the input features and the target variable. It assumes a linear association between predictors and the predicted outcome, which in our case, is the lean body mass. We utilized this model as a baseline method due to its simplicity and interpretability. The model fitting involves minimizing the sum of squared differences between the predicted and actual values [36].

Random forest.

This ensemble learning technique constructs numerous decision trees and combines their outputs to generate final predictions. Each tree in the forest is built independently, and the final prediction is derived from an aggregation of individual tree predictions. Random Forest is known for its’ robustness to overfitting and ability to handle complex relationships in the data, making them suitable for predicting lean body mass in our study [37].

We implemented a Random Forest regression model with 100 trees with no limitation on tree depth and used mean squared error as the optimization criteria.

XGBoost.

XGBoost, also known as Extreme Gradient Boosting, is an ensemble learning method that builds decision trees in a step-by-step manner, creating a strong predictive model. It minimizes errors by learning from previous model iterations and placing more emphasis on mispredicted instances. This method enables the effective capture of intricate connections within the data. XGBoost is known for its efficiency, speed, and high predictive performance, making it a valuable choice for lean body mass prediction [38].

We implemented a XGBoost regression model with a max depth of 6, learning rate of 0.3 and used a full L2 regularization term on the weights, with uniform sampling on the training instances.

LassoNET.

Neural networks, serve as a foundational tool in machine learning, capable of discerning patterns and making predictions. LassoNet, a specific neural network framework, enhances this process by integrating feature selection and model fitting into its optimization objective. It incorporates a residual connection from input to output, allowing only features with active connections to participate in hidden layers. A lasso (L1) penalty is strategically employed in the residual layer, promoting sparsity, and thereby selecting the most relevant features [39]. This is achieved by optimizing the following objective function: (5) Where L(θ, W) is a mean-squared error loss function with the following constraint: (6)

Here, θ are the weights of the residual skip layer, W are the weights of the overall network, j represents the feature number, M is the hierarchy parameter controlling the relative strength of the linear and nonlinear components, and λ is the L1-penalty coefficient and is a learned parameter which controls the complexity of the fitted model.

What sets LassoNet apart is its ability to efficiently navigate a regularization path, seamlessly traversing progressive levels of sparsity through proximal gradient methods and warm restarts. This distinctive feature allows LassoNet to adaptively select varying numbers of features while simultaneously fitting flexible nonlinear neural network models. This dynamic and data-driven approach to feature selection and predictive modeling positions LassoNet as an ideal candidate for crafting lean mass prediction equations.

We implemented a LassoNET regression model with 10-fold cross-validation, a hidden layer size of 100, a hierarchy parameter (M) of 10, Adam optimizer, no dropouts, no batches, and early stoppage after 10 epochs with no improvement. Improvement was defined as at least a 1% decrease compared to the previous epoch.

Permutation importance.

To understand the impact of individual features on the predictive model, we analyzed the importance of features using the permutation importance method. Permutation importance is an ML technique used to evaluate the significance of features in predictive models. It involves shuffling the values of each feature independently and measuring the subsequent impact on the model’s performance. The extent to which the model’s accuracy decreases after permuting a specific feature reflects its importance: the greater the decrease, the more crucial the feature is for the model’s predictive capacity. This method aids in identifying the most influential predictors in a model, offering insights into feature relevance, and aiding in feature selection [40]. We used ten permutations (repeats) per model for our analysis.

Total lean body mass.

The evaluation metrics for total lean body mass prediction are visually represented in Fig 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1.

Evaluation Metrics and for Total lean body mass prediction–Evaluation of the prediction task for each of the models on the testing data (NHANES), using R² (A) and MAPE (B) metrics, as well as evaluation on the validation data (Look AHEAD) using R² (C) and MAPE (D) metrics.

https://doi.org/10.1371/journal.pone.0309830.g001

For the NHANES dataset, our models demonstrated a MAPE of [4.22, 3.14, 3.16, 3.02], and an R² of [0.73, 0.85, 0.84, 0.86] for the anthropometric equation, random forest, XGBoost and LassoNet, respectively. These results indicate almost no difference between the models, but already some improvement over the linear regression based anthropometric equation–strengthening the case for more the use of sophisticated models for this task.

For the Look AHEAD dataset, our models achieved a MAPE of [3.93, 3.76, 3.82, 3.49] and an R² of [0.67, 0.69, 0.68, 0.74] for the anthropometric equation, random forest, XGBoost and LassoNet, respectively. We see a drop in performance, as expected since the model was not trained on the Look AHEAD data, which also contains a domain shift. Additionally, while we can see the Random Forest and XGBoost models slightly outperforming the anthropometric equations, LassoNet shows a substantial jump in the prediction task, with an 11% decrease in MAPE, as well as 10% increase in R² when comparing with the equations.

In addition to the MAPE and R² value, we calculated the Root Mean Square Error (RMSE) for each model. For TLM prediction, the RMSE values were [3.38, 3.19, 3.24, 3.09] for the anthropometric equation, Random Forest, XGBoost, and LassoNet models, respectively, when applied to the NHANES testing dataset. On the Look AHEAD validation dataset, the RMSE values were [5.29, 4.59, 4.69, 4.35] for the same models.

Fig 2 shows the actual and predicted values for TLM for the three models: Random Forest, XGBoost and LassoNet, as well as the anthropometric equation to be used as reference. In each subplot, the red line represents the ground truth, which is the ideal results where the predicted values exactly match the actual values. The green line represents the fit line, which is the best linear fit to the data points.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2.

Predicted vs Actual Total Lean Body Mass–prediction of total lean body mass adjusted for weight with testing data (NHANES) for the Anthropometric equation (A), Random Forest (B), XGBoost (C) and LassoNet (D), and the validation data (Look AHEAD) for the Anthropometric equation (E), Random Forest (F), XGBoost (G) and LassoNet (H). The red line depicts a perfect linear fit, while the green line is the model’s actual linear fit.

https://doi.org/10.1371/journal.pone.0309830.g002

The permutation importance plots for TLM prediction (Fig 3) show the importance of each feature in the prediction task. As expected, all models depended heavily on the subject’s total weight to predict their lean mass, with sex being the second most crucial factor. Additional factors that affected the decision for all models are the waist circumference and the height.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3.

Premutation Importance plots for Total Lean Body Mass prediction–Permutation importance plots for each one of the prediction models: Random Forest (A), XGBoost (B), LassoNet (C).

https://doi.org/10.1371/journal.pone.0309830.g003

Age was a minor factor for all the models, and unlike the other models the ethnicity also had a small effect on the prediction for the LassoNet model.

Appendicular lean mass and appendicular skeletal muscle mass.

As part of our analysis, we predicted the ALM and ASMM to evaluate the generalizability of the methods used to predict TLM. Since the anthropometric equations were designed to predict TLM, we used linear regression as a comparison metric instead. As with the TLM prediction, all values were normalized by their corresponding body weights to provide mass percentages. The evaluation metrics for ALM and ASMM prediction are visually represented in Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4.

Evaluation Metrics and for Appendicular Lean Mass and Appendicular Skeletal Muscle Mass prediction–Evaluation of the prediction task for each of the models on the validation data (Look AHEAD) for prediction of Appendicular Lean Mass using R² (A) and MAPE (B) metrics, as well as evaluation on the validation data (Look AHEAD) for prediction of Appendicular Skeletal Muscle Mass using R² (C) and MAPE (D) metrics.

https://doi.org/10.1371/journal.pone.0309830.g004

Similarly to the TLM prediction, the results for the NHANES dataset were exceptionally good for all models, with R² values higher than 0.79 and MAPE values lower than 1.9 for all models with no substantial differences between them. However, for the Look AHEAD dataset our models demonstrated a MAPE of [1.93, 2.03, 2.04, 1.82], and an R² of [0.7, 0.67, 0.66, 0.73] for linear regression, random forest, XGBoost and LassoNet on ALM prediction, and a MAPE of [1.82, 1.88, 1.92, 1.74], and an R² of [0.68, 0.66, 0.65, 0.71] for linear regression, random forest, XGBoost and LassoNet on ASMM prediction. RMSE values were [2.45, 2.5, 2.55, 2.35] kg for the anthropometric equation, Random Forest, XGBoost, and LassoNet models, respectively, for the ALM prediction and [2.32, 2.4, 2.45, 2.24] kg for the same models in the ASMM prediction.

This demonstrates that the prediction methods do not work only for the specific TLM prediction task but can be useful for prediction of other metrics. As before, the LassoNet model performed substantially better than the other prediction methods, but in this case, we see that linear regression actually performs better than the other ML methods. This goes to show the robustness of linear regression, as well as the benefit of using LassoNet, that outperforms all other traditional methods.

The permutation importance plots for ALM and ASMM (Fig 5) show the importance of each feature in the prediction task. Similarly to the TLM prediction, all models depended heavily on the subject’s total weight to predict their lean mass, with sex being the second most crucial factor. Waist circumference played a bigger role in these predictions’ tasks than in the TLM prediction, and height played a minor factor as well.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5.

Premutation Importance plots for Appendicular Lean Mass and Appendicular Skeletal Muscle Mass prediction–Permutation importance plots for each one of the models for ALM prediction: Linear Regression (A) Random Forest (B), XGBoost (C), LassoNet (D), as well as for ASMM prediction: Linear Regression (E) Random Forest (F), XGBoost (G), LassoNet (H).

https://doi.org/10.1371/journal.pone.0309830.g005

However, compared to the TLM prediction, there is more emphasis on ethnicity, with the non-Hispanic Black ethnicity being a factor for all the models. Additionally, the LassoNet model did account for other ethnicities as well, which in turn led to an increase in the model’s accuracy. This was true for both the ALM and ASMM models.