^{1}

^{2}

^{3}

^{3}

^{4}

^{1}

The authors have declared that no competing interests exist.

The most frequently used approach for protein structure prediction is currently homology modeling. The 3D model building phase of this methodology is critical for obtaining an accurate and biologically useful prediction. The most widely employed tool to perform this task is MODELLER. This program implements the “modeling by satisfaction of spatial restraints” strategy and its core algorithm has not been altered significantly since the early 1990s. In this work, we have explored the idea of modifying MODELLER with two effective, yet computationally light strategies to improve its 3D modeling performance. Firstly, we have investigated how the level of accuracy in the estimation of structural variability between a target protein and its templates in the form of

Proteins are fundamental biological molecules that carry out countless activities in living beings. Since the function of proteins is dictated by their three-dimensional atomic structures, acquiring structural details of proteins provides deep insights into their function. Currently, the most frequently used computational approach for protein structure prediction is template-based modeling. In this approach, a target protein is modeled using the experimentally-derived structural information of a template protein assumed to have a similar structure to the target. MODELLER is the most frequently used program for template-based 3D model building. Despite its success, its predictions are not always accurate enough to be useful in Biomedical Research. Here, we show that it is possible to greatly increase the performance of MODELLER by modifying two aspects of its algorithm. First, we demonstrate that providing the program with accurate estimations of local target-template structural divergence greatly increases the quality of its predictions. Additionally, we show that modifying MODELLER’s scoring function with statistical potential energetic terms also helps to improve modeling quality. This work will be useful in future research, since it reports practical strategies to improve the performance of this core tool in Structural Bioinformatics.

Homology modeling (HM) is a fast and reliable TBM method in which a target protein is modeled by using as a structural template an homologous protein. HM predictions usually consist of three phases. In the first, the sequence of the target is used to search for suitable templates in the PDB [

The overall accuracy of HM has remarkably increased in the last 25 years. While a major factor for this advancement has been the increase of the size of sequence and structural databases [

MODELLER [

The core MODELLER algorithm was developed in the early 1990s and it was essentially left unchanged over the years. Despite its importance, there have been relatively few attempts to improve it.

In 2015, Meier and Söding designed a novel probabilistic framework for building HDDRs [

Researchers from Lee’s group developed a modified version of MODELLER which they have been using in CASP experiments [

Although these seminal studies have shown that the core MODELLER algorithm has room for improvement, most of its users employ its original version, probably because existing modifications either depend on additional packages to install, or are computationally too expensive (e.g., the CSA algorithm alone was reported to increase computational times by a factor of ~130). Since MODELLER is a core tool in Structural Bioinformatics, it is of paramount importance to investigate in detail the inner working of its algorithm and to develop it further. Here, we have explored two computationally light strategies to improve it in terms of 3D modeling quality.

Particular attention has been dedicated in understanding how the level of accuracy in the estimation of structural variability between the target and templates expressed as

To rigorously validate these approaches, we have benchmarked them using protein targets from a diverse set of high-resolution structures from the PDB and we quantified the individual impact on 3D modeling of each modification. This information will be useful in future research, since it shows in which areas there is still room for improvement and in which areas it might be difficult to advance further.

The MODELLER approach relies on the generation of HDSRs for interatomic distances and dihedral angles [

For a couple of atoms _{t}. The distance _{m} between _{t} with a standard deviation

In MODELLER _{MOD} values) through an histogram-based approach [

MODELLER allows to take advantage of multiple templates, a strategy that (when templates are chosen adequately) usually outperforms single-template modeling [_{m}, MODELLER uses the following _{u} is a template-specific weight, _{t,u} and _{u} are the distance observed in template _{u} is a function of the local sequence similarity between the target and template

The total objective function of MODELLER (_{TOT}) can be expressed as follows:
_{PHYS} contains five physical terms (see _{HOM} contains HDSRs terms. In this work, the weights for _{PHYS} and _{HOM} were always left to 1.0 (therefore they are omitted from the formula above).

In order to benchmark modifications of MODELLER, we built an analysis set of selected target proteins. We obtained 926 X-ray structure chains from PISCES [

the maximum mutual sequence identity (SeqId) among the chains was 10%;

their structures had a resolution < 2.0 Å and R-factor < 0.25;

they contained no missing residues due to lacking electron density;

their length was between 70 and 700 residues.

These chains were our target candidates. To obtain their templates, we culled from PISCES another set using similar filters, except that this time the maximum mutual SeqId was 90%. We removed from this larger set all the targets, obtaining 6224 chains. Each target was then aligned to these chains using TM-align [

the SeqId in the structural alignment built by TM-align was between 15% and 95%;

the two TM-scores [

We retained for each target only its top five templates in terms of TM-score (normalized on the target length). In this way, we obtained a final set of 225 target chains (suitable templates could not be found for 701 targets, a result of using only high-resolution template structures). For each target, we performed single-template modeling only with its top template and therefore we had 225 single-template models, which constituted the Analysis Single-template (AS) set. 118 targets had at least two templates (with an average of 3.3), which constituted the Analysis Multiple-templates (AM) set.

The average SeqId for the AS target-template alignments is 0.38. Improving the performance of MODELLER with targets having templates with a SeqId < 0.40 is important, because these cases are the most frequent ones in Biomedical Research [

In order to align target-template pairs we employed the accurate HHalign program [

Whenever specified, we also employed target-template alignments built with TM-align in order to assess the effect on 3D modeling of HDDRs derived from error-free structural alignments.

For all benchmarks we used MODELLER version 9.21. In order to modify its objective function terms and optimization schedules we interfaced with its Python API. To modify the restraints parameters we employed Python scripts to edit the default restraints files generated by the program (see the “Restraints files building” section).

In MODELLER, the final quality of a model is largely determined in the MDSA phase. In this work, unless otherwise stated, we employed the default

The approach used to evaluate the quality of an homology model was to build 16 different copies of it (hereinafter defined as decoys), and to report as an overall quality score (see below) the average score of the 16 decoys.

To evaluate the quality of the backbones we used the GDT-HA metric [

_{m} between atoms _{t} is the distance between the template atoms equivalent to _{n} is the distance between _{n}_{m} is equal to its corresponding _{n} (see

In the case of multiple-template HDDRs, we demonstrate that the combination of optimal _{n}

Whenever using _{n}_{n}_{n}

Distributions of the _{n}_{MOD} (B) values observed in the AS models for the four HDDR groups of MODELLER. Beside the names of the restraints groups, their mean values are reported.

In MODELLER, the restraints used in the 3D model building phase are supplied in a specific file. In this work, we explored how the choice of parameters for the HDDRs influences 3D modeling. Therefore, our approach for building restraints files was to let MODELLER generate its default restraints files and then to modify it by leaving unaltered all the stereochemical and homology-derived dihedral angle restraints and by modifying only the HDDRs parameters. The Python code we used to customize restraints files is available at

To understand the effect of using error-containing _{n}_{n}_{n}_{n}_{SEL}), we added to the _{n} values of the pair a list _{i} is the _{i} is a random error extracted from a Laplace distribution with location 0 and scale parameter _{i} values being distributed approximately as exponentials, which resemble the original _{n}_{SEL}, a _{n}_{n}_{obs} to 25.0*_{obs}, where _{obs} is the mean of the _{n}_{SEL} was selected. The _{n}_{SEL} (thus giving practically the same level of perturbation of _{SEL}). Note that larger _{n}_{pt}) would be equal to:
_{grp} is the mean _{n}_{SEL} is near 1, the mean of a _{n}_{SEL} is near 0, its mean tends to the “global” mean observed for the corresponding group in our whole _{n}_{n}_{n}_{n}_{n}

Average GDT-HA (A) and lDDT (B) scores of the AS models as a function of the uniform _{MOD} values.

To simulate various levels of accuracy in _{n}_{SEL} values (linearly spacing from 0.0 to 0.9). For each _{SEL}, we generated 5 sets of perturbed _{n}_{SEL} value). For a certain _{SEL} value, the quality score for a 3D model was recorded as the average score of all its 40 decoys.

To quantify in terms of PCC the actual amount of perturbation introduced in the _{n}_{MODEL}. This score is computed as:
_{R} is the number of perturbed _{n}_{u,r} indicates the observed PCC between the list of _{n}_{SEL} values and the average _{MODEL} observed in the AS and AM sets is almost perfectly linear (see

In this work, we explored the effect of including in the objective function of MODELLER terms for interatomic distance statistical potentials. These potentials are developed with the aim of recognizing native-like protein conformations [

We employed the DOPE potential [

The Lee group previously included the DFIRE [

When including statistical potential terms, the MODELLER objective function becomes:
_{SP} contains the statistical potentials terms and _{SP} is their weight. For obtaining best 3D modeling results, we tested several values of _{SP}.

We employed statistical potentials using a contact shell value of 8.0 Å. Higher values can be safely avoided because the terms of DOPE and DFIRE start to acquire a flat shape over the 8.0 Å threshold (see

Gaussian HDDRs are the heart of the MODELLER approach. At first, we explored how the use of optimal _{n}_{n}_{MOD} values. An improvement is also observed for local all-atom quality, as the average lDDT score increases by 4.2%. Increments in GDT-HA and lDDT are seen for 224/225 and 225/225 AS models respectively (see

(A) and (B) GDT-HA and lDDT scores of the AS models built with _{MOD} (reported on the x-axis) and with optimal _{n}

Strategy | GDT-HA | lDDT | MolProbity score |
---|---|---|---|

MODELLER^{a} |
0.6014 (-) | 0.6563 (-) | 3.0104 (-) |

OPTIMAL^{b} |
0.6377 (+6.0%) |
0.6842 (+4.2%) |
3.0311 (+0.7%) |

MODELLER-SLOW^{c} |
0.6036 (+0.4%) |
0.6594 (+0.5%) |
2.8512 (-5.3%) |

OPTIMAL-SLOW | 0.6377 (+6.0%) |
0.6853 (+4.4%) |
2.9039 (-3.5%) |

MODELLER-TMalign^{d} |
0.6383 (+6.1%) |
0.6951 (+5.9%) |
3.0411 (+1.0%) |

OPTIMAL-TMalign | 0.6805 (+13.2%) |
0.7259 (+10.6%) |
3.0870 (+2.5%) |

The “GDT-HA”, “lDDT” and “MolProbity score” columns report the average values for those metrics. Percent improvements are computed with respect to the scores of the default MODELLER (first row).

^{a}The “MODELLER” prefix indicates that the strategy employs HDDRs generated by MODELLER.

^{b}The “OPTIMAL” prefix indicates the use of optimal HDDRs.

^{c}The “SLOW” suffix indicates the use of the

^{d}The “TMalign” prefix indicates the use of target-template alignment built through TM-align.

*Asterisks denote a statistically significant difference (according to a Wilcoxon signed-rank test with a significance level of 0.05) between the scores of a strategy and the scores of the default MODELLER. See

Increasing target-template alignment quality is one of the current challenges in TBM. In our AS models, the average accuracy of HHalign sequence alignments with respect to error-free TM-align structural alignments is 0.87 (see _{MOD} values and TM-align alignments, the average GDT-HA and lDDT scores improve by 6.1% and 5.9% respectively over the scores obtained with _{MOD} values and HHalign alignments (see

It might be thought that _{n}

Next, we explored the effect of optimal HDDRs in multiple-template modeling, which has never been assessed before. As shown in

Strategy | GDT-HA | lDDT | MolProbity score |
---|---|---|---|

MODELLER | 0.6287 (-) | 0.6819 (-) | 3.0725 (-) |

OPTIMAL | 0.8733 (+38.9%) |
0.8106 (+18.9%) |
3.1478 (+2.4%) |

MODELLER-SLOW | 0.6310 (+0.4%) |
0.6850 (+0.5%) |
2.9143 (-5.2%) |

OPTIMAL-SLOW | 0.8747 (+39.1%) |
0.8133 (+19.3%) |
3.0475 (-0.8%) |

OPTIMAL-U^{a} |
0.7438 (+18.3%) |
0.7427 (+8.9%) |
3.1744 (+3.3%) |

MODELLER-ST^{b} |
0.6168 (-1.9%) |
0.6683 (-2.0%) |
3.0231 (-1.6%) |

OPTIMAL-ST | 0.6557 (+4.3%) |
0.6986 (+2.5%) |
3.0398 (-1.1%) |

MODELLER-TMalign | 0.6645 (+5.7%) |
0.7165 (+5.1%) |
3.0529 (-0.6%) |

OPTIMAL-TMalign | 0.9222 (+46.7%) |
0.8498 (+24.6%) |
3.1044 (+1.0%) |

See

^{a}The “U” suffix indicates the use of uniform template weights for multiple-template HDDRs.

^{b}The “ST” suffix indicates that only the top template for each target was used (thus resulting in single-template modeling).

*Asterisks denote a statistically significant difference (according to a Wilcoxon signed-rank test with a significance level of 0.05) between the scores of a strategy and the scores of the default MODELLER. See

Optimal HDDRs increase even more the beneficial effect of using multiple templates. With MODELLER-generated restraints, employing multiple templates leads to an improvement of 1.9% and 2.0% in the average GDT-HA and lDDT of the AM models over single-template modeling performed with top-templates (see the MODELLER-ST strategy in

The reason for this large improvement is the following. In MODELLER, the _{n}_{t} as close as possible to the target distance _{n}, that is, the template with lowest _{n}

The importance of the template-weighting scheme [_{n}_{u} =

Our data shows that if the best template can be identified for each restrained distance, a substantial improvement in 3D modeling quality can be reached. A relevant matter is therefore to understand whether for a single residue (on which several HDDRs are usually acting) or for some stretch of contiguous residues, the best template always happens to be the same, or instead if multiple templates are effectively used together. _{n}

In both single and multiple-template modeling, the use of optimal HDDRs appears to decrease the stereochemical quality of models, as seen by increased MolProbity scores (see

As first demonstrated in [_{MOD} values are weakly correlated with their optimal counterparts. In the AS models, the distributions of _{n}_{MOD} values are markedly different (see _{n}

(A) Distributions for the PCCs between _{MOD} and _{n}

In the previous section we have seen that the use of optimal _{n}_{n}_{n}

_{n}_{MODEL} of the AS models is approximately 0.9, the average GDT-HA decreases by 2.6%. Further increasing the amount of random perturbation in _{MODEL} approximates 0, the average GDT-HA is 0.6056 (resulting in a 5.0% decrease with respect to the optimal state). This score is 0.8% higher than the average GDT-HA obtained using the default _{MOD} values, which is 0.6009. Although the difference between these two scores is statistically significant (Wilcoxon signed-rank test, p-value = 1.6e-5) it is only minimal from a structural point of view. In other words, in single-template modeling, provided that the average _{n}

(A) and (B) Average GDT-HA and lDDT scores of the AS models as a function of their average Cα-Cα _{MODEL} values (see the “

Next, we performed perturbation experiments with multiple-template models (see _{MODEL} approximates 0, the average GDT-HA now becomes 9.1% lower than the one obtained using the default MODELLER. This behavior is likely to be explained by the fact that in perturbation experiments the OL template weighting scheme was employed. When this scheme is applied with optimal (or near-optimal) _{n}

This data shows that if we were able to predict _{n}

In order to identify the optimal way to incorporate the DOPE potential within MODELLER, we performed benchmarks with the AS single-template models by tuning _{SP} values from 0.1 to 3.5 and by employing HDDRs bearing either _{MOD} or _{n}_{SP} vary greatly.

(A) to (C) Quality scores of the AS models. (D) to (F) Quality scores of the AM models. (A) through (F) The horizontal dashed lines correspond to the scores obtained when modeling with MODELLER-generated (blue color) or optimal (orange) HDDRs without the use of DOPE.

With _{MOD} values, the maximum increase in GDT-HA is observed with a _{SP} of 0.5. As shown in _{SP}, the average GDT-HA improves by a statistically significant 1.3% with respect to the default MODELLER. At the same time, the average lDDT score increases by 2.0%, showing that the use of DOPE also aids local modeling. Of note, when applying DOPE along with the

Strategy | GDT-HA | lDDT | MolProbity score |
---|---|---|---|

MODELLER | 0.6014 (-) | 0.6563 (-) | 3.0104 (-) |

OPTIMAL | 0.6377 (+6.0%) |
0.6842 (+4.2%) |
3.0311 (+0.7%) |

MODELLER-DOPE-0.5^{a} |
0.6089 (+1.3%) |
0.6692 (+2.0%) |
2.1138 (-29.8%) |

MODELLER-SLOW-DOPE-0.5 | 0.6112 (+1.6%) |
0.6746 (+2.8%) |
2.0344 (-32.4%) |

MODELLER-DOPE-3.5 | 0.5631 (-6.4%) |
0.6397 (-2.5%) |
2.9977 (-0.4%) |

OPTIMAL-DOPE-0.5 | 0.6549 (+8.9%) |
0.7029 (+7.1%) |
2.2960 (-23.7%) |

OPTIMAL-DOPE-3.5 | 0.6885 (+14.5%) |
0.7158 (+9.1%) |
2.6280 (-12.7%) |

See

^{a}The “DOPE-X.X” suffix indicates the use of DOPE with a _{SP} of X.X.

*Asterisks denote a statistically significant difference (according to a Wilcoxon signed-rank test with a significance level of 0.05) between the scores of a strategy and the scores of the default MODELLER. See

When modeling with _{n}_{SP} of 3.5. In this case, DOPE increases the average GDT-HA and lDDT scores by 8.0% and 4.6% with respect to the scores obtained with the same restraints and the standard objective function of MODELLER. The increments in these two metrics are extremely large if computed with respect to the default MODELLER protocol (14.5% and 9.1%).

Effects brought by the use _{n}_{SP} of 3.5) on the 3D modeling of target _{n}_{SP} of 3.5 (pale cyan, shown on the right) the helices are repositioned in a native-like conformation. Figures rendered with PyMOL [

Remarkably, the same _{SP} of 3.5 leads to a large decrease in modeling quality when DOPE is applied along with _{MOD} values: in this case, the average GDT-HA and lDDT scores decrease by a large 6.4% and 2.5% with respect to the score obtained without using DOPE.

This data shows that in single-template modeling, the addition of DOPE is much more effective with _{n}_{MOD} values. Additional insights into this behaviour were provided by the analysis of DOPE energetic landscapes. _{MOD} values, applying DOPE with increasingly high _{SP} values leads to a decrease in both GDT-HA and DOPE energies. These energies eventually become even lower than the native target structure one. It seems that in the DOPE landscape, near-native conformations are not at an absolute minimum. On the other hand, when modeling with single-template optimal HDDRs, increasing _{SP} values leads to improvements in GDT-HA while maintaining DOPE energies relatively high. Similar trends are observed in the landscapes of almost all AS models. We speculate that this behaviour is caused by the fact that optimal HDDRs strongly restrain those regions of models which are structurally conserved between the native structures and templates, while they weakly restrain divergent regions. This probably allows to pinpoint the effect of DOPE in the divergent regions (where its addition likely improves modeling over the use of the standard MODELLER objective function) and to keep “rigid” the conserved regions (which are already extremely well-modeled and where DOPE can hardly improve the situation), thus giving rise to a synergistic effect.

DOPE energy landscapes for target (A) _{SP} of X.X. The green dots correspond to the DOPE-minimized native target structure.

Next, we explored the effect of DOPE in multiple-template modeling (see _{SP} is 0.5, which results in an average increase in GDT-HA and lDDT of 0.6% and 1.6% with respect to the scores obtained with the default MODELLER. By employing DOPE with this _{SP} along with the _{SP}, we assist to a decrease in 3D modeling qualities.

Strategy | GDT-HA | lDDT | MolProbity score |
---|---|---|---|

MODELLER | 0.6287 (-) | 0.6819 (-) | 3.0725 (-) |

OPTIMAL | 0.8733 (+38.9%) |
0.8106 (+18.9%) |
3.1478 (+2.4%) |

MODELLER-DOPE-0.5 | 0.6327 (+0.6%) |
0.6926 (+1.6%) |
2.2086 (-28.1%) |

MODELLER-SLOW-DOPE-0.5 | 0.6347 (+1.0%) |
0.6971 (+2.2%) |
2.1152 (-31.2%) |

MODELLER-DOPE-3.5 | 0.5646 (-10.2%) |
0.6453 (-5.4%) |
3.1267 (+1.8%) |

OPTIMAL-DOPE-0.5 | 0.8736 (+39.0%) |
0.8229 (+20.7%) |
2.5635 (-16.6%) |

OPTIMAL-DOPE-3.5 | 0.8519 (+35.5%) |
0.8061 (+18.2%) |
2.7520 (-10.4%) |

See

*Asterisks denote a statistically significant difference (according to a Wilcoxon signed-rank test with a significance level of 0.05) between the scores of a strategy and the scores of the default MODELLER. See

The results observed when combining DOPE with optimal multiple-template HDDRs are different. No value of _{SP} is able to bring a relevant improvement in GDT-HA. As _{SP} increases over 1.0, the scores even start to decrease in a significant way, although it seems that DOPE is able to bring at least a small improvement in lDDT.

This counterintuitive behaviour can in part be explained from the analysis of DOPE energy landscapes. _{SP} values leads to a decrease in DOPE energies and GDT-HA. The plots show that the models built with optimal HDDRs seem to be attracted towards a local energy minimum of DOPE, which does not correspond to the native state, but is located relatively near it. Therefore, when using optimal restraints, minimizing the DOPE of a structure distant from the native state (like in the case of single-template modeling), tends to increase its GDT-HA, but when the structure is already very close to the native state (such as in the case of multiple-template modeling), it tends to decrease its GDT-HA.

In terms of stereochemichal quality, the use of DOPE seems to be highly beneficial in both single and multiple-template modeling and with both MODELLER-generated and optimal HDDRs (see _{MOD} values and DOPE with a _{SP} of 0.5, the average MolProbity score of the AS models decreases by a large 29.8% with respect to the default MODELLER. Additional improvements in MolProbity scores are observed when coupling DOPE to the

We also tested the effect of adding DFIRE in the objective function of MODELLER. Overall, DFIRE seems to have very similar effects to the ones described for DOPE (see _{MOD} values, DOPE seems to slightly outperform DFIRE in terms of all-atom local quality (expressed by lDDT scores). When using a _{SP} of 0.5 and _{MOD} values, DOPE yields for the AS models an average lDDT score 0.5% higher than the one obtained with DFIRE, a small but statistically significant improvement (Wilcoxon signed-rank test, p-value = 4.6e-35). Therefore, we suggest that in MODELLER, DOPE should be preferred over DFIRE.

Improving the quality of HM predictions is clearly an area of great relevance in Biomedical Research [

The use of optimal _{n}_{n}

However, as first shown by the Lee group [_{n}_{n}

The other strategy that we have investigated is the inclusion of statistical potential terms, such as DOPE, in the objective function of MODELLER. We show that employing such potentials in the 3D model building phase of MODELLER robustly increases 3D modeling quality and provides a fast and effective way to improve the stereochemical details models. In order to allow the user community of MODELLER to deploy this strategy in their modeling pipelines, we share the Python code implementing it. In future research, it will be interesting to see if there exist potentials with an even more beneficial effect on 3D model building in MODELLER.

Our results have implications also for other Structural Bioinformatics tools. RosettaCM and I-TASSER borrow from MODELLER the use of HDDRs [

Of note, in the protein structure refinement field, restraints are built from a starting model and the aim is to guide the model towards its native conformation [_{n}_{n}

The development of deep learning techniques [_{n}_{n}

Note how by default the objective function does not include any “physical” attractive term between non-bonded atoms (Lennard-Jones and Coulomb potential terms from CHARMM22 [

(PDF)

(PDF)

See

(PDF)

See

(PDF)

(A) SeqId histogram of the pairwise target-template alignments in the AS models obtained using TM-align and HHalign. (B) Target coverage histograms of the same alignments. (C) Chain length histograms of the 225 AS targets, the 118 AM targets and all the 472 template chains of the analysis set. (D) CATH classes frequencies of the AS and AM targets compared to those in the entire CATH 4.2.0 database [

(PDF)

(A) to (E) the Cα-Cα _{n}_{SEL} levels using the perturbation scheme described in the “Methods” section of the main text. The observed PCCs between the perturbed and the original _{n}_{n}_{n}_{MODEL} values of the AS and AM models in _{n}_{SEL}. On average, each _{SEL} value allows to obtain almost exactly the desired level of perturbation (quantified as _{MODEL}). Data for the four HDDRs groups of MODELLER is shown. (G) AS models. (H) AM models.

(PDF)

(A) Forms of the 12561 terms of DOPE [

(PDF)

The x-axis reports the SeqId between the target and template sequences in TM-align alignments. The y-axis reports the accuracy of the corresponding HHalign alignment. The accuracy is computed as the ratio _{m}/_{m}, where _{m} is the total number of matches in the TM-align alignment and _{m} is the number of “correct” matches in HHalign alingments (that is, those HHalign matches which are also found in the TM-align alignment). The average accuracy is 0.87.

(PDF)

The horizontal dashed lines correspond to the scores obtained when modeling with MODELLER-generated (blue color) or optimal (orange) HDDRs without the use of statistical potentials. (A) to (C) quality scores of the AS models. (D) to (F) quality scores of the AM models.

(PDF)

(PDF)

(PDF)

(PDF)

(PDF)

The authors wish to acknowledge Fabio Mastrantuono and Fransceso Pesce for helpful discussions and their precious help.

This work is dedicated to the memory of our beloved mentor Prof. Francesco Bossa.

Dear Dr JANSON,

Thank you very much for submitting your manuscript 'Revisiting the “satisfaction of spatial restraints” approach of MODELLER for protein homology modeling' for review by PLOS Computational Biology. Your manuscript has been fully evaluated by the PLOS Computational Biology editorial team and in this case also by independent peer reviewers. The reviewers appreciated the attention to an important problem, but raised some substantial concerns about the manuscript as it currently stands. While your manuscript cannot be accepted in its present form, we are willing to consider a revised version in which the issues raised by both reviewers have been adequately addressed. We cannot, of course, promise publication at that time.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

Your revisions should address the specific points made by each reviewer. Please return the revised version within the next 60 days. If you anticipate any delay in its return, we ask that you let us know the expected resubmission date by email at

In addition, when you are ready to resubmit, please be prepared to provide the following:

(1) A detailed list of your responses to the review comments and the changes you have made in the manuscript. We require a file of this nature before your manuscript is passed back to the editors.

(2) A copy of your manuscript with the changes highlighted (encouraged). We encourage authors, if possible to show clearly where changes have been made to their manuscript e.g. by highlighting text.

(3) A striking still image to accompany your article (optional). If the image is judged to be suitable by the editors, it may be featured on our website and might be chosen as the issue image for that month. These square, high-quality images should be accompanied by a short caption. Please note as well that there should be no copyright restrictions on the use of the image, so that it can be published under the Open-Access license and be subject only to appropriate attribution.

Before you resubmit your manuscript, please consult our Submission Checklist to ensure your manuscript is formatted correctly for PLOS Computational Biology:

- Figures uploaded separately as TIFF or EPS files (if you wish, your figures may remain in your main manuscript file in addition).

- Supporting Information uploaded as separate files, titled Dataset, Figure, Table, Text, Protocol, Audio, or Video.

- Funding information in the 'Financial Disclosure' box in the online system.

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool,

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see

We are sorry that we cannot be more positive about your manuscript at this stage, but if you have any concerns or questions, please do not hesitate to contact us.

Sincerely,

Bert L. de Groot

Associate Editor

PLOS Computational Biology

Arne Elofsson

Deputy Editor

PLOS Computational Biology

A link appears below if there are any accompanying review attachments. If you believe any reviews to be missing, please contact

[LINK]

Reviewer's Responses to Questions

Reviewer #1: The manuscript describes an analysis of the potential to improve homology modelling using “satisfaction of spatial restraints” in the widely used MODELLER package. If the structural divergence between target and template is optimally modelled there is a large room for improvement in particular for multiple templates. Improvements (2% for single and 11% for multiple templates) are expected if the predicted structural divergence correlate >0.6 compared to the true divergence. In addition, the authors also investigate the possibility to include a statistical potential in the objective function of the MODELLER and show that using the build in DOPE statistical potential yields a small but consistent improvement in model quality. Code to for the latter is provided in a git repo.

Overall this manuscript was a nice read and present results that can be used to consistently improve modelling results. It also establish an upper limit on model quality that can be gained if the templates are used optimally. However, if I should provide some criticism, a large part of the analysis is done using information from the native structure when setting the local weights. If you know exactly which residues to move and to which degree, you should expect large improvements.

1) The authors provide some perturbation analysis by randomly changing some fraction of weights from their optimal value, thereby reducing the correlation from 1.0 to 0.0 and show that improvements are expected when the correlation is >0.6 (Fig 5). Looking at Fig S2, 0.6 correlation corresponds to changing only 30% (0.3) of the residues. Thus, 70% of the residues have their structural divergence at their optimal and 30% are random. This scenario is highly unrealistic; 0.6 correlation doesn’t seem too high, but since correlation is really effected by small number of random points buried among perfect predictions, the actual required prediction performance might be much higher. I suggest that the perturbation analysis is performed in a more rigorous way that sample distributions more likely to originate from a model quality assessment prediction method, where each estimate would have some uncertainty, Or better use a proper model quality assessment program, like ProQ3D or QMEAN to get realistic estimates.

2) Best performance gain can potentially be obtained using multiple templates. However, again here, the fact the authors are using the knowledge on which template is optimal obfuscates the true value of this results. We already know that if you are able to always pick the best model you would outperform any group in CASP, the problem is to pick the best model/template. Thus, it is crucial that effect of errors in the estimates are investigated more throughly.

3) For multiple templates how are the delta(d_ij) for different templates distributed through the target sequence? Is it different templates that dominates in different regions, or are they intertwined? i.e. is it effectively using more than one template for any given region or is it more picking the best template for each region?

4) It is a bit unclear on how the local weights are implemented, do you provide a custom made restraints file to Modeller? or did you find any other API to interface with the Modeller functions? Anyway I think it would be useful if you in addition to the code you already provide, also provide code for running Modeller with the optimal weight (if native is available), or user-specified given a list of local predicted CA-CA distances.

Reviewer #2: This manuscript describes a study of MODELLER, a widely-used software tool for protein homology modeling.

Given that MODELLER is widely used, further study of this program and even a small improvement are always desirable.

In this manuscript, the authors have studied the relationship between modeling quality and estimation of the difference of an inter-atom distance between target and template. The authors claim that 1) a more accurate estimation of the difference may improve modeling accuracy and 2) modeling quality may be increased by incorporating some statistical potentials into MODELLER. These findings are not very new, but the manuscript provides sufficient data and analysis to back up them, which to the best of my knowledge is not widely available in the literature. The authors have also released source code for the incorporation of DOPE and DFIRE into MODELLER, although this is not very new (Similar code is available at the MODELLER website).

Some minor concerns:

1) lines 46-47, it is fine to say that template-based modeling is the most popular, but I am not sure if it is fine to claim that the most successful approach is template-based modeling since this method fails on the modeling of many membrane proteins. In particular, in the past 2-3 years template-free modeling has made a very good progress and now its accuracy is comparable or even better as long as the target protein does not have very good templates. Further, currently the best template-free modeling also works well on membrane proteins.

2) lines 69-71, some revision is needed here. In addition to algorithm advance, the enlargement of both sequence and structure databases is also a very important factor for the improvement of HM modeling.

**********

Large-scale datasets should be made available via a public repository as described in the

Reviewer #1: Yes

Reviewer #2: Yes

**********

PLOS authors have the option to publish the peer review history of their article (

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Reviewer #1: Yes: Björn Wallner

Reviewer #2: No

Dear Dr JANSON,

We are pleased to inform you that your manuscript 'Revisiting the “satisfaction of spatial restraints” approach of MODELLER for protein homology modeling' has been provisionally accepted for publication in PLOS Computational Biology.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. Please be aware that it may take several days for you to receive this email; during this time no action is required by you. Once you have received these formatting requests, please note that your manuscript will not be scheduled for publication until you have made the required changes.

In the meantime, please log into Editorial Manager at

One of the goals of PLOS is to make science accessible to educators and the public. PLOS staff issue occasional press releases and make early versions of PLOS Computational Biology articles available to science writers and journalists. PLOS staff also collaborate with Communication and Public Information Offices and would be happy to work with the relevant people at your institution or funding agency. If your institution or funding agency is interested in promoting your findings, please ask them to coordinate their releases with PLOS (contact

Thank you again for supporting Open Access publishing. We look forward to publishing your paper in PLOS Computational Biology.

Sincerely,

Bert L. de Groot

Associate Editor

PLOS Computational Biology

Arne Elofsson

Deputy Editor

PLOS Computational Biology

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #1: All my comments have been adequately addressed.

**********

Have all data underlying the figures and results presented in the manuscript been provided?

Large-scale datasets should be made available via a public repository as described in the

PLOS Computational Biology

Reviewer #1: Yes

**********

PLOS authors have the option to publish the peer review history of their article (

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review?

For information about this choice, including consent withdrawal, please see our

Reviewer #1: Yes: Björn Wallner

PCOMPBIOL-D-19-01048R1

Revisiting the “satisfaction of spatial restraints” approach of MODELLER for protein homology modeling

Dear Dr JANSON,

I am pleased to inform you that your manuscript has been formally accepted for publication in PLOS Computational Biology. Your manuscript is now with our production department and you will be notified of the publication date in due course.

The corresponding author will soon be receiving a typeset proof for review, to ensure errors have not been introduced during production. Please review the PDF proof of your manuscript carefully, as this is the last chance to correct any errors. Please note that major changes, or those which affect the scientific understanding of the work, will likely cause delays to the publication date of your manuscript.

Soon after your final files are uploaded, unless you have opted out, the early version of your manuscript will be published online. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.

Thank you again for supporting PLOS Computational Biology and open-access publishing. We are looking forward to publishing your work!

With kind regards,

Laura Mallard

PLOS Computational Biology | Carlyle House, Carlyle Road, Cambridge CB4 3DN | United Kingdom