Introduction

The importance of structural biology in biochemistry is taught in nearly every biochemistry textbook used in biochemistry lectures. Protein structure aids researchers in various capacities, such as drug discovery, protein engineering, biotherapeutic development, and many other biotechnological applications. Typically, protein structures are elucidated using 1 or a combination of 3 techniques: x-ray crystallography [1], NMR spectroscopy [2], or cryo-electron microscopy [3]. These protein structure-solving techniques are widely practiced but are typically reserved for training in graduate school [4]. This is mainly due to the complexity and time it takes to produce results; it does not fit into a “bite”-sized module or semester class schedule. Just as effort in computer hardware and software development transformed the old days of assembling DNA sequences by hand from gel electrophoresis, there has been a substantial push to leverage computational power to solve protein 3D structures. Every other year since 1994, hundreds of research groups have competed in the Critical Assessment of Structure Prediction (CASP), which has served as the longstanding benchmark for the accuracy and measure of feasibility for predicting protein structures [5].

In 2020 at CASP14, the first ever “success” (>90% accuracy in predicting protein structure) was achieved by the artificial intelligence company DeepMind’s (now Google DeepMind) AlphaFold2 [5,6]. AlphaFold2 broke previous boundaries by employing large machine-learning models that could predict highly accurate structures for proteins with sequence homology to other proteins that already had solved structures and in proteins that exhibited little to no sequence homology to known sequences [6]. The latter directly showed machine learning’s decisive role in a global or template-free approach [7], i.e., not having a reference structure as a proverbial “blueprint” to build a model. The inherent impact on the biochemical research community that AlphaFold2 has had and continues to have cannot be understated. Recently, it has been used to solve systems so large and complex that they had previously eluded researchers using traditional methods for solving protein structures, such as the cytoplasmic ring of the nuclear pore complex [8]. AlphaFold2 can also supply virtual drug library screening structures that are in many biopharmaceutical companies’ pipelines. However, it should be noted that cautious evaluation of the final model should be taken into consideration before drawing any conclusions [9].

We developed a learning module that introduced the 3D protein structure prediction software AlphaFold2 to a senior-level biochemistry laboratory class. AlphaFold2 is an ideal bioinformatics tool since it fits the criteria of a “click-and-go” software. It is executed by running a “wrapper script” that performs all steps needed to produce a 3D protein structure, with minimal parameter optimization and coding required by the end user. The other benefit of using AlphaFold2 to introduce undergraduates to bioinformatics is that much of the knowledge needed to interpret the results of AlphaFold2 extends from a basic understanding of protein folding and structure. All of this combined makes AlphaFold2 an ideal tool to explore for introduction at the undergraduate biochemistry level.

The AlphaFold2 module was divided into two 3-h class sessions with a total of 64 undergraduate students. Students and faculty were surveyed with standardized questions adapted from the Colorado Learning Attitudes in Science Survey (CLASS) Field [10] to evaluate the module. CLASS question sets have been used and modified to assess student perception of undergraduate classwork in physics, chemistry, biology, and computer science [10–13]. To emphasize the novelty of AlphaFold2 and its unique applications, protein structures were predicted for protein sequences isolated from the MPOX 22 global outbreak USA virus isolate (ON563414.3). Students were tasked with evaluating the resulting predicted structures in both a qualitative and quantitative capacity. Finally, students were given a worksheet (S1 File) to evaluate their structure models and to help them assign functional annotations based on domain sequence homology to the NCBI Conserved Domain Database [14].

Results

Students and experts were surveyed for their responses to standardized CLASS questions meant to probe attitudes to learning biochemistry in the classroom and how a module focused on implementing a novel tool could affect this perception (Table 1 and Figs 1 and S1–S3 and S2 File). Experts included faculty, staff, and postdoctoral researchers with a doctoral degree in the Department of Biochemistry at Texas A&M University. The experts agreed (as defined in the Methods section, >66.6% agreement, see S3 Fig) for all questions. Statistical analysis was carried out between the 3 sets of response groups: pre-module students (PS), post-module students (PoS), and experts (EX). Since not all participants in the PS survey completed the PoS survey, direct statistical analysis between these student participant groups was avoided to not introduce systemic bias. In most cases, student responses started to approach experts’ answers after the module (Figs 2A–2C and S7 File).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Overview of the AlphaFold2 module.

(A) Simple flowchart detailing the breakdown of each lecture and class work accomplished between both days of the module. The red bolded text is expanded in (B) schematic of AlphaFold2 workflow from a student perspective. A fasta sequence file was used as input to AlphaFold2 (I), the features.pkl and model_ptm.pkl files were extracted from the AlphaFold2 output directory (II), MSA sequencing homology and depth were extracted (III), pLDDT and pAE values were extracted (IV), and finally, the structure was investigated by analysis against the conserved domain database (V).

https://doi.org/10.1371/journal.pcbi.1012123.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Statistical analysis of a subset of the CLASS survey responses in pre-module students (PS), post-module students (PoS), and experts (EX).

(A–C) Clustered bar charts of average response scores for 22 of the 31 CLASS questions based on the 3-point Likert scoring system. Error bars represent standard error values. *** and ^^^ denote statistical significance from computed t test statistics based on p-values (p < 0.05) for PS to EX and PoS to EX, respectively. PS, PoS, and EX are colored dark gray, light gray, and black, respectively. (D) Response sums for the 8 additional post-module questions. Responses are color-coordinated light gray, dark gray, and black for “Agree,” “Neutral,” and “Disagree,” respectively. Note that question 35 does not have the same number of responses, as students were only asked to respond under special conditions. A complete list of the questions and analysis of all the survey data can be found in the Supporting information.

https://doi.org/10.1371/journal.pcbi.1012123.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Survey statements organized by CLASS category.

Statements were closely adapted from a previous adaptation of the CLASS-BIO set [12]. Similarly, statements were changed by replacing the words “biology” and “molecular biology” with “biochemistry”.

https://doi.org/10.1371/journal.pcbi.1012123.t001

This module was formulated to show an alternative approach to students, highlighting that AlphaFold2 was a tool that could produce biologically relevant 3D structures of proteins when traditional structure elucidation methods were not feasible. Student post-module (PoS) responses were statistically indistinguishable from expert responses (EX) in questions 8, 15, and 19 (Fig 2A and 2B), while pre-module student responses (PS) were statistically distinguishable. Question 8 in the survey probed the opinions of the students on how they approach a problem after failing at it the first time. This module was a clear, practical lesson in how traditional methods can fail in the laboratory, but another approach could produce experimental results. Student opinions shifted to be statistically indistinguishable from the experts’ opinions.

Another facet of this module was that, depending on the protein sequence being analyzed, the predicted structure from AlphaFold2 could vary rapidly in model confidence metrics (Fig 3A–3L) depending on sequence homology to other known proteins. Question 15 probed student opinions on whether learning biochemistry relies on memorizing facts and definitions. PS responses were statistically different from EX responses, while PoS responses more closely resembled EX responses and were not statistically different. While only a minor shift in the average response, it speaks to the fact that laboratory classes typically use widely applicable techniques and protocols, but in real applications, these can often be met with confounded outcomes. This would require the students to rely on their understanding of biochemical principles and begin to explain why their data deviated from the expected result.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Example of data extracted and analyzed from the AlphaFold2 models.

(A, E, I) Three protein sequences from the MPOX 22 virus USA isolate had structures predicted in and rendered with ChimeraX chain coloring, indicating pLDDT value at each residue with blue to red indicating low to high pLDDT. (B, F, J) MSA depth and homology plots extracted from the features.pkl file from the AlphaFold2 output directories. Each graph corresponds to the accompanying protein structure. The number of homologous sequences in the MSA was plotted along the y-axis and colored based on sequence identity. The black line indicates the total number of sequences that aligned at a given target residue. (C, G, K) Scatter plots of the pLDDT value at each residue in the target sequence, color-coded by pLDDT value at the residue. (D, H, L) 2D heatmaps of the calculated pAE for each protein structure in A, E, and I. These structures and data were chosen as a representative sample of the class data to show the wide variety of predicted structures in terms of overall folding, local, and relative model confidence.

https://doi.org/10.1371/journal.pcbi.1012123.g003

Student opinions were questioned on whether biochemistry is related to their everyday life (Question 19); it had a similar outcome as Question 15. Before the module, PS responses were statistically different from EX responses, while post-module PoS responses were not statistically different. A probable explanation for this difference in response is that the protein sequences used in this module were from the recent (when conducting this study) 2022 MPOX outbreak USA virus isolate. Next, students stuck on biochemistry-related questions were asked if there was little to no chance that they could figure it out independently (Question 21). Interestingly, student responses diverged (Fig 2C) from expert opinions post-module. Not only did the PoS responses differ from the EX-response, but they were statistically different. This marks the only survey question where post-module student responses did not converge with expert opinions. Again, this likely stems from the complexity of biochemistry and math introduced in this module.

Most interestingly was the idea that applying methods or techniques used to understand one biochemical problem to another problem would require the issues to be connected in some way, which differed drastically between students and experts (Question 11). Student opinions remained the same after the module compared to experts (S1–S4 Figs), with a mean response of 2.24 and 2.25, respectively, compared to the expert’s mean response of 1.28. This suggests that structural proteomics and the complex problem of elucidating a protein’s 3D structure from the amino acid sequence opened the students to a complexity they likely had not previously seen in a classroom setting.

A particularly relevant question regarding introducing bioinformatics into the undergraduate curriculum is the dependency of mathematics in understanding biochemical topics (Question 26). A previous study on implementing a bioinformatics-based module into a molecular biology class showed that students agreed with experts only after the module [15]. Interestingly, student responses before and after the module agreed with the experts and were not statistically distinguishable from the experts’ responses (S4 Fig). This may be due to the apparent difference in the structure of a molecular biology versus biochemistry laboratory class and the mistaken belief that modern bioinformatics arose from the need for analyzing next-generation sequence data [16].

Post-module students were additionally surveyed with a custom set of questions (S2 File) meant to probe the effectiveness of this module and students’ perception of it (Fig 3D). About 83% (30/36) of PoS responders indicated that their curiosity about bioinformatics had increased as a direct result of this module. Additionally, approximately 94% (34/36) PoS responders indicated that the module had also introduced them to the use of computer science in biochemistry. This was an important goal when designing the module; typically in undergraduate biochemistry curriculums, there is a lack of emphasis on how computer science has played and will continue to play a significant role in the development and execution of biochemical research [17].

Question 35 asked students if they were not enjoying computational data analysis. Of 19 out of 36 that responded, approximately 94% (18/19) indicated that this module gave them a good idea of how biochemists handle large protein datasets. We wanted to show students what happens to proteins after their sequences and structures are deposited into public databases such as NCBI and the Protein Data Bank. The last 4 questions of the additional post-module survey questions, 36 to 39, tried to elucidate how practical the module was for increasing student interest in bioinformatics concerning biochemistry. Only approximately 33% (12/36) of responders indicated they were interested in pursuing additional bioinformatic-related coursework due to the module. This number is not surprising as a majority, approximately 95%, of the students indicated they planned to attend medical school or work in a different field. However, 50% (18/36) indicated they were interested in learning more about high-performance research computing and how it could apply to their research and career goals. Even more interesting was that most responders indicated that, while they prefer traditional wet-lab/field-work experimentation to work that requires computationally intensive analyses (approximately 66%, 24/36), they understood that their future career in biochemistry would rely on basic knowledge of bioinformatics/computational biochemistry (approximately 77%, 28/36).

Discussion

The field of biochemical research is becoming more reliant on bioinformatics, and it is our duty as educators to prepare coursework that reflects current techniques and methodology in undergraduate curriculums. Our novel AlphaFold2 module was designed to fit within an existing biochemistry laboratory or classroom format, with little to no bioinformatic framework needed for implementation. We chose AlphaFold2 based on its open-source nature [6], readily available implementations [18], and that the final output is a 3D-viewable file that can be interpreted through basic biochemical knowledge of protein structure and function. We concentrated on building a two-part module that focused on teaching the basics of protein-structure prediction, introducing students to the Protein Data Bank repository, having students predict the structure of an unclassified protein and extract meta-data such as sequence homology, pLDDT, and predicted aligned error (pAE) as metrics for evaluation in model confidence. We found that the approximately 64 students responded well to the module and enjoyed learning about this newer technique that has emerged in the field of biochemical research. We surveyed students before and after the module and experts independently with altered CLASS questions. We additionally analyzed students post-module with custom questions probing their resulting interest in bioinformatics from this module. Our surveys found that introducing a “hands-on” module centered around a current global health-related problem, the MPOX ‘22 outbreak, reinforced the applicability of the techniques and theories taught in the classroom in their everyday lives. Students were introduced to a complex problem facing the larger biochemical research community of predicting protein 3D structures. They were taught how to use AlphaFold2 to probe for these structures and how to evaluate the model confidence based on 3 useful metrics. While the students, mostly graduating seniors, did not seem ready to dive into bioinformatics-heavy coursework moving forward in their careers, they remained impressed by AlphaFold2. They understood the dependence biochemical research has on bioinformatics and the benefit it could bring to their research. Student engagement in the module was overwhelmingly positive, and its impact on future academic coursework selection was likely masked by the large proportion of students in the course who were graduating at the end of the semester. However, any conclusions drawn on this module’s effect on future bioinformatic coursework would be better served with surveys with a refined set of post-module questions accounting for the graduation dates of the students.

Based on our pilot study with this module, we recommend a few changes that may increase the learning potential and enjoyment for the students. First, we recommend increasing the number of proteins that each student predicts. In our pilot, each student was only assigned a single protein sequence to predict a structure for and to analyze. Many studies have shown that repetition correlates to increased retention [19]. Due to the “unpredictable” nature of the predicted structures and the level of confidence one could have in the resulting model, the comfort level of the students for viewing and analyzing protein 3D structures would likely increase by having the students repeat the post-structure prediction analysis, MSA, pLDDT, and pAE inspections, combined with increased time viewing and manipulating structures in a molecular viewing software. The learning potential would also likely improve by increasing the amount of time dedicated to the module. By devoting more class time to the module, students could spend more effort thoroughly analyzing their predicted protein structures. Further analyses could be added, such as molecular docking, to simulate a typical in silico workflow for screening drug targets. Lastly, in future adoptions of this module in any fashion, the authors recommend substituting the protein functional annotation by the NCBI Conserved Domain tool [14] with InterPro [20].

It is important to point out that in this case, the course Biochemical Techniques I is a junior/senior level biochemistry techniques laboratory meant to reinforce with bench work what students have learned in their previous biochemistry lecture courses. The laboratory is constructed to closely follow a set of published research papers from former faculty members of the department, wherein students mutate, express, and purify ribonuclease Sa and assay it for its catalytic and thermostable activity. Within the 15-week semester that the course is offered, this experimental journey takes approximately 11 weeks. The remaining 3 weeks are used to incorporate additional content, while the final week is reserved for a final and practical examination. This includes guest lecturers covering topics such as Cryo-EM, protein folding, and mass-peptide fingerprint analysis. Inclusion of this module did not require instructors to modify the course content as it became part of the additional material. However, we are aware that whenever instructors consider introducing new content into the classroom, there is always the question of what must be sacrificed to meet term timelines. If instructors do not have additional time to allot to this module or cannot sacrifice already prepared content, we suggest modifying this module in a way that includes the use of AlphaFold2 as a method of analysis such as having students utilize AlphaFold2 to predict the 3D structure of a protein they are analyzing in the lab. In the case of proteins that already have 3D structures elucidated, students could potentially evaluate the predicted protein structure complex between their protein of interest and a target protein-ligand using the AlphaFold2 multimer model.

For educators who do not have access to a high-performance computer cluster on their campus, we highly recommend utilizing ColabFold [18]. ColabFold offers the entire Alphafold2 package and runs on Google’s free cloud computing resources. The developers behind ColabFold replaced the initial MSA construction step with a significantly faster search algorithm. This results in final structure predictions typically ranging between 30 min to 1 h in our own test cases. While ColabFold results in quicker predictions, instructors should be aware that there are some limitations. These include predictions aborting due to limited RAM which typically occurs when dealing with larger proteins/complexes, and/or proteins with large levels of sequence homology among the databases that AlphaFold2 queries. We highly recommend that instructors test ColabFold on a representative subset of their proteins before giving them to their students. With these free resources offered, users can predict structures for small, simple proteins on the time scale of approximately 1 h with built-in metrics for analyzing MSA depth, pLDDT, and pAE for the target protein sequence. Students can then download their results directly to their Google Drive for downstream analysis.

Methods

Supporting information

Acknowledgments

Portions of this research were conducted with the advanced computing resources provided by Texas A&M High-Performance Research Computing. We want to thank Pedro Monagas-Valentin and Elaina Curtis for their invaluable roles as a teaching assistant and learning lab coordinator in helping administer the module, respectively. We would also like to thank all the students in Biochemical Techniques I and faculty in the Texas A&M Department of Biochemistry & Biophysics who participated in the surveys and offered constructive criticism and opinions throughout the study. We would also like to acknowledge Dr. Lisa Perez from the Texas A&M HPRC for her insight, planning, and reservation of TAMU HPRC compute nodes to ensure the students’ AlphaFold2 jobs would be completed on time.