PyDNA_CF_Simulator: A Python-based ChatGPT plugin for CF simulation using PyDNA.

To explore the possibility of GPT directly invoking python scripts for simulation tasks, we attempted to have GPT generate a pydna script representing the pSB1A2-Bca9128 example CF (cf_to_pydna.html). The pydna library shares a similar ontology with CF and includes simulators for PCR, digestion, ligation, and Gibson assembly methods. However, the resulting script from GPT required us to make several manual adjustments, including moving the pip statement, adding the DNA sequences, and removing the API requests to GenBank. Despite these corrections, GPT incorrectly used an ’Assembly’ function to simulate ligations rather than the ’+’ operand on the sequence objects, rendering the script unrunnable. This experiment led us to conclude that GPT’s current capabilities are insufficient for writing this executable representation of CF.

While Python scripts are useful, they present several challenges when used as documentation for construction files in an AI interface. Firstly, they are written in free-form Python, there are potential security issues with an interface that executes these scripts. Secondly, they assume a specific software implementation, limiting extensibility and interoperability in a multi-tool environment. Lastly, Python scripts do not readily enable inspection, a crucial feature for using CF as a specification.

To address these limitations, we developed a Python plugin wrapper, PyDNA_CF_Simulator [15], capable of parsing CF and executing the appropriate pydna syntax for simulation. We created Python classes for ConstructionFile and Polynucleotide according to the jsDoc spec, and developed functions for parsing Strings of CF shorthand or JSON into these classes. Functions were also created to interconvert between Polynucleotide and Dseqrecord representations. We then developed a function that simulates a ConstructionFile instance, executing the appropriate operations and returning the resulting product sequences. Finally, we created an API wrapper to host the simulator as a REST endpoint, along with a YAML and manifest containing the shorthand specification for communication with ChatGPT.

Testing of the Python plugin wrapper revealed several limitations. While the plugin successfully handles simple cases like PCR on short templates (pydna_plugin_test.html and pydna_plugin_test.mov), its token limit in the low thousands significantly curtails its utility with larger DNA sequences. This limit is far from sufficient to encode complex structures like plasmid sequences, let alone the millions+ tokens required to express a genome sequence. Due to its limited utility, we have not submitted PyDNA_CF_Simulator for inclusion as an official ChatGPT plugin. However, the code is available on Github under the open-source MIT license.

Further limitations were found within the pydna library itself. Pydna’s inability to simulate Golden Gate reactions, a cornerstone of modern synthetic biology, greatly restricts its utility for a wide range of experiments. Although the source code includes a script for it, it is not fully implemented. While Golden Gate could be described as sequential digestion and ligation steps, which are implemented, this is not equivalent to the simultaneous cutting and ligation that occurs in the actual process which requires additional logic. Additionally, pydna allows non-DNA letters, even permitting the entire alphabet as syntax.

While the Python plugin wrapper effectively delegates the simulation task to reliable, well-tested code, it has notable limitations. A significant challenge with this type of interface is the absence of visualization and persistence for both the resulting sequences and the Construction File itself. Ideally, an additional interface would be integrated into the workflow to provide users with a clearer understanding of the process and its outcomes. These findings highlight the necessity for further development and enhancements to the AI interface, particularly in the areas of user interface design and strategies to circumvent token limits.

ConstructionFileSimulator: A Java-based tool for validation and simulation of CF.

There are two distinct types of software that could be developed for simulating Construction Files (CFs): one that validates the CF, and another that calculates the product. While these objectives may seem similar, they lead to different design decisions and implementations. For instance, consider a Golden Gate assembly of three fragments, where one fragment has compatible ends on both sides and thus will re-ligate. A tool focused on calculating the product would correctly simulate this scenario and return the single-fragment product. However, a tool focused on validating a CF would instead identify this scenario as a problem, alerting the user to the potential issue rather than simply returning the result. This focus on error detection and prevention is crucial for ensuring the validity and success of genetic engineering experiments.

With this validation objective in mind, we developed the first iteration of ConstructionFileSimulator (CFS) in Java [16]. We employed a programming style reminiscent of Functional Programming with mostly-pure functions and immutable classes. It interprets CF shorthand text into a ConstructionFile object, subsequently simulating the expected reaction product step by step. If an error arises during simulation, it triggers an error response which terminates the operation and delivers a detailed message to guide corrective action.

The relationship between CF operations and simulator functions in CFS is largely one-to-one, but the concurrent development of the CF syntax, CFS, and wetlab usage has led to the need for backward compatibility with past versions of CF. As a result, CFS can handle a broader array of syntax than the specified shorthand, and the codebase contains more complexity than strictly necessary. It also supports PCA (Polymerase Chain Assembly), SOE (Splicing by Overlap Extension), and Klenow (Klenow extension) operations which are not in the specification. From a system architecture perspective, it’s worth noting that a strict one-to-one correspondence between operations and functions is not always the most efficient or effective design. For example, lower-level functions such as reverse complementation (RevComp.java) are used across multiple algorithms and are therefore implemented as standalone functions rather than being associated with specific operations. Furthermore, to accommodate a variety of PCR-like scenarios, we generalized these techniques in the simulation. While this abstraction was challenging to express in shorthand, it provides a compact solution at the functional level. The CFS codebase also includes several exploratory and vestigial features that we have omitted from this discussion for the sake of focus.

Within this architecture, the project houses two PCR simulators, each designed to address specific experimental scenarios. The simpler one, encoded in the method perfect18Simulation, is only activated when a singular template and two oligos are present, with both oligos perfectly matching the template over 18 bp at their 3’ ends. This condition is usually met for standard cloning experiments. However, for non-standard scenarios, such as site-directed mutagenesis involving 20-mer oligonucleotides with a central mismatch, a more mechanistic simulation is needed. This includes simulating PCA, SOEing, or Klenow Extension, where template varieties from single-stranded to double-stranded, and their quantities from 0 to n, must be considered. To accommodate these scenarios, the PCRSimulator employs a backup algorithm, which mimics pairwise DNA annealing and extension. It checks for alignments where the 3’ six bases of the oligo exactly match the template, then uses JAligner and Tm calculations for further detection of annealing sites. However, this more complex function, while generally reliable, occasionally struggled with scenarios that a simpler algorithm could handle correctly. Additionally, it was computationally demanding, causing failures for longer templates and occasional inability to detect obvious annealing sites. To mitigate this, the simpler version is used as a first attempt before falling back to the more mechanistic simulation when necessary. The simulator can now handle more scenarios than outlined in the specification documents, including unique cases like mixtures of single-stranded and double-stranded templates. Both algorithms have been rigorously tested and confirmed to work on linear and circular templates, including inverse PCRs, and they handle 5’ modifications, 5’ extensions, and common issues such as multiple annealing sites and orientation errors.

The Digest operation uses a REBASE database-derived file for restriction enzyme information, making it capable of handling more enzymes than mentioned in the specification. It correctly handles degenerate cutters, both 5’ and 3’ extensions, and appropriately assigns phosphates to the 5’ modifications of freshly cut DNAs. Though there is a method in the code (cutOnce) that simulates a single cutting event, the Digest operation is assumed to mean ’cut to completion’ and thus does not support partial digests.

In the simulation of ligation, the presence of a 5’ phosphate and matching sticky ends are checked, and two matching ends of two input Polynucleotides are concatenated into one. This process is repeated until only one fragment remains. If its ends are compatible, it is denoted as a circular DNA, and the sticky ends are integrated into the sequence field of the resulting Polynucleotide.

The simulation of GoldenGate primarily involves cutting with the type IIS enzyme and ligating the fragments, with additional checks for orientation, number of sites in the molecule, and the appropriateness of the generated sticky ends. Gibson simulation finds exact 20 bp matches between homologous ends and connects the DNAs pairwise.

The simulation of transformation, while implemented, is currently limited to checking that the product is circular. This is because transformation of a bacterium with a DNA requires it to be circular. However, it’s worth noting that CFS fully enables the generation of in vitro linear DNAs, which can be useful in certain scenarios, such as library fabrication.

CFS includes rigorous checks for possible design errors and provide comprehensive error messages when triggered. Over the course of three years, our use of the CFS for validating wetlab designs, along with its extensive application by over 100 students, has enabled us to identify and rectify numerous bugs. This iterative process led to the creation of a multitude of unit tests for various edge case scenarios, enhancing the reliability and robustness of our simulator. The development history is documented as issues in the ConstructionFileSimulator repository on Github.

CFS’s most straightforward interface is its SimulatorView Swing GUI, launched by executing the jar file without arguments. This interface accepts a construction file’s shorthand text and outputs the final step’s product. As illustrated in Fig 4, we supplied the GUI with steps parsed by ChatGPT from the native invasin text, along with the sequences of the three input plasmid sequences (Chats/ invasin_cf.txt). The resulting sequence of pBACr-AraInvasin matches the expected map and aligns with sequenced isolates, validating the simulator’s accuracy and utility. We have provided an array of real-world examples (found in the supplemental Examples folder), showcasing the successful application of CFS. These examples feature experiments that involve degenerate bases, the creation of libraries, SOEing, PCA, and Klenow extension, all of which the simulator correctly handles.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Simulation of Invasin Construction File in a script editor.

SimulatorView, a simple GUI included with ConstructionFileSimulator, accepts the text of a construction file and outputs the product of the final step. In this instance, the GUI is provided with the steps parsed by ChatGPT, along with the sequences of the three input plasmid sequences. The complete document can be found in the supplementary file ’invasin_cf.txt’. Upon clicking ’run’, the construction file is simulated step-by-step. The resulting sequence of pBACr-AraInvasin aligns with the expected map and is consistent with sequenced isolates, demonstrating the accuracy and utility of the simulation.

https://doi.org/10.1371/journal.pone.0294469.g004

Mitigating clerical mistakes with ’experiment’ objects in ConstructionFileSimulator.

While simulating a CF is an effective way to detect technical errors in experimental design, such as oligo design issues, it doesn’t account for clerical errors that often occur in larger experiments involving multiple CFs or during collaborations among research teams. These errors, such as maintaining different versions of input sequences, are surprisingly common and can severely impact the success of an experiment. To address these issues, we’ve introduced the concept of an ’Experiment’ object into CFS.

The creation of an ’Experiment’ object begins by passing a hard drive path to a folder containing all relevant files to a parser. This includes sequence files in TSV or GenBank format (.gb,.seq,.str, and.ape), CFs expressed as plain text files, and additional sequence files, primarily for oligos, in a TSV format that also allows additional columns. This format is particularly useful as it aligns with the IDT oligo form, reducing the risk of error when copying and pasting between what is simulated and what is ordered. The parser then outputs an ’Experiment’ object that encapsulates all the provided information. Once the ’Experiment’ is created, it can be simulated to ensure the accuracy of the documentation.

Executing a list of CFs requires additional analysis to determine the correct order of execution. This is crucial to ensure that the products of earlier files can be used as inputs for later files. For example, in the pTP2_reporter example, a series of unrelated experiments was used to construct a reporter plasmid. To correctly simulate this, the software must identify the order of execution for each CF. We also had to consider the potential for name reuse, such as "pcrpdt" to refer to products of intermediate steps. To address this, the CFS implementation of ConstructionFile includes an explicit singular output from the entire file, which is set as the product of the last step during parsing. This addition, while not explicitly part of the specification, is necessary to resolve potential conflicts and implies that a ConstructionFile describes not only an ordered list of steps but also a specific product outcome.

The need for this higher-order ’Experiment’ object is heavily dependent on the user interface. Our current approach treats files as contents of a folder, but other systems might use a database, potentially reducing the impact of clerical errors if the design and simulation functions were integrated. Furthermore, the exact content and format of an ’Experiment’ are yet to be defined. Within the CFS, it encompasses sequences and CFs, but a more comprehensive specification could include measurement data, analysis, and more. Therefore, while this ’Experiment’ functionality is part of the CFS project, we currently propose no standards for it and present it as an exploratory feature.

An ’Experiment’ folder can be parsed and simulated using the SimulateExperimentDirectory function. This function is executed when the user runs the jar from the command line and passes in the path to the folder as a parameter. SimulatorView will also execute this function when such a folder is dragged-and-dropped on the GUI. Upon execution, the simulator generates a GenBank file for each product sequence and creates two log files: C5seqs.txt, which contains all sequences (inputs, intermediates, and products), and C5log.txt, which provides a detailed account of all events that occurred during execution. These log statements are also outputted to the command line when the jar is run from there. This information is helpful for identifying and correcting errors in the experiment’s design or documentation.

There are two supplemental examples of ’Experiment’ folders that can be run with CFS. The Lycopene2 example demonstrates a scenario where several construction files are performed in parallel using a shared set of oligos in different combinations. The pTP2_reporter example illustrates a chain of sequential CFs where the product of one becomes an input to another. A demonstration of running CFS on this example is available at cfs_experiment.mov.

C6-tools: Simplifying CF simulations and oligo design in Google Sheets.

The Java implementation of CFS is well-tested, reliable, and effective for validating correct CF. However, students have found it somewhat challenging to identify errors in the CF. The log file details all events, which, while helpful in pinpointing errors, can result in a complex interaction akin to code debugging. Typically, we run simulations through the IDE and leverage its debugging tools. A significant part of this challenge stems from the lack of visual representation during simulation. Although this issue could be addressed by creating more graphical user interfaces, this also presents another learning hurdle.

Driven by these usability concerns, we ventured into developing a variant of CFS, named C6-Tools [17], using Google Apps Script (JavaScript) within a Google Sheet. In addition to simulation functions, we also integrated algorithms for oligo design. Displaying individual design and simulation events in a 2D spreadsheet grid significantly simplifies the visualization of ongoing operations and error identification. Additionally, this interface is highly familiar and requires little explanation for new users and can be easily accessed via url. While C6-Tools offers a lower entry barrier compared to CFS, it is a newer tool and has not been as extensively tested.

Initially, our aim in developing C6-Tools was to leverage GPT-4 to automatically translate the Java code into Apps Script. However, the majority of the functions proved this task to be not as straightforward. We were unsuccessful directly converting our Java classes into Apps Script with a single prompt as illustrated in the supplementary chat wherein we attempted to convert the Gibson algorithm (java_simulator_conversion.html). The task is complicated by the length of the algorithms, but also the complexity and nuance of the logic. One notable complication was the Sheets’ inability to accept objects as cell values, necessitating their additional management as JSON. Despite these hurdles, ChatGPT was instrumental in facilitating this process with extensive prompting and revision.

Given the prevalence and versatility of Gibson Assembly and Golden Gate cloning in modern genetic engineering, traditional methods like digestion and ligation have become less relevant. Therefore, we decided not to include support for these older ’cut and paste’ methods in C6-Tools. Additionally, this led to significant simplification of the code. Though Polynucleotide and its tracking of end chemistry is needed to simulate separate digestion and ligation steps, it is not needed to accurately simulate PCR, Golden Gate, and Gibson methods which can be well handled by simple sequence strings. Nevertheless, we include a class definition for Polynucleotide as an option for future development in JS.

Nonetheless, constraints such as the lack of a testing environment, the inability to import libraries, and the non-portability of the code hamper further development of C6-Tools in its current form. In the case of Apps Script, the scripts must be either within the Sheet file, requiring duplication, or transcluded from a library with a complete wrapper. This quality of Apps Script limits the ability to maintain and extend the library. However, for users who may wish to customize their own version of C6-Tools, the independence offered by the current implementation may be preferable over a development team’s oversight.