^{1}

^{2}

^{¤a}

^{2}

^{3}

^{2}

^{4}

^{*}

^{¤b}

The authors have declared that no competing interests exist.

Conceived and designed the experiments: HL FJVC. Performed the experiments: AS. Analyzed the data: FJVC HL AS. Contributed reagents/materials/analysis tools: FJVC HL. Wrote the paper: AS FJVC HL. Concept of functional inference: HL FJVC. Designed the software for inference: AS.

Current address: IGM, RWTH Aachen, Aachen, Germany.

Current address: Department of Biomedical Engineering & Division of Biokinesiology and Physical Therapy, University of Southern California, Los Angeles, California, United States of America.

In systems and computational biology, much effort is devoted to functional identification of systems and networks at the molecular-or cellular scale. However, similarly important networks exist at anatomical scales such as the tendon network of human fingers: the complex array of collagen fibers that transmits and distributes muscle forces to finger joints. This network is critical to the versatility of the human hand, and its function has been debated since at least the 16^{th} century. Here, we experimentally infer the structure (both topology and parameter values) of this network through sparse interrogation with force inputs. A population of models representing this structure co-evolves in simulation with a population of informative future force inputs via the predator-prey estimation-exploration algorithm. Model fitness depends on their ability to explain experimental data, while the fitness of future force inputs depends on causing maximal functional discrepancy among current models. We validate our approach by inferring two known synthetic Latex networks, and one anatomical tendon network harvested from a cadaver's middle finger. We find that functionally similar but structurally diverse models can exist within a narrow range of the training set and cross-validation errors. For the Latex networks, models with low training set error [<4%] and resembling the known network have the smallest cross-validation errors [∼5%]. The low training set [<4%] and cross validation [<7.2%] errors for models for the cadaveric specimen demonstrate what, to our knowledge, is the first experimental inference of the functional structure of complex anatomical networks. This work expands current bioinformatics inference approaches by demonstrating that sparse, yet informative interrogation of biological specimens holds significant computational advantages in accurate and efficient inference over random testing, or assuming model topology and only inferring parameters values. These findings also hold clues to both our evolutionary history and the development of versatile machines.

In science and medicine alike, one of the critical steps to understand the working of organisms is to identify how a given individual is similar or different from others. Only then can the specific features of an individual be distinguished from the general properties of that species. However, doing enough input-output experiments on a given organism to obtain a reliable description of its function (i.e., a model) can often harm the organism, or require too much time when testing perishable tissues or human subjects. We have met this challenge by demonstrating that our novel algorithm can accelerate the extraction of accurate functional models in complex tissues by continually tailoring each successive experiment to be more informative. We apply this new method to the problem of describing how the tendons of the fingers interact, which has puzzled scientists and clinicians since the time of Da Vinci. This new computational-experimental method now enables fresh research directions in biological and medical research by allowing the experimental extraction of accurate functional models with minimal damage to the organism. For example, it will allow a better understanding of similarities and differences among related species, and the development of personalized medical treatment.

Much attention is given to functional networks (e.g., scale-free, small world and others) resulting from the complex interactions between their constituents [e.g., 1–5]. For example, the mechanisms of module assembly in biological molecular networks

Tendon networks at anatomical scales are intricate and poorly understood componentsof the neuromuscular control of the hand. Understanding their functional characteristics is critical to gaining insight into the brain-body co-evolution that has facilitated dexterous manipulation in modern humans, as well as improving clinical rehabilitation strategies in orthopedic and neurological conditions. The complexity of tendon networks of the fingers is legendary, and thus the so-called Winslow's rhombus is a generic topological approximation that has been widely adopted since the 18^{th} Century as proposed by the famous Danish-born anatomist J.B. Winslow in 1732

This work is enabled and motivated by our earlier work in

We describe (i) the development and implementation of the estimation-exploration algorithm for arbitrary networks of strings, and (ii) its application to the experimental inference of two known networks made of synthetic Latex, and one extensor mechanism network excised from the middle finger of a cadaveric human hand.

The key concept in the estimation-exploration inference process (

Steps (a)–(c) are performed cyclically until a termination criterion is met after which the best model (d) is chosen. (c) represents the evolved models from step (b) being actuated by a random set of input forces (or tests) shown using arrows. The length of each arrow represents the magnitude of the force applied. The actuation directions may or may not be held fixed (here, shown fixed). The actuation force set that generates the maximum discrepancy in the force-motion response between the evolved models is chosen to perform the experiment in (a).

We first describe the modeling processes for Stages II and III, which are identical for the synthetic Latex and cadaveric networks because they are both modeled as networks of elastic strings. Details of how we measureand apply input/output data to generate load sets in Stage I forboth experimental systems are described thereafter. We assume that the mechanical function of the target network system is well approximated by a model of discrete, interconnected/sliding strings. We further assume that all the input and output nodes are accessible for otherwise hidden networks, as is the case for the synthetic and anatomical networks tested. The state of a hidden network is described by a set of actuating forces at the input nodes, those resulting at the grounded output nodes, and the distances between the input and output nodes after the network attains equilibrium. Input and resultant forces comprise the load set while the inter-nodal distances comprise the deformation data. Both force and deformation data are used to evolve the models to best explain a network since ignoring either information may result in more number of experiments (in Stage I), see supporting information S1.

We evolve a population of models (both topology and their parametric attributes) bottom-up from a primordial mesh of strings (

Length of each string behaves as a topological and parametric design variable. For the informative equilibrium configurations that the mesh observes, if some strings remain slack, they are eliminated from the topology. Only taut strings in some or all configurations get retained. Length and cross sections of strings are evolved as design variables. (b) Each taut string in the parent mesh is modeled as a large deformation truss finite element. Slack strings do not support the external loads. Analysis performed is geometrically (for synthetic targets) and materially (for the finger extensor mechanism) nonlinear and equilibrium is achieved through efficient, quadratically convergent Newton-Raphson iterations.

Stage II assumes access to the most informative database of load sets, and consists of four parts, (i)

_{o} and cross section area _{i}_{i}_{j}_{j}_{1}, _{2}) and (_{3}, _{4}) as shown. The new string length is_{i}_{j}_{o}, since the string collapses in compression, both _{i}_{j}_{e}_{e}

_{R}_{DS}_{R}_{D}_{D}

Although comparing the topologies of the evolved models with the known target network is a secondary (structural) fitness criterion, this is only possible with the synthetic Latex networks.

_{new}_{U}−_{L})_{1}+_{2}) represents its altered or mutated value about _{new}_{L}and _{U} are the specified lower and upper bounds on _{1} and _{2} are chosen such to encourage both small and large changes. For all the experiments performed, _{1} = −8 and _{2} = 9 are used for which, changes in ^{−4}(_{U}−_{L}) and 2.72(_{U}−_{L}). If _{new}_{L}, _{new}_{L}. Otherwise, if _{new}_{U}, _{new}_{U}. Changes in free parameters are performed as follows. If a uniformly distributed random number _{rate}_{rate}_{new}_{L}_{L}_{L}_{L}

A few additional evolutionary strategies are also used so that the models can evolve better and faster. In the first, very small changes (<1%) in the free parameters are performed but with a higher rate (_{rate}_{rate}

After the free parameters of a chosen number of models are evolved using the informative input-output data (Stage II,

Once the population of models has evolved to explain the previous informative datasetsas per the termination criteria (Stage II,

Ideally, conducting the inference process in real time requires the availability of a large parallel computing network to evolve the models and the most informative actuation tests shortly after each new load set is added to the database. Because this is not feasible (see discussion) with manual application of loads and the need to test perishable tissue, we did the next best thing: collected the experimental data for the Latex and tendinous networks in dedicated experimental sessions—and ran the algorithm off-line using those data sets. Whenever the algorithm requests the next most informative test (found as described below), we provide the load set corresponding to the nearest neighbor (in the least squared sense) to it from the available load sets.

In our experiments, the direction of each test (input force) is fixed in the global frame, and we therefore only evolve their magnitudes. These magnitudes _{new}_{new}_{new}_{L}_{U}_{new}_{L}_{U}_{DS}_{R}_{kj}_{D}_{kj}

As mentioned above, conducting the inference process in real time requires the availability of a large, parallel computing network to evolve the models and the most informative actuation tests shortly after each new load set is added to the database. Because this is not feasible for this first study, we collected data for both Latex and tendinous networks in dedicated experimental sessions—and ran the algorithm off-line using those data sets. In all three experiments (inference of two Latex and one tendinous networks), we collected 72 input-output data sets in response to 72 different combinations of tests in random order to prevent any bias due to loading order (see details in Stage I below). These tests were designed to be evenly distributed in the 3D input space: The input nodes were tugged with combinations of four force levels of 1.25 N, 2.50 N, 3.75 N and 5.00 N and the resulting static force-deformation responses were measured for a total of 64 load sets (4^{3} = 64). Additionally, we interspersed eight more load sets for cross validation. Here, tensions of 1.9 N and 4.4 N (2^{3} = 8 combinations) were employed to pull each input string.

All tests were chosen to amply deform the cadaveric network and yet be conservative to not tear the collagenous cadaver tissue. Further, by introducing the tests used for cross validation uniformly between those used to generate the training set, we ensured that the cross-validation load sets were distinct from any training load set.

For all three networks, the experiments for data collection lasted about 8 hours, which is well within the time frame for studies of cadaveric tissue

The collection of the 72 load sets allowed us to execute and validate the inference algorithm off-line. Whenever the algorithm requested the next most informative test, we provided the load set corresponding to the nearest neighbor (in the least squared sense) to it from the 64 training load sets. Within each cycle of the estimation-exploration algorithm, after the models were evolved, their cross-validation was performed with 8 additional load sets.

In Stage I, we first picked a single load set from the experimental data at random to seed the database. This load set formed the initial database used in Stage II (

We inferred two arbitrary networks: one resembling the three letters ‘AFH’ fused together (

The grounded nodes are shown using squares and the actuation forces are depicted using the arrows. Each model is evolved until they see 20 informative experimental data sets. (a) The ‘AFH’ target. (b) adapted Winslow's Rhombus or ‘aWR’ target. This is an adaptation of the Zancoli's representation

Both networks had five input-output nodes, each tied to a tether attached via an alligator clip to a swiveling, grounded electronic dynamometer with a 2

(a) The interpretation by Zancoli

When loading the ‘AFH’ and ‘aWR’ networks, the location of the input nodes was allowed to change as the networks deformed, but the lines of action were maintained constant. For the ‘AFH’ network, the directions for the tests were fixed along −135°, −90° and −45°, respectively from the horizontal at the left, center and right bottom input nodes (see

As in our prior cadaveric work (e.g.,

Synthetic Target | Model Number | Training set error (%) | Cross validation error (%) |

Models inferred from the ‘AFH’ target ( |
(i) | 2.9 | 4.2 |

(ii) | 3.3 | 5.0 | |

(iii) | 2.8 | 3.6 | |

(iv) | 2.8 | 4.7 | |

(v) | 2.8 | 4.5 | |

(vi) | 3.0 | 5.7 | |

(vii) | 4.0 | 5.2 | |

(viii) | 2.6 | 6.1 | |

Models inferred from the ‘aWR’ target ( |
(i) | 2.6 | 5.5 |

(ii) | 2.4 | 4.5 | |

(iii) | 2.7 | 5.5 | |

(iv) | 3.7 | 5.3 | |

(v) | 2.3 | 5.8 | |

(vi) | 2.6 | 6.1 | |

(vii) | 3.3 | 5.6 | |

(viii) | 3.6 | 6.4 |

Likewise, models functionally similar to the ‘aWR’ network differ in structure within

The number of functional evaluations required to infer the ‘AFH’ and ‘aWR’ networks is c. 0.7 vs. 0.5 million, respectively. The CPU times for model evolution and generation of the most informative tests on three different machines for the synthetic targets are presented in Supporting Information,

To confirm if evolving informative tests improves the inference process, we performed an additional three baseline inference runs for each target network using 20 random data sets. The

Training set (left column) and cross validation (right column) errors when (a) the ‘AFH’ target is inferred and (b) when the ‘aWR’ network is inferred. Solid lines represent errors with sequential informative experimental data when both topologic and parametric inference is performed. Dotted lines correspond to cases when random experimental data is sequentially employed for inference. Dashed lines show errors for only parametric inference with basic (

We also performed simple parametric fitting to infer the target networks in

In all cases, inferring the topology of the target network yields significantly better results (

Errors incurred during the inference of the finger extensor mechanism. (a) Training set errors. (b) Cross validation errors. Progression of the error values are depicted as the number of data sets are introduced for model evolution. Error values are depicted for five executions for the topologic and parametric inference using informative tests. Bars represent standard errors. Solid lines correspond to the mean error when informative data is employed in topological and parametric inference of the target. Dotted lines show mean errors when sequential random data sets are used (error bars depicted for 20 executions). Dashed lines represent mean errors (error bars depicted for 20 executions) when only parametric evolution is performed using the basic topology where in only 6 strings connect all accessible (input and output) ports and no other interconnection is allowed (

Model Number | Training set error (%) | Cross validation error (%) | |

Models inferred from the ‘Finger extensor mechanism’ ( |
(i) | 3.8 | 6.0 |

(ii) | 4.0 | 7.0 | |

(iii) | 5.8 | 6.9 | |

(iv) | 7.1 | 5.8 | |

(v) | 7.9 | 7.0 | |

(vi) | 5.8 | 7.0 | |

(vii) | 5.4 | 6.6 | |

(viii) | 4.2 | 7.2 | |

(ix) | 6.7 | 7.1 | |

(x) | 5.7 | 7.1 |

Therefore, we chose models (i) and (iv) from

(a) model (i) in

Lastly, similar to the Latex networks (^{th} data set is introduced. However, the trend in

We demonstrate, to the best of our knowledge, the first functional inference of a complex anatomical structure using sparse experimentation. Without reverting to exploratory dissection (which is disruptive) or structural imaging (which is expensive and not necessarily informative of mechanical interactions under loading), we infer functional structure of a biological tendinous network by co-evolving the models with informative tests. We began by validating and calibrating our novel methodology using two synthetic Latex target networks, and then applied our method to the real-world problem of inference of the functional structure of the tendinous extensor mechanism tissue excised from a cadaver hand. Notwithstanding (i) the specific optimization procedure (we used a stochastic hill climber search, which are hotly debated by its supporters and retractors, but any other optimization that is suitable to the problem at hand can be used), or (ii) our current inability to run the estimation-exploration algorithm real time (we collected the experimental data for the Latex and tendinous networks in dedicated experimental sessions—and ran the algorithm off-line using those data sets), our results clearly illuminate and demonstrate several important features and concepts about this approach. These include (i) the powerful utility of a novel, general purpose predator-prey estimation-exploration algorithm for topologic and parametric inference of physical systems, and (ii) the particular functional characteristic of our test system: the extensor mechanism of the fingers whose structure and function have been debated since at least the 16^{th} century.

In this first application of the predator-prey estimation-exploration algorithm

We remark that, for the tendinous specimen, we needed to extract the experimental data within the first 8 hours of its excision to avoid structural and/or material degradation. Evolving the models with the most informative tests, as predicted in simulation in stage III of the inference process, was not possible with the available computational resources; the overall inference process took much more than 8 hours. Rather, we sought to access the closest possible tests that were informative, if not the most informative, from the experiments performed a priori. We further remark that for both, synthetic and biological networks, generating the most informative test was not possible with the experimental setup used (

We also show that simultaneous topological and parametric inference yields better results than the parametric inference alone. Most system identification for biomechanical models is limited to parametric inference

A clear distinction needs to be made between topology and parameter values. Models are assembled by exploring the space of possible combinations of building blocks (in this case, strings and nodes). A specific model topology is a specific connectivity map among a specific set of building blocks (i.e., string connectivity). The model parameters are the individual properties of each building block (i.e., length of strings). In practice, however, it is necessary to “parameterize” the topology to be able to encode (i.e., represent) it so that an algorithm can search the topological space. In our case, we defined our primordial mesh of strings (

On the methodological side, several issues are key to accurate inference. These include accurate assumptions pertaining to (a) the material properties and the strain-deformation models, (b) informative experiments capturing full network functionality and interaction conditions, (c) automated experimental setup, (d) the primordial connectivity representation, and (e) the choice of the algorithm used for model evolution. If inappropriate assumptions are used, the experimental data may not be within the set of predictions of any feasible model topology or parameter values, or the candidate tests generated by the inference process may not be informative.

The inference of the extensor mechanism was also performed assuming constant elastic modulus (1 GPa) as opposed to nonlinear stress-strain relationship for tendons used from

First and third columns: Best models are evolved with linear elastic tendon properties (E = 1 GPa). The models should emulate the informative experimental data from the cadaver hand. String numbers shown with the evolved networks. Second and fourth columns: Cross validation errors (×100 for percentage) for corresponding models on the left. The first bar represents the error when the corresponding network on the left is intact. The following bars depict the error when the strings are snapped one at a time. The errors for the intact models are equal to or greater than 30%. From each model, elimination of certain strings lowers the cross validation errors suggesting that the intact models do not depict the functionality of the actual extensor mechanism accurately. If erroneous postulations are employed, the experimental data obtained by the inference process may not be informative after all.

All the string networks were inferred here using the large deformation-small strain assumption. Models may also be evolved using the Green-Lagrange large-strain

We inferred the hand extensor mechanism by considering only a part of its overall functionality. We were limited to testing the anatomical specimen as it lay flat on a hydrated surface, as opposed to wrapped over finger joints. Even so, all string modules, as described in the classical Winslow's model (

To reproduce the most informative tests recommended by Stage III of the estimation exploration algorithm, the setup should be fully automated and computer-controlled. Independent motors should be mounted with the respective input tethers wrapped tightly around them to control the tensions. In addition to using digital force scales, force sensors (e.g., strain gages) should also be used to control the motor rotations via a feedback loop. Achieving the most informative test with the automated setup (as opposed to manual) will be less painstaking and more accurate.

The Latex and biological specimens are inferred using the string model representations shown in Supporting Information,

All factors mentioned above, and the noise involved in experimental data influence the landscape of the objective function. In view of this, functionally similar but topologically and parametrically diverse models obtained through the Random Mutation Hill Climber (RMHC, a variant of Genetic Algorithm that employs only mutation) could all be local optima existing very close to the global one in the design space. Due to the noise present in the data, it is not expected for a global optimum to have significantly lower

One of the goals was to confirm whether the Winslow's Rhombus (

The predator-prey approach is an optimal experiment design (OED) method wherein through competition, most informative tests (optimal sample points) are generated to evolve the best models that explain these tests. However, this approach is unlike other OED methods, e.g., D-optimal, L-optimal and minimax-optimal wherein model parameters (or their functions) are determined by minimizing, for instance, generalized variance. In Bayesian type OEDs, prior information on model parameters is assumed.

With regard to the study of complex biomechanical structures by anatomists, biomechanists and biologists, most previous work has naturally focused on inferring the structure of the tendinous networks via dissection or imaging. In contrast, we interrogate biological networks through a non-invasive computational machine learning procedure. Invasive techniques may damage the tissue, while imaging methods may miss critical functional interactions (e.g., seen only under specific loading conditions). Our non-destructive inference method yields both topological and parametric information. For example, the specific number and lengths of the tendinous members of the extensor mechanism affect the distribution of tension to the finger joints

This raises the important issues of uniqueness and observability, which are central to computational model inference—and critical in the context of biological populations that naturally exhibit anatomical variability. Some 2D sub-topologies can be equivalent to each other under certain parameter and loading sets

From the computational perspective, our use of populations of models forrandom mutation based hill climber search is very much conducive to the maintenance of model diversity (i.e., alternative hypotheses) to understand the uniqueness and observability of model topologies. This allows our search in this large dimensional space to proceed along multiple alternative paths that do not favor any particular local minimum. In all of our results with synthetic or anatomical networks, we find families of solutions: multiple different, yet functionally similar, topologies. This suggests that (i) additional data are necessary to further constrain the search, (ii) that different functional domains (such as deformation during finger flexion) are necessary to make the differences across various models observable, or (iii) that there are indeed functionally similar implementations for the domain of behavior that we studied (load transmission in this case). The latter idea is quite intriguing from the evolutionary perspective as it agrees with the well-documented natural variability in the gross anatomical structure of the extensor mechanism across humans

From the clinical perspective, damage to this network can cause severe dysfunction of manipulation (e.g., mallet finger, swan-neck or boutonnière deformity

We establish here the information mode – whether force or motion (distances between the accessible ports) data, or both will be required through a data set for inferring the functionality and physical manifestation of the target network. We illustrate by inferring two ‘simulated’ targets, the ‘A’ network and a minor adaptation of the Winslow's Rhombus (

(TIFF)

Respective representative member string fabrics used to infer the ‘AFH’ and ‘aWR’ targets (^{2}. In (b) 54 member strings are employed. Length bounds used are [0.01 70] mm and cross sections are evolved within the same limits. (c)–(d) present respective basic topologies used to parametrically fit the informative force-motion experimental data from the ‘AFH’ and ‘aWR’ targets. (e) Representative member string fabric used to infer the finger extensor tendinous network. Seventy-one member strings are used. For each member string, lengths were evolved with the limits of 0.01 mm and 33 mm respectively while their cross sections were bound within [0.01 11] mm^{2}. The upper bounds on member string lengths and cross sections were chosen based on the overall length of the tissue (

(TIFF)

The Random Mutation Hill Climber Algorithm (RMHC) implemented in the predator-prey strategy is capable of yielding solutions that are close to a global optimum if adequate computational resources are provided. Minimization of ^{2})(1−½ sin(4_{i}_{i}_{i}_{i}

(TIFF)

Training and cross-validation errors when the estimation-exploration algorithm is performed using the Winslow's Rhombus (

(TIFF)

CPU Time (seconds) for model evolution (8 models) and generation of the most informative test for the ‘AFH’ synthetic target.

(DOCX)

CPU Time (seconds) for model evolution (8 models) and generation of the most informative test for the ‘aWR’ synthetic target.

(DOCX)

Assistance by Dr. S. S. Roach, Manish Kurse, William Pollachek, William Seidel and Lisa Toth in conducting the experiments is sincerely acknowledged.

^{rd}, et al