Data loading and pre-treatment

We propose two input files format: either plain text files or HDF5 data, the latter being an open-source file format with advanced database features. As experimental acquisitions are prone to multiple component noise, we provided, as an option to the analysis workflow, smoothing (Savitzky-Golay method) and median filter functions, to exclude outliers. The input and output signals are interpolated to a chosen time interval (Δt). Both signals can be cut between two given time points to study continuous recordings while computing TFs on chunks of signal (Fig 1C). As an option, boxcar function of variable duration can be used in place of the input signal. Note that Iliski was not coded to handle complex-valued signals.

TF computation options

Two main types of TF computation are proposed: deconvolution or function optimization. The former is straightforward, either Toeplitz or Fourier deconvolution, and does not require any specific settings. The latter is the optimization of a parametric function, which requires further settings depending on the chosen algorithm. Beside the proposed fitting functions, the users can input their own function in the graphical interface or add it to the default ones by modifying a text file (the procedure is described in the Iliski Manual). The TF dimension is user-defined, with setting of the TF duration and ‘Sampling Time’ parameters that match the original data or can be augmented by non-linear interpolation. Optimization of parameters can be done with various Matlab algorithms, each coming with pros and cons (see Results section) (Fig 1C, middle).

Evaluation of the TF accuracy

A TF is evaluated comparing its prediction–the convolution of the input signal and the TF—to the expected output. Two metrics are used in Iliski: the Pearson coefficient (corrcoef function, Matlab, Fig 1C, right) and the residual sum of squares. The former was chosen to have a metric solely focusing on the dynamic, allowing for inter-subject comparisons, while the latter evaluates the overall fit, considering the amplitude of the prediction. The cost function of all the optimization algorithms tested in this article is the residual sum of squares (hereinafter referred to as "residuals").

Post-computation

The results structure is arranged to be as informative as possible while avoiding useless repetition of data. Iliski allows for loading previously computed results structures to check them again. After TF computation, results structure can be saved either as XLS file, readable by any Excel-like software, or as a MAT-file (MATLAB formatted binary file format), but it is also available in Matlab workspace to be exported in various data formats by the user.

Implementation

Iliski is accessible both as a GUI and as a set of functions to be used in scripts. It has been developed using Matlab R2018a, with the following dependencies: Optimization Toolbox, Signal Processing Toolbox and Global Optimization Toolbox.

Common user errors are thoroughly prevented by various messages and fail safes. In parallel, all errors are treated and saved in a log file, to allow for efficient bug-fixing by any developer. We purposely kept just a few parameters to modify through the GUI, with the goal of providing an easy-to-use tool for people not used to these functions. In most cases, Matlab default parameters of each deconvolution/optimization function worked well with our data, and we believe that it can be extended to many biological datasets. However, a user skilled with Matlab and optimization algorithms can easily modify the parameters of each function used.

Animal research

This study uses already published data of animal experimentation (Aydin et al.). All animal care and experimentations were performed in accordance with the INSERM Animal Care and Use Committee guidelines (protocol numbers CEEA34.SC.122.12 and CEEA34.SC.123.12).

Choosing the best TF computation approach

Fig 2 shows TF computation with different settings over the same couple of signals: neuronal (Ca²⁺) activations and vascular (red blood cells velocity) flow increases recorded in a mouse upon odor application. Our example data display unavoidable and complex noise coming from many sources: the biological system, the optical setup, the electronics, etc. Deconvolution with Fourier or Toeplitz approaches predicts the vascular responses very well for a given data set. However, the high-frequency noise is amplified by deconvolution [24] and transmitted to the TF, the predictions are not robust across data sets and the actual dynamics of neurovascular coupling is completely hidden in the TF noise (Fig 2). In this example, we show what we regard as a typical case of overfitting. The TF is capturing the high frequency noise of the system because it does not have any previous expectations for the shape of the relationship between the input and the output signals. This contrasts to the optimization of a parametric function approach which, although it imposes constraints on the shape of the TF, gives meaningful neurovascular relationship and does not need noise clearing. In blood flow-based neuroimaging, the standard function used to represent neurovascular coupling is composed of one or two Γ functions, depending on the nature of the signals, i.e. purely vascular or based on oxygen level[30].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Comparison of deconvolution and optimization algorithms on a batch of data.

Odor stimulation elicited a neuronal response in the Olfactory Bulb of a mouse, reported by a calcium-dependent fluorescent signal (in blue, left panel), providing the input of TF computation. Output is given by the vascular response, measured as the change in speed of red blood cells flowing inside a capillary proximal to the recorded neuronal activation (in yellow, right panel). Both experimental data have been resampled at 50ms and used to compute a set of TFs (in orange) either with direct deconvolution approaches (Fourier or Toeplitz methods, middle-upper panel TFs) or with 1-Γ function optimization performed by 3 different algorithms (middle-lower panel TFs). Complex TFs bring accurate prediction but amplify the noise of the data used to deconvolve them, with a consequent loss of robustness on other datasets. Smoother TFs are less accurate on the training dataset, but much robust when applied to test datasets.

https://doi.org/10.1371/journal.pcbi.1008614.g002

Below is the one Γ-driven function we used with our data.

Where p₁,.., p₄ are the parameters to optimize, and H is the Heaviside function that includes a time-shift parameter (p₃), In some cases, the time shift significantly improved the prediction and is a known biological phenomenon to consider[31]. Its four parameters are not all independent from one another, e.g., p₁, p₂ and p₄ all impact the TF amplitude. This inter-dependency between the parameters brings an ill-posed optimization problem with multiple local minima of the cost function, the sum of the residual squares, in the 4D space of the parameters. To tackle the function optimization problem, we chose standard algorithms (all implemented in the Optimization Toolbox of Matlab) to encompass the main available options.

A derivative-free optimization method [6] is provided by the fminsearch function in Matlab, which uses the Nelder-Mead simplex algorithm. This approach on our data produced a TF with more than one underivable point that is not representative of the smooth dynamic of neurovascular coupling.

Another common option is provided by Quasi-Newton optimization algorithms, which uses an approximation of the Jacobian: for this approach too, we tested an unconstrained built-in method (fminunc function, Matlab). This prediction is, overall, as good as with fminsearch (Fig 2, Pearson coefficients, fminunc vs. fminsearch: 0.95 vs. 0.96), but the onset phase is not properly fit. Moreover, although not evident from the plot, the optimized time shift was negative (-120 ms), implying that the onset of the vascular response precedes the neuronal activation.

All the optimization methods tested above are deterministic, meaning that repeating them with the same initial parameters will bring the same result. The pitfall of these methods when applied to ill-posed problem is that optimization process will get attracted to the nearest local minimum, regardless of the many other deeper minima, which may be far away in the parameters space. In other words, deterministic algorithms are sensitive to the initial parameters set before starting the optimization.

Non-deterministic algorithms exist to overcome the local minimum issue, adding some level of randomness in the optimization process, and for this purpose Iliski uses the Simulated Annealing algorithm. Each optimization run can yield a different result, reaching possibly a different cost function’s minimum each time. We define as ‘run’ a single application of the optimization with a given set of initial values, and ‘iteration’ the ensemble of runs sharing the same initial values. By running the algorithm multiple times, one can choose the result with the lowest residual, while avoiding TFs which shape are biologically not acceptable. In fact, to represent the NVC, a TF cannot start at 0 sec, because of the delay due to the cellular cascades triggering the vascular response, and it must be smooth to comply with the progression of biological processes. In Aydin et al. (2020), we described a workflow of runs and iterations to get to biologically consistent TFs (see Supplementary Fig 1 in Aydin et al. [25]). To speed up computation, we imposed bounds over the parameters. Note that such bounds can be set through the Iliski GUI for any constrainable algorithm.

Using our data, Simulated Annealing gave a smooth TF and a prediction as good as fminsearch for the onset phase of the vascular response. The data shown in Fig 2 is representative of the rest of the data. In fact, optimization of TFs using neural and vascular recordings from other mice, tested with the same odor stimulation, produced similar residual values of the cost function across the 3 optimization algorithms presented above (1-way ANOVA, F(2, 17) = 0.035, p = 0.97, Fig 3A). However, as in the example of Fig 2, deterministic algorithms are prone to biologically inconsistent TFs (Fig 3B). The non-deterministic, Simulated Annealing algorithm with subsequent iterations method allows to efficiently exclude these TFs and obtain the best trade-off between prediction performance and biological consistency at the cost of a longer computation time. Direct deconvolution is a good option when the goal is the prediction quality within the training database. Deterministic optimization algorithms are fast but yield to TFs that may have biologically inconsistent dynamics. Note that for all the computations we used a short Δt (50 ms) for interpolation to preserve most of the information.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Prediction performance of different optimization algorithms.

(A) 3 algorithms were compared in terms of the residuals of the cost function of the optimized TF on 7 mice datasets (Derivative free algorithm failed in optimizing a TF in a mouse). No significant difference was found across the 3 methods. (B) However, simulated annealing was the only approach to provide TFs consistent with the nature of biological data (TF with no more than 1 non-derivable point), while both the other deterministic methods run into inconsistent TFs in roughly 60% of the cases.

https://doi.org/10.1371/journal.pcbi.1008614.g003

Evaluating the number of runs in a non-deterministic case

As already mentioned, the Simulated Annealing algorithm requires several runs and iterations to obtain a good TF, where ‘run’ means a single optimization and ‘iteration’ an ensemble of runs sharing the same initial values of the fitting function. In our experience, starting the optimization with a ‘bad’ TF—whose shape is different from what is expected for the processed dataset—helps to collect more local minima in a pool of optimization runs. For example, in our previous study [25], we proposed iterations of 50 runs and started with the initial values of the standard TF (one Γ HRF) which, peaking at 5 seconds, turned to be much slower than any of the optimized TFs. The sequence of 50-runs iterations stopped when, within an iteration, no clear improvement was found in the optimized TF [25]. On average, 2 iterations were sufficient to get a stable TF with Pearson coefficient above 0.9. Here, we investigated if a higher number of runs is beneficial to the detection of the minimum of the cost function and if it prevents the need for further iterations. We compared 50 and 200 runs with single and double iterations, in cascade (Fig 4A). In a mouse dataset, we observed a non-significant trend towards more scattered TFs shapes for computation using 50 runs versus 200 runs (1-way ANOVA, F(3, 16) = 2.086, p = 0.14, Fig 4B). Similarly, the quality of the TFs did not significantly improve with increasing runs (1-way ANOVA, F (3, 16) = 2.299, p = 0.12). As a result, TFs with fast dynamics (peaking within 1 and 2 sec), was a common feature independently of the adopted protocol (Fig 4C). In a dataset from another mouse (Fig 4D and 4E), TFs with sparse time to peak values after 200 runs improved after a second iteration, with the same number of runs (2.3 ± 0.3 s VS 1.5 ± 0.1 s (mean ± SEM), two-tailed T-test, unpaired, p = 0.02 < 0.05). Note that this compression of TF dynamics was not accompanied by a significant improvement of the TF quality (residuals: 10.1 ± 2.1 vs. 6.9 ± 0.8 (mean ± SEM) for 200 and 200 + 200 runs respectively, two-tailed T-test, unpaired, p = 0.19). To conclude, depending on the input/output signals, non-deterministic algorithms can produce TFs with different dynamics but close performances in the prediction. The choice of a specific optimization process, with more or less iterations and runs, becomes crucial when the interest is not limited to the prediction quality, but extends to the temporal dynamics of the TF. Because of the noise, TFs with distinct shapes can yield very close residual values.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Influence of the number of runs and iterations on the TF shape and quality.

(A) Using the Simulated Annealing algorithm, we tested 4 protocols of 50 or 200 optimization runs, either done a single time or repeated (5 TFs computed for each protocol). (B) Residuals of the cost function do not significantly differ across the protocols, although the protocols with the highest number or runs show a trend of smaller residuals. (C) Similarly, there was no significant difference for TFs time-to-peak values. (D, left) Same protocols comparison on a dataset from a different mouse revealed a sparse dynamic of optimized TFs, even if the best TFs were selected on a pool of many TFs (200 runs). (D, right) A second iteration of 200 runs gave more homogeneous TFs dynamics. (E) Quantification of the dynamic heterogeneity was made by measuring the time-to-peak which resulted in a scattered distribution for the 200 runs protocol, packed up repeating the same iteration a second time. Residual values, not reported in this figure, were not significantly different for mouse 1 and 2.

https://doi.org/10.1371/journal.pcbi.1008614.g004

Guide to choose the algorithm best fitting your needs

We provide a decision diagram to choose the best approach to compute a TF based on the features of the user’s dataset (Fig 5). Nonetheless, we believe it is always a good choice to test different approaches before making the final choice.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Decision tree to help choosing the most efficient method to compute a TF with Iliski, based on the data features.

https://doi.org/10.1371/journal.pcbi.1008614.g005

Availability and future directions

Iliski is open-source and freely available under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Iliski is maintained on GitLab, enabling user-friendly bug report and community work to make the tool fit the users’ need. It can be found here: https://github.com/alike-aydin/Iliski.

In the neurovascular imaging field, computing the hemodynamic response function is paramount to interpreting vascular activation in terms of neural activation. In any other field, computing TFs may be of help to go deeper in the interpretation of the results. For these reasons, we think it is extremely helpful to have a data analysis tool which lets fast testing of different algorithms with a user-friendly interface.