^{1}

^{2}

^{1}

^{2}

^{1}

^{*}

The authors have declared that no competing interests exist.

Conceived and designed the experiments: NCB OHB DHB GKA. Performed the experiments: NCB OHB. Analyzed the data: NCB. Wrote the paper: NCB GKA.

Several domains of neuroscience offer map-like models that link location on the cortical surface to properties of sensory representation. Within cortical visual areas V1, V2, and V3, algebraic transformations can relate position in the visual field to the retinotopic representation on the flattened cortical sheet. A limit to the practical application of this structure-function model is that the cortex, while topologically a two-dimensional surface, is curved. Flattening of the curved surface to a plane unavoidably introduces local geometric distortions that are not accounted for in idealized models. Here, we show that this limitation is overcome by correcting the geometric distortion induced by cortical flattening. We use a mass-spring-damper simulation to create a registration between functional MRI retinotopic mapping data of visual areas V1, V2, and V3 and an algebraic model of retinotopy. This registration is then applied to the flattened cortical surface anatomy to create an anatomical template that is linked to the algebraic retinotopic model. This registered cortical template can be used to accurately predict the location and retinotopic organization of these early visual areas from cortical anatomy alone. Moreover, we show that prediction accuracy remains when extrapolating beyond the range of data used to inform the model, indicating that the registration reflects the retinotopic organization of visual cortex. We provide code for the mass-spring-damper technique, which has general utility for the registration of cortical structure and function beyond the visual cortex.

A two-dimensional projection of the visual world, termed a retinotopic map, is spread across the striate and extra-striate areas of the human brain. The organization of retinotopic maps has been described with algebraic functions that map position in the visual field to points on the cortical surface. These functions represent the cortical surface as a flat sheet. In fact, the surface of the brain is intrinsically curved. Flattening the cortical surface thus introduces geometric distortions of the cortical sheet that limit the fitting of algebraic functions to actual brain imaging data. We present a technique to fix the problem of geometric distortions. We collected retinotopic mapping data using functional MRI from a group of people. We treated the cortical surface as a mass-spring-damper system and corrected the topology of the cortical surface to register the functional imaging data to an algebraic model of retinotopic organization. From this registration we construct a template that is able to predict the retinotopic organization of cortical visual areas V1, V2, and V3 using only the brain anatomy of a subject. The accuracy of this prediction is comparable to that of functional measurement itself.

The human occipital cortex contains multiple representations of the visual field, starting with primary visual cortex (V1; also called striate cortex). V1 lies primarily within the calcarine sulcus and represents the contralateral visual hemifield. The cortical surface dorsal and ventral to V1 contains the neighboring extrastriate regions V2 and V3, each of which represents a complete visual hemifield that is split into the upper visual quarterfield, ventral to V1, and the lower visual quarterfield, dorsal to V1. These three distinct retinotopic maps are organized on the cortical surface by distance from the fovea (eccentricity) and angle from the vertical meridian (polar angle)

(

This visual area organization is readily demonstrated in people by performing retinotopic mapping using functional magnetic resonance imaging (fMRI). When examined in a population of subjects, the qualitative topographical organization of V1–V3 has been found to be consistent

The functional organization of retinotopic cortex may be captured in an algebraic model. Early algebraic models of V1–V3 used a log-polar transform to relate visual field position to location on the flattened cortical surface

In the limited case of area V1, which resides in a single sulcal fold, we have shown that an algebraic model can be fit to retinotopic mapping data on the flattened cortical surface

Here we provide a means to link fMRI data from visual areas V1–V3 to an algebraic 2D model of retinotopy in the presence of geometric cortical distortion. One might first consider solving this challenge by modifying the algebraic model to better match the data. Mathematically, however, it is both difficult and poorly descriptive of the fundamental structure of retinotopic organization to tailor a 2D model to the local distortions in geometry present in flattened cortical data. Instead, we propose to register the functional data of the flattened cortical surface to the algebraic model. Such a technique distorts the flattened cortical representation to align the functional data to the algebraic model and is thus flexible enough to correct the geometric distortions introduced by flattening. In this approach, the challenge becomes devising a registration technique that is flexible enough to correct the undesired distortions and adequately align to the algebraic model yet sufficiently constrained so that the resulting, registered anatomy retains its structure enough to support generalization of the algebraic model beyond the extent of the data used in the registration.

Mass-spring-damper (MSD) systems are commonly used in the simulation of the deformation of materials and objects

Similar to our prior work in V1, we obtain across-subject retinotopic mapping data that is then aggregated within a standard cortical surface atlas

The result of the MSD simulation maps individual vertices within the cortical surface atlas to a specific visual area and visual field position. We show that this mapping may be used to accurately predict the retinotopic organization of extrastriate cortex in a novel subject who's brain anatomy is brought into register with the cortical surface atlas. Further, because the algebraic model is continuous, we find that the mapping may be used to accurately predict retinotopic data collected from beyond the eccentricity range of data used in the aggregate to derive the mapping.

This study was approved by the University of Pennsylvania Institutional Review Board, and all subjects provided written consent.

A total of 25 subjects (15 female, mean age 24, range 20–42) participated in fMRI scanning experiments. All subjects had normal or corrected-to-normal vision. Experimental data from all subjects have been reported previously

The first dataset, D_{10°}, contained 19 subjects all of whom were scanned for 27 minutes using a sweeping bar stimulus that extended to 10° of eccentricity within a central 20° aperture. The bar stimulus consisted of a single sweeping 2.5°-thick bar that flickered at 5 Hz

The second dataset, D_{20°}, contained 6 subjects. Subjects fixated on either the left or right edge of the screen for 64 minutes while 16 iterations of standard “ring and wedge” stimuli swept in the periphery

BOLD fMRI data (TR = 3 s, 3 mm isotropic voxels) and anatomical images (T1-weighted, 1 mm isotropic voxels) were collected at 3 Tesla. The FMRIB Software Library (FSL) toolkit (

For subjects in the D_{10°} dataset, a population average hemodynamic response (HRF) _{20°}, a subject-specific HRF was derived from a separate blocked visual stimulation scan. Global signal, cardiac and respiratory fluctuations (when available) _{10°}) or by identification of the peak of a Gaussian fit to the weights of a set of finite impulse response covariates (dataset D_{20°})

Aggregate retinotopic maps of each dataset were produced separately for polar angle and eccentricity by finding the weighted mean polar angle and eccentricity of all subjects at each aligned vertex position. Mean polar angles and eccentricities were weighted by the _{q}_{•Q} ^{2})/(Σ_{q}_{•Q}

All vertices within _{0}, defined as the most anterior point on the anatomically defined V1 border _{0} lay at the intersection of the equator of the spherical _{0}. A shear transformation, present also in our previous treatment of V1

Data from D_{10°} were registered to a modified version of the banded double-sech model proposed by Schira _{0} (described above) on the ^{2}. These “anatomical springs” ensured that warping introduced during the simulation would respect anatomical constraints. Additionally, for each vertex with an above confidence threshold assignment of eccentricity and polar angle in the dataset aggregate, a “model spring” with one fixed and one free end was connected between the vertex (free end) and the position predicted by the algebraic model for the aggregate observed polar angle and eccentricity of the vertex (fixed end). Because there are multiple such points (^{2}. To prevent vertices distant from the algebraic model but with polar angle and eccentricity assignments nonetheless above our _{0} is _{0}|; for the endpoint of a model spring, the magnitude of the force is 4_{0}| exp(−64(_{0})^{2}), which approximately models the force of a parabolic spring at small distances. An additional force was applied to all pairs of vertices not bonded by springs such that any such pair of vertices within a given distance ^{2}; in our simulations,

The algebraic model of retinotopic organization was modified from that of Schira

Simulation was performed by numerical integration of the system using a time-step size of 5 ms. At each step _{t+1}_{t}_{t} ∂t_{t} ∂t^{2}/2 and velocities were updated such that _{t}_{+1} = _{t}_{t} ∂t_{t}_{t}_{t}^{2}⋅g/s^{2}. Energies were examined every 10 steps and KEs were rescaled whenever the total energy (PE+KE) exceeded the initial energy (PE_{0}) by at least 2 rad^{2} g/s^{2} due to numerical drift. Simulations were run with a step size of 2 ms for 5,000 steps (10 s). After simulation, the resulting configuration was minimized by a simple gradient descent search using a gradient step distance of 0.005 for 500 steps or until convergence. Source code for the simulation is provided via a gitHub repository (

By simulating the system until a low PE is achieved, we allow the constraints imposed by both the cortical anatomy and the functional model to relax into a solution that respects both kinds of information. Because the simulation incorporates KE, a nonlocal energy minimum may be found; it is therefore beneficial to use simulated annealing. Four simulations of 10 s (5,000 steps) each were performed such that the final arrangement of vertices in each simulation was used as the starting arrangement for the next simulation; spring ideal lengths were not recalculated, however, and the velocities were re-randomized such that the KE of the system was 10 rad^{2} g/s^{2} at the beginning of each simulation. The final arrangement of the four simulations with the lowest PE was chosen as the arrangement of the corrected topology. Vertices were assigned model polar angle and eccentricity values from their positions in the corrected topology by inverting the algebraic model of retinotopy. In other words, if the algebraic model of retinotopy predicts that a point (

Retinotopic mapping data was obtained from 19 subjects to an eccentricity of 10° of visual angle (dataset D_{10°}) (

Registration via MSD simulation brought the aggregate retinotopic mapping values into alignment with the algebraic model by warping the cortex. The magnitude and direction of warping induced by this registration (

(

When aggregated within the cortical surface atlas, polar angle organization is largely consistent across subjects. A flattened aggregate map of the confidence-weighted mean polar angle of the 19 subjects in our 10° eccentricity dataset D_{10°} is shown in

(_{10°} shown in the cortical surface atlas space. (

To examine how well our template predicts a subject not previously seen, we calculated leave-one-out errors. To do so, the aggregate polar angle data was obtained from 18 subjects. The cortical surface atlas was then warped by MSD simulation to match the aggregate to the algebraic model of retinotopy. Finally, the algebraic model was projected back to the surface atlas and used to predict the polar angle organization of the left out subject.

Leave-one-out errors in the polar angle prediction were non-uniform across striate and extrastriate cortex (

Polar Angle Error |
|||||

Area | Absolute |
Signed |
Aggregate |
Unregistered |
Split-half |

V1 | 10.48° | 2.01° | 21.24° | 33.20° | 13.78° |

V2 | 11.12° | −3.17° | 16.15° | 37.52° | 7.13° |

V3 | 11.73° | 3.35° | 20.51° | 37.34° | 9.86° |

All | 10.94° | 0.58° | 16.48° | 37.28° | 7.50° |

Eccentricity Error |
|||||

Area | Absolute | Signed | Aggregate | Unregistered | Split-half |

V1 | 0.41° | 0.26° | 0.61° | 6.20° | 0.68° |

V2 | 0.34° | −0.06° | 0.96° | 10.38° | 0.50° |

V3 | 0.33° | 0.10° | 1.08° | 3.07° | 0.63° |

All | 0.37° | 0.15° | 0.96° | 5.94° | 0.50° |

Errors are calculated in a typical leave-one-out fashion in which each subject is compared to the prediction found using all other subjects; all significant vertices between 1.25° and 8.75° of eccentricity are included, and the reported errors represent the median of all vertices from all subjects.

Median absolute leave-one-out error between expected and observed values of all vertices.

Median signed leave-one-out error, expected value minus observed value, of all vertices.

Median absolute leave-one-out error, as calculated by predicting the polar angle and eccentricity of the left-out subject from the confidence-weighted mean of all other subjects.

Median absolute error between observed values and those predicted by the algebraic model of retinotopy prior to any registration.

Median absolute error between observed values from two identical 20 minute scans.

The quality of the polar angle predictions provided by the MSD approach may be compared to the prediction accuracy obtained using only the aggregate retinotopy data (similar to the approach of _{10°} and used each of these maps to predict the polar angle and eccentricity of the excluded subject. The median absolute leave-one-out polar angle error between all significant vertices of all subjects and the appropriate leave-one-out aggregate vertices was 23.27° (

Finally, we examined how well the algebraic model of retinotopic organization, prior to spring registration to the aggregate data, predicts retinotopy in individuals. Again, the median absolute polar angle error of 34.12° was much greater than that obtained following MSD warping of the aggregate data to the algebraic model. This indicates that our approach corrects consequential distortions introduced by cortical flattening.

_{10°}. As with polar angle, the organization of eccentricity is consistent across subjects combined in the cortical surface atlas space, but sharp bends in the iso-eccentric contours,

(_{10°} shown in the _{20°} shown in the cortical patch corrected by MSD warping to the D_{10°} dataset. Although this dataset includes eccentricities beyond those used to discover the corrected topology, the 20° aggregate data is in good (although not perfect) agreement with the prediction.

Median absolute leave-one-out errors for eccentricity were low across V1–V3 (_{10°} were 0.41° and 0.05° respectively (

As was found for polar angle, simply using the aggregate polar angle data without MSD registration to the algebraic model resulted in substantially worse prediction accuracy for left-out subjects (median absolute error of 1.53°;

The accuracy of polar angle and eccentricity prediction suggests that the algebraic model following MSD warping fits the retinotopic arrangement in regions V1–V3 well. This accuracy of prediction, however, does not necessarily indicate that the algebraic model is a good general representation of retinotopic organization. This is because MSD warping could in principle force the retinotopic data to match any locally smooth model which would then serve to regularize the data in the face of noise and thus improve prediction. While the “anatomical” springs used in the MSD simulation make an extreme warping to a very poor algebraic model implausible, an explicit test of the generalizability of the approach is desirable. If the algebraic model of retinotopy accurately describes the functional arrangement of the visual cortex, our approach should extrapolate to the prediction of eccentricity and polar angle in regions of visual cortex beyond the retinotopic mapping data.

To test the generality of the algebraic model and our template, we compared the anatomical template of retinotopy derived from D_{10°} to the aggregate retinotopic mapping data from D_{20°}, which consists of a separate set of subjects whose retinotopic maps were found using different techniques that doubled the mapped eccentricity range to 20° (see

For polar angle, the median absolute and signed errors between the measured and predicted value were 14.58° and 0.99° respectively. Note that these errors are comparable to those from the D_{10°} leave-one-out analyses despite the fact that the D_{20°} data extends beyond the 10° of eccentricity used to fit the model.

We next examined eccentricity prediction. _{20°} in the corrected cortical surface space found using D_{10°} (the D_{20°} aggregate in the original cortical atlas space is presented in the Supplemental _{20°} and the eccentricity template was 0.77°. Notably, this error is lower in the region from 1.25° to 8.75° (median absolute error: 0.59°) and higher in the region from 8.75°–18.75° (median absolute error: 2.33°). This suggests that our ability to fit extended data with our template is good but imperfect.

Our prediction error incorporates both the imperfections of our template as well as error in the measurement of retinotopy in the individual subject to be predicted. We have previously reported that the error in measured polar angle and eccentricity between two identical 20 minute retinotopic scans is ∼0.75° of eccentricity and ∼7.76° of polar angle in area V1

We have described a technique to register functional data on the cortical surface to a 2D algebraic model of cortical organization. This approach allows us to predict the location and organization of visual areas V1–V3 in individual subjects based only upon an anatomical image of their brain. The overall prediction error for V1–V3 (10.93° of polar angle, 0.51° of eccentricity) is actually somewhat lower than the error we observed in our previous V1 template alone (11.43° of polar angle, 0.91° of eccentricity)

In addition to good prediction accuracy, the anatomical template of V1–V3 retinotopy had generally small and uniform prediction bias. An exception to this property was found at the dorsal V3/V3A border, where our template consistently over-predicts the observed polar angle values (

One possible explanation of this attenuated reversal is that it is the result of a poor across-subject anatomical registration due to variability in sulcal topology between subjects that could not be aligned. Such a problem would result in poorly aligned vertices and variable values contributing to the aggregate at this location. Examination of the sulcal curvature of individual subjects, however, does not support the idea of greater sulcal variability in this region (

An alternate explanation of the error near the dorsal V3 boundary is that individual differences in the mapping between structure and function create an area of relatively poor fit. Indeed, if we assume that the anatomical registrations provided by FreeSurfer are unbiased, there do appear to be significant differences in the location of the V3/V3A boundary between subjects (

It is entirely possible that the dorsal V3 border, and more generally the quality of the entire template, could be improved with modifications of our approach. We presented here a particular algebraic model of retinotopy

More broadly, we consider the key insight of our work to be that geometric distortion of the flattened cortical surface limits the application of idealized models of cortical organization to empirical measurements of cortical function. These distortions, whether introduced by the developmental process of cortical folding or the digital process of cortical flattening, may be corrected by warping the cortical surface to bring function and model into alignment. Here, we demonstrated the practical value of this approach by creating an anatomical template of retinotopic organization. We expect that other early sensory areas such as the sensorimotor and auditory cortex, as well as higher level visual areas such as motion and face sensitive cortex, could be modeled using similar methods. This paper and its supplemental materials are intended as a guide for these kinds of studies.

Polar angle prediction error for (A) V1, (B) V2, and (C) V3. In each figure, the gray line shows the median absolute leave-one-out error for vertices based on their predicted polar angle. The black line shows a best-fit fifth order polynomial to the median. The shaded regions show the extent of the upper and lower quartile of the errors. A spike in median absolute error can be seen near 90° of polar angle in both V2 and V3; this spike is due to error in a region near the foveal confluence (

(TIF)

Eccentricity prediction error for (A) V1, (B) V2, and (C) V3. In each figure, the gray line shows the median absolute leave-one-out error for vertices based on their predicted polar angle. The black line shows a best-fit fifth order polynomial to the median. The shaded regions show the extent of the upper and lower quartile of the errors.

(TIF)

Split-halves test/retest errors for (A) eccentricity and (B) polar angle. Test/retest error at a particular vertex position were calculated as the median absolute difference in measured (_{10°}. The dashed black line shows the Hinds

(EPS)

Exploration of the dorsal V3/V3A border. (A) The standard deviation (left) and median (right) sulcal curvature across all subjects and hemispheres. The dashed red outline indicates the region from which plots in B are taken. (B) Contour plots of polar angle for all subjects' dorsal V2/V3/V3A regions. Contour lines are drawn at 0°, 30°, 60°, 90°, 120°, 150°, and 180°. In the lower right corner, the 150° contour lines for all subjects are shown together. Although the V1/V2 border reversal is relatively conserved, much less agreement can be found for the V3/V3A reversal.

(TIF)

Supplemental

(PDF)

Supplemental notes regarding the parameterization of the simulation and Schira model. This document provides a brief description of our simulation and model parameterization as well as information about where to find additional materials such as source code.

(DOC)