Figure 1.
Overview of the processing steps (left-hand side) and the image registration and stacking procedures (right-hand side).
To facilitate the registration of in vivo, ex vivo and histology images, the tumor orientation was tracked by color coding the different tumor surfaces and by creating a reference cutting plane. This reference plane was created by slicing off a thin section of the whole tumor volume along the longest tumor axis subsequent to fixation. Although the reference plane is not physically present in in vivo MRI, the knowledge of its orientation is crucial to perform image resampling prior to registering in vivo MRI with ex vivo MRI [16].
Figure 2.
Illustration of subcutaneous tumor position.
Tumor at onset of dissection (A) and as a 3D in vivo MRI tumor volume rendering (B), the subcutaneous side of the tumor is marked in green. A yellow line represents the cutting plane orientation along the longest tumor axis and perpendicular to the subcutaneous tumor side. The second row images show the corresponding slices of in vivo MRI (C), ex vivo MRI (D) and as histological section (E).
Figure 3.
The distribution of separated image channels from a H&E section.
We used the red image channel to perform the registration as it provides the best separation between signal intensities of necrotic and viable volumes of interest (VOIs) and presumably the best image contrast. A H&E stained histological section (A) and three separate color channels, red-green-blue, (B–D) with corresponding histogram distributions of the vital (green) and necrotic (red) tumor regions (E–G).
Figure 4.
Final registration results for five tumors.
Registered ex vivo T2*-w MRI (first column), in vivo T2*-w MRI (second column), registered color 3D histology (third column), and checkerboard view of in vivo and registered histology (fourth column).
Table 1.
Average root mean squared distances (mm) for the different registration steps, averaged over all 5 subjects.
Table 2.
Summarized registration results averaged over all five subjects.
Figure 5.
The illustration of signal intensity correspondence between in vivo T2*-W MRI and registered 3D histology for three VOIs (e.g., necrotic-red, viable-green, and hemorrhagic-blue).
The 3D correspondence of tumor histology and in vivo MRI enables extraction of MRI characteristics for histologically defined regions. This is illustrated in scatter plot (B) and using the histogram-based probability density function of the registered histology (C) which clearly separated the different tissue types in the H&E stained images. The corresponding probability density function of in vivo 3D-T2*w MRI (A) demonstrates that viable and hemorrhagic regions cannot be separated using solely in vivo 3D-T2*w MRI signal intensities. Nevertheless, necrotic regions can be effectively separated from the other two histologically confirmed regions.
Figure 6.
The group-wise probability density functions distributions of in vivo 3D-T2*-w MRI.
Three VOIs (e.g., necrotic-red, viable-green, and hemorrhagic-blue) were annotated in histological sections and used for segmentation of automatically aligned in vivo MRI. The excessive hemorrhagic regions are visible in two out of five subjects. It demonstrates the similar gray value ranges for each VOI. When considering the probability density function for all tumors, the necrotic regions were significantly different from other histologically confirmed regions (i.e., viable and hemorrhagic). However, the viable and the hemorrhagic regions showed a large overlap in T2*-w MRI signal intensity.
Figure 7.
Details from a H&E stained section and its corresponding MRI slice.
Histological section (A–B) shows the difference in histological appearance, whereas the MRI appearance in 3D T2*-w MRI (C) is similar. The necrotic segmentation, superimposed on 3D T2*-w, is shown in red (D).
Figure 8.
Multi-stained histology dataset.
H&E (A), Goldner (B), van Gieson (C) and Peroxidase (D) stained consecutive sections with the corresponding in vivo MRI (E) and intermediate ex vivo MRI (F). The slice thickness of all histology sections is 4 µm, and the distance between the consecutive histology sections (A–D) is 8 µm.