When comparing structures or functions across brains, it is common to label the gross anatomy of brain image data and to compare the structures or functions that lie within anatomically labeled regions. Since brains differ in their anatomy [1–10], it would seem reasonable to refer to the anatomy of many brains when labeling an individual subject's brain image. Atlases are manually labeled brains used as references. Using every atlas from a group of atlases independent of each other was found to give labeling results superior to those obtained by selecting the closest matching single atlas from the group, the average atlas, or an individual atlas, for the case of confocal microscopy images of bee brains [11]. However, labeling a subject's brain image with many different brains presents unreasonable demands on human labelers, who may not be consistent in their label assignments [12–15]. Fully automated labeling would facilitate large-scale labeling efforts while adding efficiency and consistency.

Image registration software (reviewed in [16–18]) may be used to coregister subject and atlas brain images, thereby labeling the subject images with superimposed atlas labels. There exist many different nonlinear image registration and feature-matching approaches to this problem [19–62]. Mindboggle software (see below) offers certain advantages over most of these approaches: it does not make the same assumptions about preserving topography from brain to brain, is relatively fast, and it performed well in comparison tests with standard image registration software packages (AIR, SPM2, ANIMAL, and linear registration with FLIRT) and in artificial lesion tests [63].

Having an automated registration or feature-matching program and a database of atlases introduces the problem of how to reconcile the multiple atlas label sets when labeling a single subject's brain. Labels could be assigned based on the selection or construction of similar or representative anatomy from these atlases. It is becoming more common to label subject brain image data with a single, composite atlas representing some average of multiple brain atlases (an average brain atlas) or retaining information about the differences between the atlases or between the atlases and the subject brain image (a probabilistic brain atlas).

Average brain atlases attempt to assign to each voxel (volume element) a representative value associated with image intensity or anatomical label. An intensity-based average brain atlas is the voxelwise mean intensity across individual brain images after linear [64–66] or nonlinear [67] coregistration. Additionally or alternatively, an average brain atlas may represent average sulcus shapes and positions computed in the original brain image space [8, 68, 69] or in an alternative space such as on a sphere [70]. An example of a label-based average brain atlas was constructed by Hammers, et. al. [71], where the majority label was computed for each voxel across 20 manually labeled brains after nonlinear registration to the MNI152 [64] template using SPM99 [27]. The use of an average atlas presupposes that there is such a thing as a representative brain and does not usually account for variability across brains.

Probabilistic brain atlases, on the other hand, do provide additional statistical information across the population used to construct the atlas [62, 72–82]. This information may be related to the variance of landmark positions [73], probability of anatomical labels [44, 79, 83, 84], probability of tissue classes [80], or multiple anatomical dimensions, for example characteristics of surface geometry and Bayesian priors associated with neighborhood relations between labels [62], and the multi-dimensional atlases under development by Mazziotta and Zilles and their colleagues [72, 77, 78, 81]. An abstract representation of a database of manually labeled brains can also serve as a probabilistic atlas; for example, expert neural networks trained on a learning database of such brains [48] or graphs relating parametric surfaces [36]. However, there are only two examples known by the authors in which a complete cortical atlas is constructed from multiple label sets where each label set was assigned manually [62, 71], rather than by automating the labeling of many brains without independent validation of the labeling technique. As with average brain atlases, probabilistic atlases have primarily been used as templates to which a subject brain is transformed and compared. This comparison presupposes that the single transform will account for differences between the subject brain image and each of the multiple brain images that were used to construct the atlas.

In this paper, we have chosen to extend the use of an individual atlas to multiple atlases in a recently introduced, fully automated, feature-based nonlinear labeling method called Mindboggle (freely downloadable, open source Matlab code) [63, 85]. Rather than use a single (average or probabilistic) atlas, Mindboggle employs each atlas in a database independently to label the cortical voxels of a subject brain image, and for each voxel chooses the majority label assigned by the different atlases. We explore the effects of using two different labeling schemes and variable numbers of atlases on labeling accuracy and on the numbers of labels assigned per voxel.

We found there to be greater disagreement between atlases as the number of atlases increases, as one would expect. This is clearly demonstrated in Figures 4 and 5. Figure 4 displays the anatomical distribution of the number of different labels assigned by the atlases to each voxel. Figure 5 plots the total number of voxels with a given number of different labels per voxel. Both figures present their data as a function of the number of atlases. If we compare Figure 4 with Figure 3 (it is the same subject), the disagreements are clustered about anatomical boundaries, with the highest numbers of labels per voxel at the boundaries between multiple anatomical regions, as one would expect.

Figure 6A demonstrates the variability in labeling errors when different single atlases are used to label one subject. Figure 6B demonstrates the effect of the use of multiple atlases on labeling errors for the same subject. Figure 6B indicates that increasing the number of atlases reduces labeling errors in Mindboggle. Figure 7 demonstrates that the two subject populations, manually labeled with slightly different parcellation schemes, give clearly separable labeling results, and that the label agreement between manual labels and (voxelwise majority) Mindboggle labels remains distinct between the two subject groups even as the number of atlases increases. Each member from the first group of 10 subjects was labeled with one atlas from the same group, then two, three, up to nine atlases, with each atlas selected at random from the remaining unselected atlases. The same procedure was repeated for each member from the second group of 10 subjects. For each member of the combined subject population, one to 19 atlases are selected at random from either subject group.

If we divide the voxels into groups, by the number of different labels per voxel, as in Figure 10, we may see that there is an inverse relationship between the number of different labels and the label agreement with manual labels. Therefore, the number of labels per voxel provides a rough confidence measure for the majority label assigned to each voxel.

Therefore, Mindboggle is sensitive to variance in the subject population and to the parcellation scheme used to manually label the atlases, in particular to the vertical planes that are used to define boundaries to large regions (occipital lobe, frontal and temporal poles). These planes are not positioned by the sulcus piece matching stage but by an automated identification and matching of specific anatomical landmarks. The definitions of these landmarks may be different between parcellation schemes and may not be as consistently or as accurately determined manually or automatically in one scheme versus another. Some of the differences between the results obtained by the two subject groups (see Methods: Image Acquisition) may be attributed to the broader sampling in the second group of subjects (three races versus one, unknown vs. right-handed, and much wider age range). We can expect even greater deviations from brains that are very young, very old, or inflicted with a pathological condition, something we are presently investigating.

Even with this dependence of absolute results on parcellation scheme, we may determine whether there is a relative improvement of results across all subjects as a function of the number of atlases used to obtain the voxelwise majority labels. From Table 1 and the accompanying graph in Figure 8, we may see that increasing the number of atlases asymptotically increases mean label agreement with manual labels. A one-way ANOVA was performed to test if the means are the same for the label agreements obtained by the different numbers of atlases. A multiple comparison test was then performed using Tukey's honestly significantly difference criterion to determine which pairs of means are significantly different. We see from Figure 9 that simply increasing the number of atlases from one to at least four results in a statistically significant increase in label agreement (p ≪ 10^-6 for all comparisons), and further increasing the number of atlases to at least nine (or at least seven for the first set of subjects) results in a statistically significant increase in label agreement compared with using three atlases. However, the increase in label agreement from four to five or more atlases is not statistically significant for the mixed subject group.

A	All subjects	σ	II	Group 1	σ	II	Group 2	σ	II
1	74.39	(3.90)	0.20	79.05	(1.06)	0.17	73.55	(2.43)	0.21
2	73.75	(2.81)	0.21	78.38	(1.32)	0.18	72.68	(2.07)	0.21
3	76.54	(2.93)	0.19	80.19	(1.42)	0.17	75.30	(2.00)	0.20
4	77.73	(2.78)	0.18	80.95	(1.43)	0.16	76.31	(1.40)	0.19
5	77.88	(2.67)	0.18	81.98	(1.39)	0.15	77.24	(1.83)	0.19
6	78.30	(2.52)	0.18	81.58	(1.42)	0.16	77.45	(1.66)	0.18
7	78.56	(2.75)	0.18	82.09	(1.37)	0.15	77.76	(1.78)	0.18
8	78.67	(2.86)	0.18	82.21	(1.28)	0.15	77.87	(1.93)	0.18
9	79.18	(2.59)	0.17	82.50	(1.20)	0.15	78.15	(1.94)	0.18
10	78.91	(2.60)	0.17
11	79.25	(2.75)	0.17
12	79.47	(2.78)	0.17
13	79.47	(2.86)	0.17
14	79.41	(2.84)	0.17
15	79.41	(2.83)	0.17
16	79.48	(2.57)	0.17
17	79.59	(2.74)	0.17
18	79.61	(2.76)	0.17
19	79.62	(2.74)	0.17

Percent label agreements and type II errors (II) are given for each number of atlases (A) used to label subject group 1, group 2, and all subjects. The accompanying graph is in Figure 6. Standard deviations are in parentheses (for type II errors, σ equals 0.02 for all subjects and 0.01 for groups 1 and 2).

One should not conclude based on these data that atlas databases need only contain four or five atlases to be representative. The standard deviations for our subject pool were high enough to warrant further investigation into sources of error. These sources include morphological dissimilarities between subject subpopulations, different parcellation schemes, and limitations of the Mindboggle algorithm. Interestingly, Kittler et al. [86] found that the classification performance of the voting rule applied to face and voice biometric data also peaked at four to five experts (atlases).

These results corroborate the conclusion of a study on atlas selection strategies applied to confocal microscopy images of bee brains, that labeling a brain image using every one of a group of atlases gives results superior to selecting an individual atlas [11]. However, when they tested the individual atlas condition, they chose only a single favorable atlas from a group of 20, whereas in the present study we ran tests using each and every single individual atlas from a group of 20.

The majority voting rule is probably not the optimal way to decide on a voxel's label [86–89], especially if the selected atlases deviate considerably from the subject brain to be labeled. A missing or unusual structure in a subject brain represented in only a minority of the atlases would most likely result in an inappropriate label. Rather than simply weighting the contribution of each of the atlases equally, each atlas vote for each subject voxel could be weighted by a function of the matching cost for the structure containing that voxel, since Mindboggle's matching cost function is intended to determine degree of correspondence between structures across brains.

For a single atlas to label a single subject, Mindboggle presently takes less than 17 minutes after linear registration and gray matter segmentation on a 2.2 GHz Pentium IV processor running Redhat Linux 9.0 on a PC with 1 GB memory: 1.3 minutes to construct a sulcus skeleton, 2.5 minutes to divide the skeleton with an interhemispheric plane, 3.4 minutes to construct and tally data on sulcus pieces, 2.5 minutes to find matching pieces in the atlas and to transform them from the atlas to the subject brain, and the remaining 7 minutes to warp and propagate labels through the gray matter mask. For each additional atlas, matching, warping, and labeling takes under 10 minutes if performed sequentially. For example, labeling a subject using five atlases would take 17 minutes if conducted in parallel, or an hour if conducted sequentially. The run time would reduce significantly not only by running Mindboggle for each atlas in parallel, but also by implementing faster preprocessing algorithms and optimized code rewritten in a lower-level language such as C as opposed to Matlab.

We conclude that by using multiple atlases, the overall label agreement between manual labels and the majority labels assigned by these atlases significantly improves when using a nonlinear procedure such as Mindboggle.

We are now in the process of applying this multiple atlas extension of Mindboggle to anatomically label functional activity data. Combining a confidence measure for anatomical boundaries derived from multiple atlases with statistical maps of functional activity data across subjects should help to establish our level of confidence in reported functional findings.