The preprocessing of the data was carried out in accordance with figure 4.5.4.
First the masking was performed using all the scans. For every scan the 30% amplitude level of the maximum value was found. Every voxel above this level was masked through. All image mask were then AND'ed forming the final mask. This masking brought the number of voxel down from 81920 to 15625: More than 80% of the voxels discarded!
To measure the generalization of the neural network empirically the data was to split the scans in validation sets and a training set. The validation sets where constructed so they hopefully would contain every kind of noise: interrun noise, intrarun noise, ''hemodynamic response'' noise, and ''learning'' noise (different behavior at the beginning and the end of the scanning session). All though it could be nice to do some ''leave one run out'' cross validation scheme, the amount of data made the split of the basis/non-basis sets rather static (7.2).
Table 7.2: The validation sets.
Whether it is statistically correct to have all validation sets along in the SVD-PCA basis is not clear. One could argue that generalization is the ability to predicts with a completely new scan (with the same subject). In this case the validation sets should not be in the basis. On the other hand one could argue that the overfitting only takes place in the neural network, and what happens before this, of split between sets, is of no importance. Then using more scans for the basis computation would span the paradigm space better.
Some experiments showed that the hemodynamic response infested the post
flange scans, especially the deactivation flange (compare with 
). The robust statistic (pages ) could not
ignore the large amount of scans, who carried the hemodynamic response. One
could of course try to model this behavior. But this fMRI scanning produced
enough scans, so that the number of pattern for the neural network would be
sufficient even with a scans omitted: 4 post
activation flange scans were omitted
from every run, and 10 post deactivation flange scans were left out. This
brought the number of scans within a run down to 58.
The fMRI data can be split in many ways: Total numbering, Numbering with a run, run, rest/activation, first rest / activation / second rest, training/crossvalidation set, hemodynamic mask split. The task of handling these different splittings must not be ignored. Ways to handle the splitting are through a design-matrix-like data structure (see SPM 4.1.1 for the design matrix) and the masking matrix structure 4.3.1.
After the masking and the splittting of the data, a mean for the training set was calculated to center the training set for the SVD/PCA. The SVD/PCA projected scans were scaled so the their range would lay between --1 to +1.
The weight initialization was such that the weight size was normalized with the square root of the fan-in, and the extremum of the input (for the hidden weights).
The entropic errorfunction was selected with +1/--1 marking rest and activation, respectively. As regularization was the usually square weight decay chosen. The weight decay parameter was varied.
The weight optimization was a hybrid second order method with the structural approximation of the full Hessian for the output weights and diagonal approximation for the hidden weights (b approximation in figure 4.6.2). The computational complexity for the full Hessian would rise more drastically than the diagonal approximation as the number of weights under optimization is increased.
With and without gauss-newton approximation was tried.
When pruning was done, the neural network was first trained with 1000 or more of weight optimization epochs, then it was OBD-pruned with the ceil of 5% of the remaining weights. After that retrained with 100 or more weight optimization epochs, before the next 5% pruning.