Schizophrenia Diagnosis with a Transparent Multimodal MRI Fusion Approach for Early Detection and Clinical Trust

Schizophrenia is a severe mental disorder that can affect perception, thought, emo-tion, motivation, and cognition. Clinical diagnosis is based on symptoms and clinical assessment; no single MRI marker is sufficient for diagnosis.^[1] Structural and func-tional neuroimaging can nevertheless reveal group-level brain differences that may help computational models distinguish patients from healthy controls.^[2]

Structural magnetic resonance imaging (sMRI) describes anatomical properties such as gray-matter distribution, cortical thickness, and ventricular size. Functional magnetic resonance imaging (fMRI) describes activity and connectivity patterns, including correlations between functional networks. The two modalities therefore contain different but potentially complementary information. Combining them can provide a broader representation of disease-related variation than using either modality alone.^[3]

Machine-learning models can learn complex patterns from neuroimaging data, but high dimensionality, small sample sizes, and limited interpretability remain important problems.^[4] A model can achieve good predictive performance while still relying on unstable or non-clinical features. For this reason, performance and interpretability should be considered together. Explainable artificial intelligence (XAI) methods can show which features or image regions contribute to a prediction, but these explanations require cautious interpretation and independent validation.

This work develops a multimodal schizophrenia-classification pipeline using the MLSP 2014 dataset. FNC features from fMRI and SBM features from sMRI are com-bined with a multiscale attention-based classifier. Random-forest feature importance and t-SNE are used for complementary analysis. LRP and Grad-CAM are used to inspect model explanations. The main contributions are:

1. An integrated multimodal pipeline that uses both FNC and SBM information for binary schizophrenia classification;

2. A three-branch multiscale feature extractor with attention-based feature fusion before classification;

3. A combined analysis workflow that links predictive performance, feature ranking, low-dimensional clustering, ICA maps, and two XAI methods; and

4. An uncertainty-aware evaluation that reports the held-out test metrics together with 95% confidence intervals and explicitly states the limitations caused by the small sample size.

Fig. 1: Overview of the proposed multimodal pipeline. FNC and SBM information derived from fMRI and sMRI is used for classification, while Grad-CAM and LRP provide post-hoc explanations.

The rest of the paper is organized as follows. Section 2 reviews related neuroimaging and machine-learning research. Section 3 states the research gap and methodological novelty. Section 4 describes the data, preprocessing, model, algorithms, and XAI methods. Section 5 presents the results and uncertainty analysis. Section 6 discusses interpretation and limitations, and Section 7 concludes the paper.

Neuroimaging research has reported structural and functional alterations in schizophrenia, including changes in gray matter, ventricular volume, and large-scale functional networks.^[2] sMRI provides anatomical information, whereas resting-state fMRI can represent functional interaction between brain regions. These modalities are complementary, but their high dimensionality and different statistical properties make multimodal integration difficult.

Machine-learning methods have been used to classify schizophrenia from MRI-derived features. Yang et al.^[5] combined fMRI and genetic information and reported 87.25% accuracy using leave-one-out validation on 40 subjects. Their result demon-strated the potential value of multimodal information, but the sample was small and the modalities differed from the present FNC–SBM setting. Zhang et al.^[6] used 3D structural MRI and reported 92.1% accuracy and an AUC of 0.987 on an independent test set. This approach used only structural MRI. More recently, Bi et al.^[3] proposed MultiViT to fuse 3D gray-matter maps with FNC information and reported an AUC of 0.833. Their cross-attention mechanism improved multimodal fusion and provided attention-based localization.

Table 1 places the present results beside representative prior work. The values are not directly comparable because the datasets, modalities, validation procedures, and reported metrics differ. The purpose of the table is to show where the present work fits in the literature rather than to claim a universal performance advantage.

The literature therefore shows three recurring gaps. First, small neuroimaging cohorts make performance estimates uncertain. Second, feature selection can cause data leakage when it is performed before the train/test split. Third, visual explanations may be useful but should not automatically be treated as clinical biomarkers. The present revision addresses these issues by explicitly defining the leakage-safe processing order, reporting the exact held-out test size, adding statistical uncertainty, and using cautious language for XAI interpretations.

Table 1: Representative neuroimaging classification work and the position of the present method. Metrics are reported as stated in the cited papers and are not directly comparable across different datasets and validation designs.

The methodological novelty is the integration of four components in one pipeline. First, the method combines two MRI-derived feature families: 378 FNC features from fMRI and 32 SBM features from sMRI. Second, the proposed classifier uses three scale-specific feature pathways with 32, 32, and 64 convolutional filters followed by max pooling. Third, an attention module learns a weight for each scale before the feature maps are concatenated. Fourth, the predictive output is linked to feature ranking, t-SNE visualization, ICA maps, LRP, and Grad-CAM. This design therefore connects classification with several complementary forms of interpretation rather than reporting a prediction score alone.

The novelty should not be interpreted as a claim that the method is universally more accurate than all previous models. Published results use different datasets and evaluation procedures. The contribution of the present method is the combined multimodal and explanation-oriented workflow, together with an uncertainty-aware evaluation on the MLSP cohort.

Fig. 1 summarizes the complete workflow. The method uses the FNC and SBM features supplied by the MLSP 2014 Schizophrenia Classification Challenge. The processing order is important: subjects are split before any label-dependent feature selection is fitted. Scaling and SelectKBest are fitted on the training data only and are then applied to the held-out data. This prevents information from the test labels entering the feature-selection stage.

The MLSP 2014 Schizophrenia Classification Challenge provides multimodal MRI-derived features for schizophrenia classification.^[7] The training cohort used here contains 86 subjects: 40 subjects with schizophrenia and 46 healthy controls. Each subject has 410 features. Of these, 378 are FNC features derived from resting-state fMRI and 32 are SBM loadings derived from structural MRI. The challenge also provides ICA brain maps associated with the functional and structural components.

The reported evaluation uses a stratified 80/20 hold-out split. This gives 68 subjects for model development and 18 subjects for final testing. The held-out set contains 10 healthy controls and 8 subjects with schizophrenia, as reflected by the integer counts in the confusion matrices. The held-out subjects are not used to fit the scaler or the feature-selection procedure.

For each feature $x_j$, standardization is performed with the mean ${\mu }_j$ and standard deviation ${\sigma }_j$ calculated from the training subset only:

The same training values ${\mu }_j^{\left(train\right)}$ and ${\sigma }_j^{\left(train\right)}$ are then used to transform the held-out subjects.

SelectKBest with the ANOVA F statistic is used to retain the 100 highest-ranked training features. For feature $j$, the score can be written as the ratio between between-class and within-class variability:

where $C=2$ is the number of classes and $N$ is the number of training subjects. Feature ranking is performed using the training labels only. Pearson correlation is used as a descriptive analysis of linear association:

The original manuscript used the term “cross-validation” without giving enough detail. The revised manuscript therefore describes only the validation design supported by the reported results: a stratified held-out test set. No repeated-cross-validation result is claimed.

Fig. 2: The proposed MsRNN architecture for two-class classification. Each scale-specific branch uses convolutional feature extraction, followed by attention-based fusion and a final classifier.

The MLSP data provide FNC and SBM features derived from ICA-based processing. In ICA, the observed data matrix X is represented as a mixture of statistically independent sources S:

where A is a mixing matrix. An unmixing matrix W estimates the source representation:

The challenge documentation describes 28 non-artifactual fMRI ICA components and 32 sMRI ICA components. FNC values summarize pairwise correlations between functional component time courses, whereas SBM loadings represent the expression of structural gray-matter ICA maps.^[7]

The proposed MsRNN is a binary classifier for schizophrenia and healthy-control labels. As shown in Fig. 2, three scale-specific pathways are used. Each pathway contains convolutional layers with 32, 32, and 64 filters, followed by max pooling. The resulting feature maps have the documented size 32 × 32 × 64 before attention-based fusion.

For an input $X$, the three scaled representations are

where $s_i$ denotes the scale. In branch $i$, convolutional features are calculated as

where $\varphi \left(·\right)$ is the nonlinear activation and ∗ denotes convolution. After the third convolution, max pooling gives

The attention module produces a score for each scale:

which is converted to a normalized weight

The weighted maps ${\tilde{F}}_i={\alpha }_i{F}_i$ are concatenated:

The fused representation is passed to the final classifier and converted to a schizophrenia probability $\widehat{y}$ by a sigmoid output. Binary cross-entropy is used as the objective:

Table 2 lists the implementation information that is recoverable from the retained experimental record. This avoids introducing undocumented hyperparameter values during revision.

Fig. 3: Ranking of the 50 most influential neuroimaging features used for exploratory interpretation. SBM features dominate the highest ranks, while FNC features also contribute.

Table 2: Documented implementation settings of the proposed model

A set of benchmark classifiers (RF, ANN, SVM, KNN, and LR) is evaluated on the same held-out partition using the same training-only preprocessing principle. Because the archived manuscript package does not contain the executable training code, software-specific settings that are not documented in the retained record are not reconstructed or invented in this revision. This limitation is stated explicitly in Section 6.

A random-forest classifier is used as a complementary feature-ranking tool. The 50 highest-ranked FNC/SBM features are displayed in Fig. 3. Four of the top five ranked features are SBM features, indicating that structural information has a strong role in this specific fitted model. This observation is descriptive for the present cohort and should not be interpreted as proof that structural MRI is generally more informative than fMRI.

The selected features are also visualized using t-distributed stochastic neighbor embedding (t-SNE).^[8] t-SNE minimizes the divergence between high-dimensional and low-dimensional neighborhood distributions:

Fig. 4 shows the three-dimensional embedding. The two classes overlap, which is expected for a heterogeneous clinical condition and indicates that the selected features do not form perfectly separated groups.

Fig. 4: Three-dimensional t-SNE views of the selected features. The visual overlap between classes indicates that the feature space is not perfectly separable.

The ICA maps supplied with the challenge are used to relate selected FNC and SBM components to spatial brain patterns. These visualizations are used for interpretation only; they do not establish clinical biomarkers.

LRP and Grad-CAM are used as post-hoc explanation methods. LRP redistributes a model output backward through the network so that input regions receive relevance values associated with the prediction.^[9] A generic relevance-conservation step can be

written as

where $R_{i\leftarrow j}^{(l,l+1)}$ is the relevance passed from neuron $j$ in layer $l+1$ to neuron $i$ in layer $l$.

Grad-CAM uses the gradient of a target class score $y^c$ with respect to a convolutional feature map $A^k$.^[10] The weight of feature map $k$ is

and the class activation map is

These heatmaps are explanations of the fitted model, not direct measurements of disease mechanisms. The present dataset does not provide enough independent subjects to establish the stability of every highlighted region. Therefore, the revised interpretation focuses on whether the maps are spatially plausible and whether the two methods show broadly compatible attention patterns. Quantitative cross-subject explanation stability is left for future validation on a larger independent cohort.

This section reports the held-out classification results, uncertainty of the accuracy estimates, feature-space visualizations, ICA maps, and XAI outputs.

Table 3 reports the same performance values as the original manuscript. MsRNN obtained 0.8333 accuracy, 0.8571 precision, 0.7500 recall, and 0.8000 F1-score on the 18-subject held-out test set.

Table 3: Held-out classification performance. A = accuracy, P = precision, R = recall, F1 = F1-score.

Fig. 5: Original performance visualization for the proposed and benchmark classifiers. The values correspond to the held-out metrics in Table 3.

Fig. 6: Metric-profile heatmap. The figure provides an additional graphical view of accuracy, precision, recall, and F1-score without introducing new experimental values.

Because only 18 subjects are in the held-out set, a small change in the number of correct predictions produces a large change in accuracy. We therefore report 95% Wilson binomial confidence intervals for accuracy. MsRNN correctly classified 15 of 18 subjects, giving 83.33% accuracy with a 95% Wilson interval of 60.78–94.16%. SVM and LR each classified 14 of 18 subjects correctly (77.78%; 95% CI 54.78–91.00%), while RF, ANN, and KNN each classified 13 of 18 correctly (72.22%; 95% CI 49.13–87.50%). The intervals overlap substantially. Therefore, the revised manuscript does not claim a statistically proven superiority of MsRNN over the benchmark models.

Fig. 7: Held-out accuracy with 95% Wilson confidence intervals. The wide and overlapping intervals show the uncertainty caused by the small test set.

The original confusion-matrix image contained two percentage inconsistencies for ANN and KNN. Fig. 8 has been regenerated from the integer counts implied by the reported accuracy, precision, and recall values. For MsRNN, 9 of 10 healthy controls and 6 of 8 schizophrenia cases were classified correctly.

Fig. 3 shows the 50 highest-ranked features. The strongest positions are mainly SBM features, and four of the top five ranked features are structural. Fig. 4 shows that the healthy-control and schizophrenia samples are not fully separated in the t-SNE embedding. This overlap is consistent with the moderate test accuracy and supports a cautious interpretation of the classification boundary.

The main ICA components are shown in Figs. 9 and 10. These maps provide spatial context for functional and structural components. They are used to visualize pat-terns associated with the fitted model and dataset, not to establish definitive disease markers.

The structural maps include patterns near frontal and temporal regions and ventricular areas. These observations are broadly compatible with neuroimaging literature reporting structural differences in schizophrenia.^[2] However, the present analysis is exploratory and does not test these regions as independent biomarkers.

The fMRI maps show heterogeneous patterns of stronger and weaker activation/connectivity across the displayed slices. Because the present dataset is small and the analysis is model-driven, these patterns are described as candidate regions for further investigation rather than causal or diagnostic findings.

Fig. 8: Corrected held-out confusion matrices. Each cell gives the subject count and the percentage within its actual class.

Fig. 9: Top three fMRI ICA component maps used to inspect functional patterns associated with the classification analysis.

Fig. 10: Top three sMRI ICA component maps used to inspect structural patterns associated with the classification analysis.

Fig. 11: Illustrative structural visualization used in the exploratory interpretation. Color overlays show regions emphasized in the processed maps; they should not be read as independently validated clinical abnormalities.

Fig. 12: Illustrative fMRI visualization showing spatially varying activation/connectivity patterns used for exploratory interpretation.

Fig. 13 shows LRP heatmaps for schizophrenia and healthy-control examples. Higher-resolution maps provide more localized relevance patterns than the lower-resolution views in the displayed examples. This improves visual localization, but it does not by itself demonstrate that the highlighted region is a disease biomarker. Fig. 14 presents the corresponding Grad-CAM visualization. Warm colors indicate regions with larger positive contributions to the displayed class score. LRP and Grad-CAM provide two different views of model attention. Their broad spatial agree-ment in the examples is useful as a qualitative check; however, explanation consistency was not quantified over all subjects. This is a limitation of the present analysis.

Fig. 13: LRP heatmaps for example predictions. The maps show input regions that receive higher relevance under the fitted model.

Fig. 14: Grad-CAM example. Warm colors indicate image regions with larger positive contribution to the target class score. The map explains the model prediction and is not a clinical diagnostic map.

The results show that multimodal FNC and SBM information can be combined within an attention based multiscale framework. MsRNN produced the highest point estimate of accuracy in the reported held-out comparison (83.33%), with precision of 85.71%, recall of 75.00%, and F1-score of 80.00%. The model also produced a more balanced precision–recall profile than several baselines. However, the practical difference between point estimates must be interpreted together with the uncertainty analysis. The 95% confidence intervals are wide and overlap because the held-out set contains only 18 subjects. The revised paper therefore treats the performance result as preliminary rather than as proof that MsRNN is statistically better than every benchmark.

The small cohort is the main limitation. With 86 subjects and hundreds of features, overfitting is a substantial risk. To reduce leakage, scaling and ANOVA feature selection are defined as training-only operations in the revised methodology. Even with this safeguard, a single held-out split can be sensitive to which subjects enter the test set. A larger independent dataset, repeated stratified cross-validation, or nested cross-validation would provide a stronger estimate of generalization. Because the archived submission package did not include executable training code or subject-level prediction files, this revision does not invent repeated-run results or pairwise significance tests that cannot be reconstructed reliably.

Feature ranking and t-SNE provide complementary information. The high-ranked feature list contains a strong SBM contribution, while the t-SNE plot shows overlap between the two diagnostic groups. These findings suggest that structural features contribute useful discrimination in this fitted model, but the two groups remain heterogeneous. The ICA maps give anatomical context to the feature analysis, yet they should be interpreted at the group/model level.

LRP and Grad-CAM make the model easier to inspect, but post-hoc explanations have their own uncertainty. A highlighted area indicates that a fitted model used information from that area; it does not prove that the region is a clinically validated biomarker or a causal mechanism. The present figures provide qualitative examples only. A stronger XAI validation would calculate explanation consistency across sub-jects and repeated models, for example by measuring overlap or rank correlation of salient regions and then testing the same regions in an independent cohort. These steps are important before clinical use.

The manuscript has therefore been revised to use cautious clinical language. Terms such as “biomarker” are avoided unless referring to prior validated literature. The displayed regions are described as candidate or discriminative regions that require external validation. This distinction is especially important because the present cohort is small and comes from a single challenge dataset.

Finally, reproducibility is limited by missing software-level details in the archived manuscript package. The revised paper reports all settings that can be recovered reliably, including the number of modalities and features, selected-feature count, three multiscale branches, filter counts, feature-map size, attention fusion, binary loss, and held-out sample size. The optimizer, learning rate, batch size, kernel size, software versions, and random seed should be recovered from the original executable code before final publication if those records are available.

This paper presents a multimodal MRI classification framework that combines fMRI-derived FNC features and sMRI-derived SBM features with multiscale feature extraction, attention fusion, and XAI. On the 18-subject held-out test set, MsRNN achieved 83.33% accuracy, 85.71% precision, 75.00% recall, and 80.00% F1-score. The 95% Wilson interval for accuracy (60.78–94.16%) shows that the estimate remains uncertain because of the small sample. LRP and Grad-CAM provide useful model-level visual explanations, but the highlighted regions require independent validation before they can be interpreted as clinical biomarkers. Future work should use larger indepen-dent cohorts, repeated or nested validation, quantitative XAI-stability measures, and complete release of implementation settings and subject-level predictions.

Mohammad Shohelur Rahman: Conceptualization, Methodology, Investigation, Formal analysis, Visualization, Writing–Original draft, Writing–Review and editing. Ahnaf Tajwar: Conceptualization, Methodology, Investigation, Formal analysis, Visualization, Writing–Original draft, Writing–Review and editing. Tawaz Rahman: Conceptualization, Methodology, Investigation, Formal analysis, Visualization, Writing–Original draft, Writing–Review and editing. All authors reviewed and approved the final manuscript and agree to be accountable for the work.

The data used in this work are publicly available through the MLSP 2014 Schizophrenia Classification Challenge

on Kaggle, subject to the competition access terms [7]. The challenge provides FNC and SBM features and associated ICA-map resources.

The authors received no specific financial support for the research, authorship, or publication of this work.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

The authors declare that OpenAI ChatGPT was used to assist with language editing, organization of the revision, and preparation of the response to reviewers. It was not used to create or alter experimental measurements or reported model-performance values. The authors reviewed the revised text and remain responsible for the accuracy, interpretation, and integrity of the manuscript.

1. [1] R. A. McCutcheon, T. Reis Marques, O. D. Howes, Schizophrenia-an overview, JAMA Psychiatry, 2020, 77, 201–210, doi: 10.1001/jamapsychiatry.2019.3360.
2. [2] O. D. Howes, C. Cummings, G. E. Chapman, E. Shatalina, Neuroimaging in schizophrenia: an overview of findings and their implications for synaptic changes, Neuropsychopharmacology, 2023, 48, 151–167, doi: 10.1038/s41386-022-01426-x.
3. [3] Y. Bi, A. Abrol, Z. Fu, V. D. Calhoun, A multimodal vision transformer for interpretable fusion of functional and structural neuroimaging data, Human Brain Mapping, 2024, 45, 26783, doi: 10.1002/hbm.26783.
4. [4] J. W. Lai, C. K. E. Ang, U. R. Acharya, K. H. Cheong, Schizophrenia: a survey of artificial intelligence techniques applied to detection and classification, International Journal of Environmental Research and Public Health, 2021, 18, 6099, doi: 10.3390/ijerph18116099.
5. [5] H. Yang, J. Liu, J. Sui, G. Pearlson, V. D. Calhoun, A hybrid machine learning method for fusing fMRI and genetic data: combining both improves classification of schizophrenia, Frontiers in human neuroscience, 2010, 4, 192, doi: 10.3389/fnhum.2010.00192.
6. [6] J. Zhang, V. M. Rao, Y. Tian, Y. Yang, N. Acosta, Z. Wan, P.-Y. Lee, C. Zhang, L. S. Kegeles, S. A. Small, Detecting schizophrenia with 3d structural brain MRI using deep learning, Scientific Reports, 2023, 13(1), 14433, doi: 10.1038/s41598-023-41359-z.
7. [7] E. J. Castro, M. Cetin, N. Cota, R. F. Silva, Vince, MLSP 2014 Schizophrenia Classification Challenge, Kaggle, 2014, https://www.kaggle.com/competitions/mlsp-2014-mri.
8. [8] Y. Wang, L. Chen, J. Jo, Y. Wang, Joint t-SNE for comparable projections of multiple high-dimensional datasets, IEEE Transactions on Visualization and Computer Graphics, 2021, 28, 623–632, doi: 10.1109/TVCG.2021.3114765.
9. [9] Bach, S., Binder, A., Montavon, G., Klauschen, F., Mu¨ller, K.-R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation, PLOS ONE, 2015, 10, 0130140, doi: 10.1371/journal. pone.0130140.
10. [10] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra, Grad-cam: Visual explanations from deep networks via gradient-based localization, In: Proceedings of the IEEE International Conference on Computer Vision, 2017, 618–626, doi: 10.1109/ICCV.2017.74.

The views, statements, and data in all publications solely belong to the authors and contributors. GR Scholastic is not responsible for any injury resulting from the ideas, methods, or products mentioned. GR Scholastic remains neutral regarding jurisdictional claims in published maps and institutional affiliations.

This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits the non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as appropriate credit to the original author(s) and the source is given by providing a link to the Creative Commons License and changes need to be indicated if there are any. The images or other third-party material in this article are included in the article's Creative Commons License, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons License and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this License, visit: https://creativecommons.org/licenses/by-nc/4.0/

© The Author(s) 2026