Automated detection of mental conditions from text is increasingly important due to the rise of digital communication and limited access to clinicians, but the methods often suffer from insufficient explainability. In this study, we propose a novel framework to utilize linguistic dimensions to explain the attribution scores (Integrated Gradients) of tokens in a deep neural network (HerBERT) trained to detect autism and schizophrenia from textual data. In our approach, the scores are mapped to syntactic and psycholinguistic variables, which are then used to train explainable models. The feature importance derived from these models is thereby linked to linguistic dimensions, enhancing interpretability. Our results emphasize the role of syntactic clues, such as grammatical gender and case, in language processing. We also demonstrate that the deep model's attribution scores, when mapped to linguistic features, correlate with selected clinical tests. We propose that our framework can enhance the analysis and interpretation of deep neural networks used for detecting psychiatric disorders. Our approach advances explainable artificial intelligence in mental health by providing researchers with deeper insights into the linguistic cues driving deep neural model predictions.