Predicting depression and unravelling its heterogeneous influences in middle-aged and older people populations: a machine learning approach

Abstract

Background

Aging has become a global trend, and depression, as an accompanying issue, poses a significant threat to the health of middle-aged and older adults. Existing studies primarily rely on statistical methods such as logistic regression for small-scale data analysis, while research on the application of machine learning in large-scale data remains limited. Therefore, this study employs machine learning methods to explore the risk factors for depression among middle-aged and older adults in China.

Methods

Using a two-step hybrid model combining long short-term memory (LSTM) and machine learning (ML), we compared 20 depression risk/protective factors in a balanced panel dataset of middle-aged and elderly Chinese adults (N = 3706; aged 45–94; 64.65% female; 41.20% middle-aged) from the China Health and Retirement Longitudinal Study (CHARLS). Data were collected across five waves (2011, 2013, 2015, 2018, and 2020). The LSTM model predicted risk factors for the fifth wave via data from the preceding four waves. Five ML models were then used to classify depression (yes/no) based on these factors, which included demographic, lifestyle, health, and socioeconomic variables.

Results

The LSTM model effectively predicted depression-related variables (mean square error = 0.067). The average AUC of the five ML models ranged from 0.78 to 0.82. The key predictive factors were disability, life satisfaction, activities of daily living (ADL) impairment, chronic diseases, and self-reported memory. For the middle-aged group, the top three factors were disability, life satisfaction, and chronic diseases; for the Older people group, they were life satisfaction, chronic diseases, and ADL impairment.

Conclusion

The two-step hybrid model ("LSTM + ML") effectively predicted depression over 2 years via demographic and health data, aiding early diagnosis and intervention.

Introduction

Depression is one of the most prevalent mental health issues worldwide, exerting a significant effect on the burden of diseases worldwide [1]. Approximately 311 million people have been diagnosed with depression globally [2], representing an increase of over 18% over the past decade [3, 4]. Depression ranks as the second leading cause of disability worldwide, accounting for 3.0% of global disability-adjusted life years [5]. Given its distinctive physiological characteristics and social implications, the prevalence of depression is highest among middle-aged and Older people individuals aged 55–74 years [6, 7]. Consequently, there is a pressing need for heightened societal attention to the potential risks of depression within this demographic.

Owing to the sizable and rapidly growing Older people population in China, the prevalence of depression is relatively high [8]. Conservatively estimated, the number of depression patients in China has exceeded 54 million, with a depression incidence rate of 22.7% among those aged 60 and above. However, more than 90% of patients fail to receive regular treatment promptly [9]. Depression typically has a severe effect on the psychological and physical health of Older people individuals, compromising their quality of life, increasing mortality rates, and incurring substantial healthcare costs [10]. Therefore, early prediction of depression risk among future Older people individuals has significant implications for hospitals, nursing homes, communities, and families, facilitating the implementation of timely and effective prevention and intervention measures to significantly increase the quality of life of Older people individuals and reduce medical expenses [11].

Currently, research on the prediction of depression among the Older people population has focused primarily on several aspects. First, the diagnosis of depression via biological indicators, although highly accurate, fails to effectively predict future depression risk [12, 13]. Second, cross-sectional statistical data are employed to explore the relationships between depression and risk factors; however, such data cannot accurately predict future depression risk [14]. Third, depression analysis is conducted via semantic recognition and data mining techniques on the basis of information posted by users on social media platforms; however, the applicability of this method is limited, particularly for the Older people population. The utility of these three types of research findings remains constrained, especially concerning early intervention and timely treatment for depression. Hence, the prediction of depression among different Older people populations via demographic characteristics and general health information is highly important.

In the present study, risk factors for depression in Older people individuals can be categorized into three main groups as follows: (a) demographic factors such as age [15], sex [16], marital status [17], and place of residence [18]; (b) health-related risk factors such as self-rated health status [19], activities of daily living (ADL) impairment [20], alcohol consumption [21], smoking [22], sleep duration [23], and social engagement [24]; and (c) risk factors related to chronic diseases such as cardiovascular diseases [25], kidney diseases [26], diabetes [27], arthritis [28], and dementia [29]. Although many studies have analysed the interrelationships among risk factors for depression, many methods, such as the logistic model, have been limited to linear relationships. In reality, these factors are temporally correlated. However, previous research has focused mainly on individual time points, often failing to capture temporal changes, potentially failing to fully reflect the dynamic and stable effects of these features on depression. Machine learning and deep learning methods are iterative and can analyse the nonlinear and high-dimensional correlations among risk factors simultaneously, capturing the temporal relationships among them [30]. While some studies have used long short-term memory (LSTM) models to capture the temporal changes in predictor variables [31, 32], their classification models are still limited by conventional machine learning models, which have relatively weak capabilities in handling nonlinear and high-dimensional correlations, resulting in limited overall performance. Therefore, we introduce a deep learning classification method, a convolutional neural network (CNN). CNN, through convolution operations and hierarchical feature extraction, can better capture the complex relationships among risk factors for depression, thereby improving predictive performance [33].

To the best of our knowledge, longitudinal studies on the prediction of depression in different age groups over the coming years are relatively scarce. Research has focused primarily on the Older people population [31, 32, 34], with comparatively less attention given to middle-aged individuals, thus resulting in a dearth of exploration into the differences in risk/protective factors for depression prediction between middle-aged and Older people populations. Middle-aged and Older people individuals exhibit significant differences in life stages, physical health status, and social relationships. Middle-aged individuals often experience psychological adaptation issues and anxiety when facing life changes such as retirement [35] and when children become independent [36], whereas Older people individuals are more susceptible to the effects of physical health problems, social isolation, and widowhood [37]. Additionally, middle-aged individuals may prioritize quality of life and identity [38], whereas Older people individuals may place more emphasis on physical health and family ties [39]. These differences may influence the incidence and manifestation of depression, leading to misjudgement of depression risk and inappropriate intervention measures. Therefore, there is an urgent need for a comparative analysis of depression risk factors in these two age groups through longitudinal prediction.

Currently, there is limited research on estimating depression among family-based Older people populations. However, owing to the prevailing Confucian ethic of"filial piety"in China, family-based care will remain the primary choice for Older people individuals in the future [40]. Thus, this study utilizes a large-scale representative database of Older people families in China as a sample to investigate the following issues: (1) Utilizing long short-term memory (LSTM) models to capture time series information and predict the levels of different depression risk factors in middle-aged and Older people populations over the next two years; (2) comparing the performance of five machine learning algorithms in predicting depression episodes among Older people individuals to identify the most effective one for subsequent risk factor interpretation; (3) identifying which features are risk/protective factors for depression in middle-aged and Older people populations via longitudinal data from a large community-based dataset in China, including demographic, lifestyle, and health status data; and (4) investigating differences in depression risk/protective factors between middle-aged (ages 45–60) and Older people (aged 60 +) individuals.

Materials and methods

Data description

This study utilized data from the China Health and Retirement Longitudinal Study (CHARLS) for analysis. CHARLS is a large-scale interdisciplinary survey project representative of China [41]. The CHARLS study obtained ethical approval from Peking University’s Biomedical Ethics Committee (IRB 00001052–11015), and all participants provided informed consent before data collection. For each wave of data, we conducted Little’s MCAR test on all variables, confirming that the missing data followed a Missing Completely at Random (MCAR) pattern. Given this, missing values could be safely deleted or imputed [23]. We opted to exclude samples with missing values in either independent or dependent variables from the 2011, 2013, 2015, 2018, and 2020 waves. As shown in Fig. 1, after removing cases with missing data and attrition, we obtained a balanced panel dataset comprising 3,706 middle-aged and older adults, ranging in age from 45 to 94 years, with 64.65% being female. Of the total sample, middle-aged individuals (45–60 years) accounted for 41.20%, while older adults (60 + years) constituted 58.80%. When processing the data, we did not exclude participants with depression in any of the four waves from 2011–2018. While excluding baseline depression cases might aid in model interpretation, the dynamic nature of depression over time is a crucial aspect of our study. Moreover, in real-world scenarios, individuals with depression are part of the population and should be included in analyses [31, 34].

Fig. 1

A flow chart for study population selection (a balanced five-wave panel was constructed without intentionally excluding participants with depression in 2011, 2013, 2015, and 2018)

Outcome variables

In this study, the Chinese version of the Center for Epidemiologic Studies Depression Scale- 10 (CES-D10) was utilized to assess participants'levels of depression [42]. The Chinese version of the CES-D10 is a commonly used 10-item scale in epidemiological research. Consistent with previous studies [32, 43], participants scoring 11 or higher on the CES-D10 were classified as having depression [44].

Predictor variables

In our study, owing to the varying structure of the questionnaire variables across different waves, we opted for the same predictor variables as those used in Waves 1–5. Ultimately, following similar studies in the literature [31, 34, 45], we selected 18 variables (grouped into three categories) as predictor factors. The detailed information on these variables is as follows:

(a) Demographic variables included age, sex, rural/urban community, and marital status. Age and sex were considered auxiliary inputs, as they are variables that do not need to be predicted over time. Marital status was categorized as married (married/cohabiting) or unmarried (never married/divorced/separated and widowed) based on the 2011 China Health Statistics Yearbook.

(b) Socioeconomic variables included education level, occupational status, and medical insurance (yes/no). Education level was divided into low (primary school and below) and high (junior high school and above). Occupational status was classified into agricultural work, nonagricultural work, and retired/not working.

(c) Lifestyle and health-related variables included life satisfaction, self-rated health status, social activities in the past month, self-rated memory, smoking, drinking, medical service experience in the past month, sleep duration, activities of daily living (ADL) difficulties, chronic diseases, and disability as predictor factors. Consistent with prior research, ADL impairment was measured by asking participants if they encountered difficulties in activities such as bathing, dressing, eating, getting in and out of bed, using the toilet, engaging in bowel movements, performing housework, making phone calls, taking medications, shopping, and managing finances over the past three months, with total scores ranging from 20–80. Social activity experiences (in the past month) encompassed interactions with friends, community recreation, volunteering or charity work, training, and stock fund investments, among others. Chronic diseases were defined on the basis of whether the participants were diagnosed with cardiovascular diseases, kidney diseases, liver diseases, digestive system diseases, asthma, arthritis, and mental and memory disorders, among other nine chronic illnesses. Medical service experience referred to whether participants had visited hospitals, clinics, or outpatient departments in the past month.

Statistical analysis

Model and evaluation

This study's depression prediction task is based on balanced panel data and involves two main steps (Fig. 2). Given that our analysis involves both longitudinal (LSTM) and cross-sectional (machine learning) models, ensuring data consistency across waves is crucial. LSTM requires complete temporal sequences to effectively learn time-dependent patterns, whereas traditional ML models such as RF, logistic regression, and XGBoost are designed for independent observations rather than sequential data. To maintain model comparability and ensure valid inference, we opted to construct a balanced panel dataset. Additionally, Little’s MCAR test confirmed that missing data followed a completely random pattern (MCAR), allowing for safe deletion or imputation of missing values. Based on this, we excluded samples with missing values across all waves (2011, 2013, 2015, 2018, and 2020) to ensure a consistent dataset for both deep learning and machine learning models.

Fig. 2

The architecture of the hybrid model for the estimation of depression. The raw dataset used Waves 1–4 depression data, and the output predicted data of Wave 4 and the outcome of Wave 5 were constructed into a new dataset. MSE: mean squared error; ML: machine learning; LR; logistic regression; XGBOOST: extreme gradient boosting; RF: random forest; SVM: support vector machine; CNN: convolutional neural network

In the first stage, we use the first four waves of data as the original dataset and employ an LSTM algorithm to predict the feature values for 2020. Specifically, we use waves 1–3 as features and the depression status from wave 4 as the label to train and test the LSTM model. Using the trained model, we predict the feature variables for the fifth wave (2020) on the basis of the first four waves of data.

In the second stage, we employ five common machine learning (ML) methods to investigate whether the features predicted from waves 1–4 can accurately predict the depression status in wave 5 (2020). Specifically, in this stage, we combine the LSTM-predicted feature variables with age, gender (time-invariant variables), and depression status from wave 5 to form a new dataset. Using the depression status from wave 5 as the label, we perform tenfold cross-validation to tune the hyperparameters of the five ML methods and empirically analyse the classification performance of these methods in predicting depression.

For each analysis, the entire dataset was randomly split into a training set (80% of the total sample) and a test set (20%). Following standard machine learning protocols, we conducted tenfold cross-validation and hyperparameter tuning on the training data. Details of the LSTM hyperparameter tuning can be found in Supplementary Table 1, while the hyperparameter tuning for the five machine learning models is presented in Supplementary Table 2. Model performance was assessed via accuracy, sensitivity, positive predictive value (PPV), and area under the ROC curve (AUROC) metrics. Additionally, the Brier score was selected as a calibration metric. The development and evaluation of machine learning methods were carried out via Python 3.8, which incorporates the PyTorch and Scikit-learn packages. Feature importance was determined via the SHAP package to calculate Shapley values, which were visualized through bee swarm plots and bar charts.

LSTM

Traditional machine learning models are typically designed to handle static data, where each sample is assumed to be independently and identically distributed, without considering temporal dependencies. As a result, these models often struggle with processing time-series data. Long Short-Term Memory (LSTM) networks address this limitation through memory cells and a gating mechanism, including the input gate, forget gate, and output gate, which enable the model to store long-term information while filtering out irrelevant details. In our study, LSTM is used to capture how chronic diseases, SAPH, and LS evolve over time, identifying patterns in their longitudinal interactions. The forget gate determines which information should be discarded, the input gate regulates the incorporation of new information, and the output gate generates the current hidden state. This structure allows LSTM to effectively capture dynamic temporal relationships, mitigate the vanishing gradient problem, and achieve superior performance in time-series forecasting tasks. [46].

Logistic regression

Logistic regression (LR) is a common statistical learning method used for handling binary classification problems. LR models the log-odds function to map the linear combination of input features to probability values between [0, 1]. It is characterized by its simplicity, intuitiveness, and computational efficiency, but it has a limited ability to model nonlinear relationships in data.

Random forest

RF is a robust ensemble learning algorithm that makes predictions based on a collection of decision trees. Each decision tree is constructed by randomly selecting features and data samples and then combining the predictions of each tree through voting or averaging. This ensemble approach helps reduce overfitting and demonstrates good adaptability to high-dimensional data and datasets with complex interactions. Unlike LSTM, which captures temporal dependencies, RF is well-suited for analyzing non-sequential structured data and identifying key predictors through feature importance analysis. RF performs well in both classification and regression problems and exhibits efficiency and scalability when dealing with large datasets [47].

XGBOOST

XGBoost is a variant of the gradient-boosting decision tree (GBDT), which leverages gradient-boosting techniques to optimize model performance during training. Compared with traditional gradient boosting algorithms, XGBoost introduces regularization terms to prevent overfitting and employs more efficient approximation algorithms to increase training speed [47].

Convolutional neural network

A convolutional neural network (CNN) is a type of deep learning model that extracts features via components such as convolutional layers, pooling layers, and fully connected layers, and multiple such components are stacked to construct a deep network. CNNs can automatically learn features and perform well when handling large-scale datasets [33].

Results

Univariate correlation analysis of predictors of depression

Table 1 displays the results of univariate correlation analysis between predictive factors in 2018 and depression in 2020. The cross-sectional data from the 2020 China Health and Retirement Longitudinal Study included a total of 1527 (41.2%) middle-aged individuals and 2179 (58.8%) Older people individuals, comprising a total of 3706 participants. Among them, 1359 (18.8%) were diagnosed with depression. The prevalence of depression was significantly greater in females (42%) than in males (28%). In 2018, predictive factors such as rural/urban community status, marital status, education level, occupational status, medical insurance, life satisfaction, self-reported health status, social activities, smoking, alcohol consumption, self-rated memory, medical services, sleep duration, ADL impairment, chronic diseases, and disabilities exhibited significant between-group differences in the occurrence of depression in 2020.

Table 1 Predictors in 2018 and univariate analysis of their associations with depression in 2020

Prediction ability of ML models for middle-aged and Older people individuals with depression

The predictive performance of the LSTM is illustrated in Fig. 3. The mean squared error (MSE) of the validation set training curve remains stable at 0.067, indicating good predictive performance of the LSTM. Table 2 summarizes the performance of various machine learning (ML) models in predicting the development of depressive symptoms among complete data participants during sensitivity analysis.

Fig. 3

Predictive performance of LSTM for Depression. (LSTM training curve. The x-axis represents the number of epochs, and the y-axis represents the loss (mean square error, MSE). The blue line represents the training loss, and the orange line represents the validation loss.)

Table 2 Model performance in predicting courses of depressive symptoms for sensitivity analysis on only participants with complete data

Among the evaluated models, the deep learning model CNN achieves the highest accuracy (76.4%), surpassing that of general machine learning models, with the support vector machine (SVM) model achieving the highest accuracy (74.2%) and the random forest model the lowest. Sensitivity refers to the ability to correctly identify diseased individuals among all actual patients. The CNN achieves the highest sensitivity at 58.1%, followed by the SVM at 50.0%. The positive predictive value (PPV) indicates the proportion of actual patients among all diagnosed patients (higher is better). The PPV ranges from 68.3% (random forest) to 72.2% (CNN). The Brier score, which measures the mean squared difference between the predicted probabilities and actual outcomes, is lowest for the CNN at 0.161. The AUC (area under the curve) refers to the area under the ROC curve and is used to assess the performance of binary classifiers, with values closer to 1 indicating better classifier performance. The range spans from 0.775 (random forest) to 0.804 (CNN), with the CNN exhibiting the best performance (Fig. 4A). The results of the decision curve analysis are shown in Fig. 4B. Within the threshold range of 0–1, the net benefit of the CNN is significantly greater than that of other machine learning models.

Fig. 4

Predictive performance of the five machine learning models for depression. A ROC curves for the five machine learning models. The x-axis represents specificity (probability of a negative test given that the elderly did not have depression), and the y-axis represents sensitivity (probability of a positive test given that the elderly had depression). B Decision Curve Analysis for Five Machine Learning Models. The x-axis represents the threshold probability of depression, and the y-axis represents the net benefit.)

Overall, the CNN outperforms general machine learning models across all the metrics. Its significant advantage lies in handling complex nonlinear data patterns, attributed to its convolutional and pooling layer structure. By extracting features at different levels through multiple convolutional and pooling layers, CNNs achieve richer and more advanced data representations. This end-to-end learning approach enables CNNs to adapt better to data features and extract the most discriminative and predictive features, thereby enhancing model performance and generalization ability [48].

Although CNN models exhibit good overall performance, their impact in the field of data analysis remains limited because of the black-box nature of deep learning methods [48, 49]. In recent years, researchers have started using Shapley values to assess the relative contributions of each predictor in predictive models and elucidate their impact on outcomes [50]. The greatest advantage of Shapley values lies in their independence from the predictive model, making them applicable to any machine learning model [51]. Therefore, this study employs Shapley values to interpret the model results.

Importance of predictive variables for middle-aged and older people populations

This study employed the SHAP method to conduct an interpretive analysis of the CNN model with the best performance. Figure 5 presents the importance of various predictive factors within the CNN model in descending order. Figure 5A depicts the bee swarm plot, where SHAP values illustrate the impact of each feature on the model output (i.e., depression prediction) and demonstrate how these impacts vary with feature values. The y-axis represents the evaluated features, while the colors indicate the magnitude of the feature values: the farther the points are from the x-axis, the greater the impact of the feature on depression prediction. Figure 5B represents the SHAP summary bar plot, where average SHAP values display the average impact of each feature, providing a more intuitive view of each feature's contribution to the overall prediction. The results indicate that disability, life satisfaction, activities of daily living (ADL), self-rated health status, and self-reported memory are the five key predictive factors influencing depression in middle-aged and Older people individuals.

Fig. 5

Variable importance of the LSTM + CNN model for predicting depression in middle-aged and elderly populations. A SHAP values for middle-aged and elderly individuals. B Mean SHAP values for middle-aged and elderly individuals.)

Comparison of feature importance between RF and CNN

To evaluate the strength of the association between the dependent variable and key predictors, as well as the heterogeneity in feature selection across models, we first computed feature importance in the Random Forest (RF) model using impurity-based importance. This method quantifies each feature’s contribution by measuring its reduction in impurity, as represented by the Gini coefficient. The global feature importance ranking of the RF model is presented in Fig. 6C.

Fig. 6

Differences in Predictor Importance Between CNN and RF Models (6 A: Feature importance in the CNN model computed using SHAP; 6B: Feature importance in the RF model computed using SHAP, where Class 1 represents depression and Class 0 represents non-depression; 6 C: Feature importance in the RF model computed using impurity-based methods.)

Furthermore, we compared the feature importance derived from SHAP values with those obtained from the aforementioned methods (Fig. 6B vs. Figure 6C). The results indicate a high degree of consistency across different evaluation techniques, particularly for Disability, Life Satisfaction, Chronic Disease, and Activities of Daily Living (ADL), which consistently demonstrate high importance. This suggests that these variables exert a stable influence within the RF model.

A comparative analysis of CNN (Fig. 6A) and RF (Fig. 6B) based on SHAP importance reveals notable differences in feature importance rankings. While both models prioritize functional health indicators—such as ADL, self-rated health, and chronic diseases—while assigning relatively lower predictive significance to demographic characteristics (e.g., gender, age) and health behaviors (e.g., smoking, alcohol consumption), a key distinction emerges: CNN exhibits significantly higher heterogeneity in feature importance distribution, whereas RF demonstrates a more uniform weight distribution pattern.

Differences in the importance of predictive variables for middle-aged and Older people populations

On the basis of the age cut-off of 60 years in 2018, participants were divided into middle-aged and Older people samples for grouping predictions and comparisons. Figures 7A and 7C display the importance of predictive variables for middle-aged individuals, whereas Figs. 7B and 7D present the importance for Older people individuals. The results indicate discrepancies in the top three crucial predictive variables between the two age groups. Disability, life satisfaction, and chronic illness were identified as the top three predictors for the middle-aged group, whereas life satisfaction, chronic illness, and activities of daily living (ADL) impairment were prioritized for the Older people group. However, the top five crucial predictive variables were consistent across both age groups and aligned with the previously identified top five predictors from the combined middle-aged and Older people data.

Fig. 7

Differences in feature importance for the middle-aged and elderly groups in the LSTM + CNN model (A, SHAP values for the middle-aged group. C, Mean SHAP values for the middle-aged group. B, SHAP values for the elderly group. D, Mean SHAP values for the elderly group.)

Discussion

Divergence in model performance

This study utilized panel data from the first to fifth waves (2011, 2013, 2015, 2018, and 2020) of the China Health and Retirement Longitudinal Study (CHARLS), covering 3,706 middle-aged and Older people individuals aged 45 years and above. The dataset encompasses various variables, including demographics, socioeconomic status, lifestyle, and health conditions. By employing an LSTM model, we forecasted the risk factors for the fifth wave on the basis of the preceding four waves and employed five machine learning models, namely, logistic regression, XGBoost, random forest, support vector machine (SVM), and a convolutional neural network (CNN), for depression classification. The results indicated that the combined analysis framework of LSTM and other machine learning models exhibited promising performance in predicting depression among Older people individuals. Specifically, LSTM effectively captures long-term dependencies in time series through its gating mechanism, extracting dynamic patterns of health, psychological, and social features. For example, it learns how chronic disease progression affects LS over time, how reduced SAPH further accelerates this decline, and how prolonged LS deterioration eventually elevates depression risk. These findings suggest that LSTM not only captures temporal dependencies but also models how different risk factors interact dynamically over extended periods. On the basis of the LSTM-predicted features, the CNN demonstrated the best performance in depression prediction. In our designed predictive model, we observed an AUC ranging from 0.775–0.804 and an accuracy ranging from 0.732–0.764. Compared with previous studies [32, 34], our model exhibited higher accuracy.

Feature importance in middle-aged and older adults

We further utilized SHAP analysis to assess the important features in depression prediction. The results revealed that disability, life satisfaction, activities of daily living (ADL) impairment, self-rated health status, and self-reported memory were the top five crucial predictive factors for depression in middle-aged and Older people individuals. Disability emerged as the most significant predictor in the model, limiting the daily functioning of middle-aged and Older people individuals. This not only increases their life challenges but is also associated with reduced social activities, feelings of loneliness, and impacts on self-identity, thus predisposing them to emotional distress and depression. Consistent with previous research [52], our study provides support for the relationship between disability and depressive symptoms. Life satisfaction ranked as the second most important predictive factor. During middle and old age, individuals face retirement, signifying the beginning of the loss of status, power, and prestige [19]. Changes in social and life roles may lead to psychological burdens, potentially affecting the enjoyment of life, positive self-perception, and optimism, thereby reducing life quality and satisfaction [19]. Additionally, ADL impairment has been consistently associated with depressive symptoms in older adults. Feeling incompetent or dependent on others in daily activities such as cooking and bathing may lead to lowered self-esteem and psychological burdens. According to Yaka, reporting poor health status often leads to worries about the consequences of illness and emotional distress [53]. Therefore, it is reasonable to infer that poor health status may not only be associated with physical functional impairments but also serve as a stressor, exacerbating emotional distress and triggering or worsening depressive symptoms [19, 54]. Consistent with prior research [47], self-rated health status impairment has also been identified as an important predictor of depression [19]. Finally, our study suggests that while basic demographic characteristics (marital status), health behaviors (smoking, drinking), and social support (medical insurance and healthcare support) are factors influencing depressive symptoms, subjective quality of life and health status assessment are the most critical factors affecting depressive symptoms.

Discrepancies in feature importance between CNN and RF models

The differences in feature importance between CNN and RF models may stem from their distinct architectures. CNNs learn local patterns through convolutional layers and capture complex high-order feature interactions. As a result, their SHAP importance is often concentrated on a few key variables. For instance, Disability and Life Satisfaction may contain higher-level feature patterns, leading CNNs to assign greater importance to these features.

In contrast, RF employs a decision-tree-based structure, where SHAP calculations depend on the contribution of features to the decision tree splitting process. Consequently, RF places more emphasis on the independent impact of individual features on model decisions. For example, Social Activities and Drinking may exert a greater influence on specific category predictions, resulting in higher feature importance in RF.

Differential feature importance between middle-aged and older cohorts

Our study also explored differences in depression prediction between middle-aged and Older people populations. While the top five important predictive factors were similar between the two age groups, their specific importance varied slightly. For most middle-aged individuals, work and family responsibilities are the primary focus, with disabilities directly impacting their work and daily functioning, leading to increased life burdens and psychological stress [55]. Conversely, in the Older people population, the three most important predictive factors were life satisfaction, chronic diseases, and activities of daily living (ADL). Older people individuals tend to prioritize overall life satisfaction, not just basic health conditions [56]. Compared with middle-aged individuals, Older people individuals are more prone to chronic diseases such as hypertension, diabetes, and cardiovascular diseases [57]. These conditions not only affect their physical health but also may lead to increased life stress, decreased quality of life, and heightened risk of depression [58].

This study reveals the potential of involving families, communities, and nonprofessionals in the prevention and treatment of depression. By understanding the risk factors for depression, we can assist middle-aged and Older people individuals in better coping with emotional distress. Training healthcare professionals in early intervention and screening, as well as developing user-friendly depression screening tools, will contribute to improved treatment outcomes. Therefore, this research provides practical support and guidance for the prevention and treatment of depression among middle-aged and Older people people.

Limitations and future directions

This study utilized data from the China Health and Retirement Longitudinal Study. While we selected some representative features from different perspectives, the features were limited. Future research could integrate biological, genetic, neuroimaging, life events, and other available features for further investigation. Additionally, owing to the limited timeframe of the study, the LSTM model may not capture the long-term factors and dynamic changes influencing depression. Future research could address this limitation by incorporating more waves of data. To construct a balanced panel dataset, we excluded samples with missing values. Little’s MCAR test confirmed a completely random missing pattern, suggesting that future research could apply imputation methods to obtain a more complete and representative dataset. Finally, key depression risk factors—such as chronic disease count, health satisfaction, and life satisfaction—may have complex interrelations [59]. Future research could leverage longitudinal data to explore their underlying mechanisms and causal pathways, deepening the understanding of depression risk. ADL impairment patterns may differ between chronically ill and healthy older adults [60]. Future research could refine analysis using sub-models: (1) no ADL difficulties and (2) ADL impairment or need for assistance.

Conclusions

The LSTM + ML model successfully captured high-dimensional and time series information on depression risk factors in middle-aged and Older people populations. There are notable differences in depression risk factors between middle-aged and Older people individuals. The predictive model developed in this study holds significant value for the early detection and intervention of depression for healthcare professionals, including doctors, nurses, and community healthcare providers.