Validity and Reliability of Study of Osteoporotic Fractures Index in the Diagnosis of Sarcopenia in Turkish Geriatric Patients
PDF
Cite
Share
Request
Original Research
VOLUME: 5 ISSUE: 3
P: 255 - 261
December 2023

Validity and Reliability of Study of Osteoporotic Fractures Index in the Diagnosis of Sarcopenia in Turkish Geriatric Patients

Eur J Geriatric Gerontol 2023;5(3):255-261
1. University of Health Sciences Turkey, Basaksehir Cam and Sakura City Hospital, Clinic of Internal Medicine, Division of Medical Oncology, Istanbul, Turkey
2. Istanbul University-Cerrahpasa, Cerrahpasa Faculty of Medicine, Department of Internal Medicine, Division of Geriatrics, Istanbul, Turkey
3. Cizre District Health Directorate, Sırnak, Turkey
No information available.
No information available
Received Date: 19.03.2023
Accepted Date: 03.07.2023
Publish Date: 12.09.2023
PDF
Cite
Share
Request

ABSTRACT

Objective:

The objective of this study was to evaluate the reliability and validity of the Turkish version of the study of osteoporotic fractures (SOF) index, and to investigate the prognostic power of the SOF index in predicting the risk for sarcopenia in the geriatric population.

Materials and Methods:

The sample of this cross-sectional study consisted of 144 geriatric patients who applied to the outpatient clinic where this study was conducted between July 2017 to July 2018. The frailty status of patients was evaluated using the SOF index. The European Working Group on Sarcopenia in Older People diagnostic criteria were used in the diagnosis of sarcopenia. Accordingly, patients were divided into two groups as the sarcopenia and non-sarcopenia groups. Patients’ measurement results were recorded and comparatively analyzed between the groups.

Results:

The rate of sarcopenia was significantly higher in patients who were determined to be frail based on SOF index than the remaining patients (93.2% vs. 61.5%, respectively; p<0.001). The kappa value was determined as 0.608 based on the qualitative data, in substantial agreement with Cohen’s Kappa coefficient, indicating reliability. The ROC analysis revealed that the sensitivity and specificity of SOF index cut-off value of 1 in determining sarcopenia were 76.4% and 55.6%, respectively. The validity of the SOF index with a cut-off value of 1 was found as 0.659 (validity values of >0.5 indicate statistical significance).

Conclusion:

The study findings indicate that the SOF index is a feasible, valid and reliable tool, and it has a high positive prognostic value in predicting sarcopenia.

Keywords:
Frailty, reliability, sarcopenia, SOF index, validity

Introduction

The incidence of frailty and sarcopenia, both of which limit mobility, increases with age. The coexistence of frailty and sarcopenia in older patients is correlated with a higher incidence of recurrent hospitalizations, multiple drug therapies, and hospital admissions, increasing the risk for morbidity and mortality. Considering the rates of sarcopenia in the Middle East and European countries, frailty and sarcopenia rates were reported as 28.3% and 31.7%, respectively, in a study conducted with a large patient cohort in Turkey (1), and as 51.7% and 34.4%, respectively, in a study conducted in Israel (2,3).
Similarly, the frailty rate of geriatric patients in Saudi Arabia was reported as 29.2% (4), whereas the prevalence of sarcopenia was found to be 32.5% in a population-based multi-center study conducted in Iran (5). These rates were found to be lower in European countries. In a study evaluating frailty in 10 European countries and another study evaluating sarcopenia in 28 European countries, frailty, and sarcopenia rates were found as 5.8-27.3% and 11.2-20.2%, respectively (6,7).

Sarcopenia is a condition associated with aging resulting in an involuntary loss of skeletal muscle mass, reducing skeletal muscle function and strength (8). In parallel, Rogers and Evans (9)
reported that total muscle strength decreases by 30% and muscle mass decreases by 40% over the period between the second and seventh decades of life. Sarcopenia is considered one of the geriatric syndromes in addition to falls, delirium, and incontinence, which all negatively affect the quality of life (10,11).

Frailty is a multifaceted syndrome characterized by clinical deficiencies such as mobility, strength, endurance, nutrition, physical activity, physiological reserve, cognitive impairment, and depression (12,13). Early diagnosis of frailty in older patients is important in terms of taking the necessary preventive measures and determining the appropriate treatment approaches. To this end, various simple screening tools have been developed. One of these tools, the study of osteoporotic fractures (SOF) index, was developed by the SOF research group in 2008 as a feasible, rapid diagnostic test for frailty (14).

In this context, this study was conducted to assess the reliability and validity of the Turkish version of the SOF index in older patients (≥65 years) and to investigate the prognostic power of the SOF index in predicting the risk for sarcopenia in the geriatric population.

Materials and Methods

Population and Sample

The population of this cross-sectional study consisted of 845 patients aged 65 or above who applied to the geriatric outpatient clinic where this study was conducted between July 2017 to July 2018. In the G*Power program, we chose the test family as the chi-square test and the type of power analysis a priori, and when we took the effect size value as 0.3, the alpha margin of error as 0.05, the power of the study as 80%, and the Df value as 2, the minimum number of samples required for our study was 108. Patients with prosthesis or pacemaker that falsified bioelectrical impedance analysis (BIA), and those with advanced dementia, Parkinson’s disease, congestive heart failure, malignancy, rheumatologic disease, inflammatory bowel disease, and neuromuscular disease who could not perform the tests were excluded from the study. In the end, the study sample consisted of 144 patients. Patients were divided into two groups as the sarcopenia and non-sarcopenia groups. Each group consisted of 38 males and 34 females. Patients’ demographic and laboratory characteristics and anthropometric measurement results in addition to walking speed, hand grip strength, and fat-free mass index (FFMI) and SOF index scores were recorded and comparatively analyzed between the groups.

Diagnosis of Sarcopenia

The European Working Group on Sarcopenia in Older People diagnostic criteria were used in the diagnosis of sarcopenia. Accordingly, 72 (8.5%) of the 845 patients who were determined to have low muscle mass, along with low physical performance or low muscle strength, were diagnosed with sarcopenia (10).

Muscle Mass

BIA was used to determine patients’ muscle mass (10). BIA was conducted using the Bodystat Quadscan 4000 brand bioimpedance device (Bodystat Ltd., Isle of Man, UK), while the patients were in the fasting state and their bladders were empty. They were in supine position, provided that their extremities were not in contact with their body, and after the metal accessories were removed from their bodies. To this end, a total of four electrodes, two at the level of the wrist and metacarpophalangeal joint in the upper extremity and two at the level of the ankle and metatarsophalangeal joint in the lower extremity, were connected to the patients. Subsequently, patients’ age, gender, height, weight, waist and hip circumference data and activity levels were entered into the bioimpedance device in a certain order. The device calculated the FFMI based on these data. The study conducted to determine the optimal FFMI cut-off value included 30 healthy males and 30 healthy females between 20-40 years of age with normal body mass index (BMI) values. Consequently, the optimal FFMI cut-off value was determined as 13.4 kg/m2 for females and 17.1 kg/m2 for males, taking standard deviation of the mean FFMI value as “-2”. Similar to the literature, patients with less FFMI values were considered to have low muscle mass (15-17).

Muscle Strength

Hand grip test was performed using a Jamar branded (Model SH500L, Four D Rubber Company Ltd., Derbyshire, UK) hand dynamometer in order to assess muscle strength. To this end, the patients were placed in a flexion position with their elbows on the table and their arms parallel to the floor. Measurements were made three times on both right and left arms with 1-minute rest periods in between. Patients with the highest of the three measurements below 30 kg in males and 20 kg in females were considered to have “low muscle strength”.

Muscle Performance

Patients’ physical performance was assessed based on the walking speed test. To this end, the time that took the patients to walk for 6 meters in standing position was recorded in terms of seconds. A walking speed of 0.8 m/s was considered as the cut-off value and patients of both genders with a walking speed below this value were considered to be at risk for sarcopenia.

Diagnosis of Frailty

SOF Index

The SOF index is a questionnaire that consists of three parameters for frailty diagnosis;

1. Involuntary weight loss of 5% or more within three years (<%5 =1 point; ≥5% =0 point).

2. Inability to rise from a chair five consecutive times without using the arms (<5 times =1 point; 5 times =0 point).

3. Having low energy as identified by a negative answer to the question “do you feel full of energy?” (“no”=1 point; “yes”=0 point.

Accordingly, the older adults who scored 0 point, 1 point and 2-3 points are considered robust older patients, prefrail older patients, and frail older patients, respectively (18).

Validity Studies

The validity of the SOF index was determined by receiver operating characteristic (ROC) curve analysis. For this reason, non-sarcopenic patients with an SOF index value below the optimal cut-off value and sarcopenic patients with an SOF index value above the optimal cut-off value were added and then divided by the total number of patients included in the study. The result obtained was considered significant in terms of validity if above 0.5 (19).

Reliability Studies

The SOF index was administered to 144 patients at baseline, and then to 72 of these 144 patients for a second time within two weeks to measure the test-retest reliability (Cronbach’s alpha). The resulting kappa coefficients between 0.81 and 1.00, 0.61 and 0.80, 0.41 and 0.60, 0.21 and 0.40, 0.01 and 0.2, and below 0 were considered to indicate almost perfect, substantial, moderate, fair, none to slight agreement, and no agreement, respectively (20).

Statistics

Statistical analyses were carried out with SPSS 20.0 (Statistical Package for Social Sciences for Windows, version 20.0, IBM Corp., Armonk, NY, U.S., 2011) software package. Sarcopenia and non-sarcopenia groups were compared using the Pearson’s chi-squared test based on the robust, prefrail and frail classifications made according to the SOF index scores. Pearson’s correlation analysis test was used to examine the relationship between patients’ anthropometric measurement values and SOF index scores. The sensitivity and specificity values of certain SOF index cut-off scores were examined with ROC curve analysis. Probability (p) values of <0.05 were deemed to indicate statistical significance.

Results

The study sample consisted of 144 geriatric patients. Both the sarcopenia and non-sarcopenia groups consisted of 72 patients. Each group consisted of 38 males and 34 females. Distribution of patients’ demographic and laboratory characteristics and anthropometric measurement results in addition to walking speed, hand grip strength, and FFMI and SOF index scores by genders and sarcopenia groups is demonstrated in Table 1.

Sarcopenic and non-sarcopenic patients were compared based on the robust, prefrail, and frail classifications made according to their SOF index scores. Accordingly, there was a significant difference between robust and frail patients (p<0.001), but not between prefrail and frail patients or robust and prefrail patients (p>0.05), in terms of presence of sarcopenia. The comparison of the groups created based on SOF index scores in terms of presence of sarcopenia is shown in Table 2.

The ROC analysis revealed that the sensitivity and specificity of SOF index cut-off value of 1 in determining sarcopenia were 76.4% and 55.6%, respectively.

Additionally, the positive and negative predictive values of SOF index cut-off value of 1 were determined as 63.2% and 70.2%, respectively, indicating that 1 can be used as an optimal cut-off value. The ROC curve analysis of the prognostic power of SOF index in predicting the diagnosis of sarcopenia is shown in Figure 1.

There was a significant negative correlation between the SOF index scores and walking speed, BMI values, right calf, waist, and hip circumferences, FFMI scores, and right hand grip strength.

The SOF index score of 72 patients was measured for a second time within 2 weeks for assessing the reliability of the index, and the kappa coefficient was found as 0.608. The intergroup comparison and ROC analysis revealed that the validity of the SOF index cut-off value of 1 was 0.659 (validity values of >0.5 indicate statistical significance). The details of the reliability and validity studies of the SOF index are shown in Table 3.

Discussion

This study featured the assessment of the validity and reliability of the SOF index for Turkish geriatric population and the prognostic power of the SOF index in predicting the risk for sarcopenia in the geriatric population.

The findings of this study revealed that female patients diagnosed with sarcopenia had lower weight, BMI values, waist and hip circumferences, hand grip strength, and FFMI values, whereas higher SOF index scores, compared to female patients without sarcopenia. On the other hand, male patients diagnosed with sarcopenia were older and had a lower walking speed compared to male patients without sarcopenia.

In a cross-sectional study conducted with 771 geriatric patients, 359 females and 412 males, in Taiwan, 119 patients were diagnosed with sarcopenia. The distribution of the patients with sarcopenia by the BMI values revealed that sarcopenia was most common in the group with high BMI values (16 of the 31 patients with BMI values >30 kg/cm2 were diagnosed with sarcopenia), followed by the group with low BMI values (14 of the 29 patients with BMI values <18.5 kg/cm2 were diagnosed with sarcopenia) (21). In contrast, none of the patients with a high BMI value had sarcopenia (none of the 26 patients with BMI values >30 kg/cm2 was diagnosed with sarcopenia). Sarcopenia was most common in the group with low BMI values regardless of gender (8 of the 9 patients with BMI values <18.5 kg/cm2 were diagnosed with sarcopenia). The findings of this study suggest that weight, waist and hip circumferences in patients with sarcopenia are correlated with low BMI values. The said discrepancy between the study conducted in Taiwan and this study may be due to the differences in the ethnic structure of the respective societies, lifestyles, and geographical differences (22,23).

In a cohort study conducted by De Buyser et al. (14) with 191 Belgian older male adults with a mean age of 78, SOF index was used to assess frailty and hand grip strength, and dual-energy X-ray absorptiometry (DEXA) were used to diagnose sarcopenia, and frailty and sarcopenia were each detected in 7% of the cohort. The rate of sarcopenia patients with frailty was found as 23%. Unlike the said study, female patients were also included in this study and BIA was used instead of DEXA to measure muscle mass, given that BIA is performed at the bedside, is a noninvasive and inexpensive method, and gives results comparable to DEXA (24,25). In addition, walking speed of patients was also measured in the diagnosis of sarcopenia in this study in addition to muscle mass and muscle strength measurements. Consequently, the rates of male and female patients who were found to be both frail and sarcopenic were 71.0% and 82.3%, respectively. In comparison, the rate of patients with both frailty and sarcopenia was lower in De Buyser et al.’s (14) study. In another study conducted with 70 geriatric patients in India with a design comparable to this study, the patients were divided into two groups as the sarcopenia (n=42) and non-sarcopenia (n=28) groups, and the rate of frail patients in the sarcopenia group was found as 66.7% (26). The discrepancies between the findings of the relevant studies available in the literature might be attributed to the differences between the methodologies of these studies and the socio-demographic structures of the geriatric populations of the countries where these studies were conducted.

In a study conducted by Yürüyen et al. with 112 geriatric patients, walking speed and right hand grip strength measurements were performed in addition to muscle mass measurements with BIA to establish the diagnosis of sarcopenia. ROC analysis revealed that walking speed and right hand grip strength predicted the diagnosis of sarcopenia with a sensitivity and specificity of 71% and 47% [area under the curve (AUC)=0.642, p<0.001], and 65% and 50% (AUC=-0.594, p<0.001), respectively (27). Similarly, in this study, the ROC analysis revealed that walking speed and right hand grip strength predicted the diagnosis of sarcopenia with a sensitivity and specificity of 65.8% and 67.6% (AUC=0.688, p<0.001), and 61.1% and 63.9% (AUC=0.687, p<0.001). However, the prognostic power of the SOF index with a cut-off value of 1 (AUC=0.708, p<0.001; 76.4%, 55.6%) in predicting the diagnosis of sarcopenia was higher than those of walking speed and right hand grip strength.

As in many other studies that used indexes as data collection tools, first, intergroup comparison and ROC analysis were performed to assess the discriminant validity of the SOF index in this study (28,29). As a result, the validity of the SOF index with a cut-off value of 1 was found as 0.659 (validity values of >0.5 indicate statistical significance), and the kappa value was determined as 0.608 based on the qualitative data, in substantial agreement with Cohen’s Kappa coefficient, indicating reliability (30).

Study Limitations

Firstly, in this study, there is a small population, which may make the result inaccurate. Secondly, this study was conducted in a single-center, its results may not be widely generalized. However, this study has important contributions to the literature. Firstly, this is the first study that demonstrated that the SOF index can be used for the diagnosis of sarcopenia. Secondly, the SOF index was found to have high sensitivity and specificity in the diagnosis of sarcopenia in the Turkish population.

Conclusion

The rate of patients with both sarcopenia and frailty (measured with the SOF index) was higher in the Turkish geriatric population compared to the literature data. In addition, it was determined that the SOF index can be used with high validity and reliability in the Turkish population, and that it was superior to walking speed, and right hand grip strength in the diagnosis of sarcopenia. Considering that BIA, walking speed and right hand grip strength is relatively more time consuming and difficult to apply for both the physician and the patient in outpatient settings, the SOF index is a promising assessment and screening tool in the diagnosis of sarcopenia.

Ethics

Ethics Committee Approval: Analyses of the clinical data were approved by the Ethics Committee of İstanbul University-Cerrahpaşa, Cerrahpaşa Faculty of Medicine (no: 2017/259929).
Informed Consent: The ethical committee agreed to the analysis of routinely collected clinical data provided that informed consent is obtained from the patients in advance. Accordingly, all patients were fully informed of the study procedures before they gave their consent.
Peer-review: Externally peer-reviewed.

Authorship Contributions

Surgical and Medical Practices: T.K., H.Y., D.S.E., Concept: T.K., H.Y., F.F., D.S.E., A.E.C., Design: T.K., H.Y., F.F., D.S.E., A.E.C., Data Collection or Processing: T.K., H.Y., A.E.C., Analysis or Interpretation: T.K., H.Y., F.F., D.S.E., A.E.C., Literature Search: T.K., H.Y., F.F., A.E.C., Writing: T.K., H.Y., F.F., D.S.E.
Conflict of Interest: No conflict of interest was declared by the authors.
Financial Disclosure: The authors declared that this study received no financial support.

References

1
Ates Bulut E, Soysal P, Isik AT. Frequency and coincidence of geriatric syndromes according to age groups: single-center experience in Turkey between 2013 and 2017. Clin Interv Aging 2018;13:1899-1905.
2
Anani S, Goldhaber G, Brom A, Lasman N, Turpashvili N, Shenhav-Saltzman G, Avaky C, Negru L, Agbaria M, Ariam S, Portal D, Wasserstrum Y, Segal G. Frailty and Sarcopenia Assessment upon HospitalAdmission to Internal Medicine Predicts Length ofHospital Stay and Re-Admission: A ProspectiveStudy of 980 Patients. J Clin Med 2020;9:2659.
3
Lutski M, Weinstein G, Tanne D, Goldbourt U. Overweight, Obesity, and Late-Life Sarcopenia Among Men With Cardiovascular Disease, Israel. Prev Chronic Dis 2020;17:E164.
4
Alqahtani BA. Association between Physical Frailty and Sleep Quality among Saudi Older Adults: A Community-Based, Cross-Sectional Study. Int J Environ Res Public Health 2021;18:12741.
5
Dorosty A, Arero G, Chamar M, Tavakoli S. Prevalence of Sarcopenia and Its Association with Socioeconomic Status among the Elderly in Tehran. Ethiop J Health Sci 2016;26:389-96.
6
Santos-Eggimann B, Cuénoud P, Spagnoli J, Junod J. Prevalence of frailty in middle-aged and older community-dwelling Europeans living in 10 countries. J Gerontol A Biol Sci Med Sci 2009;64:675-681.
7
Ethgen O, Beaudart C, Buckinx F, Bruyère O, Reginster JY. The Future Prevalence of Sarcopenia in Europe: A Claim for Public Health Action. Calcif Tissue Int 2017;100:229-234.
8
Bauer J, Morley JE, Schols AMWJ, Ferrucci L, Cruz-Jentoft AJ, Dent E, Baracos VE, Crawford JA, Doehner W, Heymsfield SB, Jatoi A, Kalantar-Zadeh K, Lainscak M, Landi F, Laviano A, Mancuso M, Muscaritoli M, Prado CM, Strasser F, von Haehling S, Coats AJS, Anker SD. Sarcopenia: A Time for Action. An SCWD Position Paper. J Cachexia Sarcopenia Muscle 2019;10:956-961.
9
Rogers MA, Evans WJ. Changes in skeletal muscle with aging: effects of exercise training. Exerc Sport Sci Rev 1993;21:65-102.
10
Cruz-Jentoft AJ, Baeyens JP, Bauer JM, Boirie Y, Cederholm T, Landi F, Martin FC, Michel JP, Rolland Y, Schneider SM, Topinková E, Vandewoude M, Zamboni M; European Working Group on Sarcopenia in Older People. Sarcopenia: European consensus on definition and diagnosis: Report of the European Working Group on Sarcopenia in Older People. Age Ageing 2010;39:412-423.
11
Cooper C, Dere W, Evans W, Kanis JA, Rizzoli R, Sayer AA, Sieber CC, Kaufman JM, Abellan van Kan G, Boonen S, Adachi J, Mitlak B, Tsouderos Y, Rolland Y, Reginster JY. Frailty and sarcopenia: definitions and outcome parameters. Osteoporos Int 2012;23:1839-1848.
12
Cesari M, Calvani R, Marzetti E. Frailty in Older Persons. Clin Geriatr Med 2017;33:293-303.
13
Dent E, Kowal P, Hoogendijk EO. Frailty measurement in research and clinical practice: A review. Eur J Intern Med 2016;31:3-10.
14
De Buyser SL, Petrovic M, Taes YE, Toye KR, Kaufman JM, Lapauw B, Goemaere S. Validation of the FNIH sarcopenia criteria and SOF frailty index as predictors of long-term mortality in ambulatory older men. Age Ageing 2016;45:602-608.
15
McIntosh EI, Smale KB, Vallis LA. Predicting fat-free mass index and sarcopenia: a pilot study in community-dwelling older adults. Age (Dordr) 2013;35:2423-2434.
16
Campbell TM, Vallis LA. Predicting fat-free mass index and sarcopenia in assisted-living older adults. Age (Dordr) 2014;36:9674.
17
Atkins JL, Whincup PH, Morris RW, Lennon LT, Papacosta O, Wannamethee SG. Sarcopenic obesity and risk of cardiovascular disease and mortality: a population-based cohort study of older men. J Am Geriatr Soc 2014;62:253-260.
18
Ensrud KE, Ewing SK, Taylor BC, Fink HA, Cawthon PM, Stone KL, Hillier TA, Cauley JA, Hochberg MC, Rodondi N, Tracy JK, Cummings SR. Comparison of 2 frailty indexes for prediction of falls, disability, fractures, and death in older women. Arch Intern Med 2008;168:382-389.
19
Hawkins M, Elsworth GR, Osborne RH. Questionnaire validation practice: a protocol for a systematic descriptive literature review of health literacy assessments. BMJ Open 2019;9:e030753.
20
McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012;22:276-282.
21
Meng NH, Li CI, Liu CS, Lin CH, Lin WY, Chang CK, Li TC, Lin CC. Comparison of height- and weight-adjusted sarcopenia in a Taiwanese metropolitan older population. Geriatr Gerontol Int 2015;15:45-53.
22
Dorhout BG, Overdevest E, Tieland M, Nicolaou M, Weijs PJM, Snijder MB, Peters RJG, van Valkengoed IGM, Haveman-Nies A, de Groot LCPGM. Sarcopenia and its relation to protein intake across older ethnic populations in the Netherlands: the HELIUS study. Ethn Health 2022;27:705-720.
23
Zhou Y, Wu K, Shen H, Zhang J, Deng HW, Zhao LJ. Geographical differences in osteoporosis, obesity, and sarcopenia related traits in white American cohorts. Sci Rep 2019;9:12311.
24
Augustemak de Lima LR, Rech CR, Petroski EL. Utilização da impedância bioelétrica para estimativa da massa muscular esquelética em homens idosos [Use of bioelectrical impedance for the estimation of skeletal muscle mass in elderly men]. Arch Latinoam Nutr 2008;58:386-391.
25
Beshyah SA, Freemantle C, Thomas E, Johnston DG. Comparison of measurements of body composition by total body potassium, bioimpedance analysis, and dual-energy X-ray absorptiometry in hypopituitary adults before and during growth hormone treatment. Am J Clin Nutr 1995;61:1186-1194.
26
Bhurchandi S, Kumar S, Agrawal S, Acharya S, Jain S, Talwar D, Lomte S. Correlation of Sarcopenia With Modified Frailty Index as a Predictor of Outcome in Critically Ill Elderly Patients: A Cross-Sectional Study. Cureus 2021;13:e19065.
27
Comparison of nutritional risk screening tools for predictingsarcopenia in hospitalized patients. Turk J Med Sci 2017;47:1362-1369.
28
Dawkins RL, Miller JH, Menacho ST, Ramadan OI, Lysek MC, Kuhn EN, Tubbs RS, Walker ML, Walters BC, Agee BS, Rozzelle CJ. Thoracolumbar Injury Classification and Severity Score in Children: A Validity Study. Neurosurgery 2019;84:E362-E367.
29
Yue Y, Liu R, Lu J, Wang X, Zhang S, Wu A, Wang Q, Yuan Y. Reliability and validity of a new post-stroke depression scale in Chinese population. J Affect Disord 2015;174:317-323.
30
Savage JW, Moore TA, Arnold PM, Thakur N, Hsu WK, Patel AA, McCarthy K, Schroeder GD, Vaccaro AR, Dimar JR, Anderson PA. The Reliability and Validity of the Thoracolumbar Injury Classification System in Pediatric Spine Trauma. Spine (Phila Pa 1976) 2015;40:E1014-E1018.