Agreement of Physical Activity Measured Using Self-Reporting Questionnaires with Those Using Actigraph Devices, Focusing on the Correlation with Psychological State
Article information
Abstract
Purpose: This study aimed to evaluate the correlation and agreement of physical activity (PA) between data obtained from wearable Actigraph devices and self-reporting questionnaires, and to investigate the relationship between psychological state (depression, anxiety, and fatigue) and PA. Methods: A descriptive study was conducted using physical measurements and surveys. PA was measured through both the International Physical Activity Questionnaire (IPAQ) and the Actigraph GT3X+ device. The demographic characteristics of the subjects, as well as their depression, anxiety, and fatigue scores, were collected with structured questionnaires. The Spearman’s rank correlation coefficient and the Bland-Altman plot method were employed. Results: Data from 36 healthy adults were analyzed. The overall levels of PA measured using the IPAQ and the Actigraph were 1,891.69 MET min/week and 669.96 MET/day, respectively. Total levels of PA did not show a significant correlation between the two measurement methodologies. However, the moderate-intensity PA resulting from the IPAQ scores showed a significant positive correlation with the light-intensity PA recorded by the Actigraph. The Bland-Altman plot analysis demonstrated that the levels of PA as measured by the two different methods did not match. In addition, PA measured using the Actigraph showed a significant negative correlation with depression and anxiety whereas PA measured using the IPAQ showed a significant positive correlation with fatigue. Conclusion: The conclusion of this study is that the data obtained from the subjective self-reporting questionnaire and the wearable Actigraph do not correlate or match in healthy adults. Future research should investigate the relationship between depression and PA intensity through the Actigraph, or other wearable devices equipped with smartphone apps.
INTRODUCTION
Physical activity (PA) has a significant impact on health. Inadequate PA negatively affects health and is associated with various diseases [1]. Various studies on nursing [2,3] emphasize on the importance of accurate measurement of PA to determine the relationship between PA and health or disease. Most previous nursing studies on physical activities employed self-report questionnaires to measure physical activity [2,3].
Self-reporting using a questionnaire measurement method has several advantages, such as convenient and easy acquisition of data on the type or period of specific PA and low cost [4]. However, this raises the question of accuracy given the variability in the interpretation of the questions, inaccurate memory, and judgment errors [5]. Accurate PA measurement is essential, as inaccurate PA measurements can mislead the true association with other health indicators. Therefore, an objective PA measurement method is required to increase the reliability of the measurements.
Recently, several types of wearable devices have been developed to objectively measure daily PA. As technology advances, the accuracy and cost-effectiveness of the measurements have also increased. Actigraph, a type of accelerometer, is a commonly used non-invasive device that detects and records movement and measures PA [6,7]. It is also used in various studies that measure PA because the accelerometer is highly consistent with the calorimeter, the gold standard instrument for measuring PA [8,9]. The strength of the accelerometer is widely acknowledged, especially in studies with relatively small populations, because of its high accuracy and usage for the analysis of details and patterns of physical activities. However, it is not suitable for large-scale epidemiological investigations because it is difficult to record all types of physical activities due to wearing restrictions and burden on participants [10]. Therefore, the most suitable method of measuring PA should be selected based on the characteristics of the participants and the purpose of the study [11].
Measurement through a self-reporting questionnaire is often inevitable for studies on PA in clinical settings or in the community. To increase the reliability of self-reporting measurement, it is necessary to analyze its characteristics by comparing it with objective PA measurement methods and finding ways to complement it. Previous studies have found a significant correlation between physical activities measured using self-reporting questionnaires and those measured using Actigraphs [12]; however, other studies have demonstrated that the association is too weak [13] or inconsistent [14]. Therefore, further investigation is needed to compare these two methods in terms of their agreement and correlation.
Inadequate PA is associated with depression or anxiety [15]. Fatigue is also associated with PA [16] as PA can cause fatigue [17], although some studies have found the opposite result [18]. Several other studies have found that emotional problems or fatigue can influence the correlation between the results of self-report and objective measures [19-21]. Klumpp et al. [19] found that specific areas of the brain become activated and cognitive reappraisal is involved in subjective rating of sleep, which can differ from objectively measured results. They also suggested a link between emotional issues and sleep ratings. Measurement of PA may show a different relationship with depression, anxiety, or fatigue, depending on the method of measurement. Nevertheless, few studies have focused on the characteristics of each method that must be considered in the selection or complementary use. Therefore, a study is needed to measure PA using both a questionnaire and an Actigraph along with psychological states, including depression, anxiety, and fatigue, which analyzes the agreement and correlation of PA with the two measurement methods and their relationships with the psychological states.
1. Research objectives
In this study, we aimed to evaluate the correlation and agreement of PA in healthy young adults using wearable Actigraphs and self-report questionnaires, and to determine its association with depression, anxiety, and fatigue. The detailed objectives of this study are as follows:
To investigate the correlation between PA subjectively measured using self-reporting questionnaires and PA objectively measured using Actigraphs.
To assess the agreement between PA subjectively measured using self-reporting questionnaires and PA objectively measured using Actigraphs.
To explore the relationship of depression, anxiety, and fatigue with PA measured using both self-report questionnaires and Actigraphs.
METHODS
1. Study design
This cross-sectional study investigated the correlation and agreement between PA concurrently measured using self-reporting questionnaires and Actigraph (wearable devices), and their relationships with depression, anxiety, and fatigue.
2. Subjects
Healthy adults who did not have any restrictions on physical activities participated in this study. Those who had trouble performing PA due to physical issues, taking medications for cardiovascular diseases, or using upper limbs frequently when working were excluded. A total of 41 young adults from a college participated in the study. Of these, one person who decided to discontinue midway and four persons who did not use the Actigraph for more than three days were excluded from the pool. The final cohort consisted of 36 participants. Data were collected from June 30, 2019, to June 29, 2020.
3. Variable measurement
1) Demographics
The participants’ demographics included age, sex, height, weight, body mass index (BMI), education level, subjective financial stability level, smoking status, and alcohol consumption. Weight and height were measured using a digital scale (HE-53, CAS, Shanghai, China) wearing light clothes, and BMI was calculated as weight (kg) divided by the square of height (m2). Alcohol consumption was classified into four categories (“almost drink daily” for those who drink more than four times a week; “sometimes” for those who drink once in a month; “former drinker” for those who used to drink but not anymore; “never” for those who never drink), based on the Alcohol Use Disorder Identification Test.
2) Subjective measurement of PA using the International Physical Activity Questionnaire (IPAQ)
Subjective PA was measured using the IPAQ [22], which asked the participants about the time spent on vigorous and moderate physical activities, walking, and sedentary activities over the last seven days. The result was converted to the metabolic equivalent of task (MET) to illustrate the calories consumed. The formulas used to calculate PA according to intensity were as follows: light PA [3.3 (MET level)×time spent walking or mild physical activities (min.)×number of days], moderate PA [4.0 (MET level)×time spent on moderate physical activities (min.)×number of days], and vigorous PA [8.0 (MET level)×time spent on vigorous physical activities (min.)×number of days].
3) Objective measurement of PA using the Actigraph
Objective measurement of PA was done using an Actigraph accelerator (Actigraph GT3X+, Pensacola (Florida), USA) that can be worn on the participant’s wrist. This device measures PA on three axes to collect information on posture and activity and is commonly used for measuring PA because measurement reliability and validity are acknowledged [23,24]. Data were analyzed using ActiLife 6.13.4 software (ActiGraph LLC, Pensacola (Florida), USA), and the wearing time was calculated using Troiano’s algorithm [25]. The MET rate daily average calorie burn, and steps per minute, which represent total PA quantitatively, as well as light/moderate/vigorous physical activities, which denote the intensity of physical activities, were calculated. Physical activities in each intensity were determined based on the measurement unit (counts per minute [CPM]) in an Actigraph device with the application of the Freedson adult algorithm [26]. The range 0-99 CPM indicates no PA, 100-1,951 CPM indicates light PA, 1,952-5,724 CPM indicates moderate PA, and 5,725 CPM or higher indicates vigorous PA. The daily average PA in each intensity was calculated by multiplying 3.3 MET, 4.0 MET, 8.0 MET for light/moderate/vigorous PA, respectively.
4) Depression
The Center for Epidemiological Studies Depression Scale (CES-D) was used to evaluate depression [27]. We used the Korean version of the CES-D, which was edited and modified by Chon [28]. The CES-D, which is a 4-point scale from 0 to 3 points, has 20 items on symptoms or frequency of depression in the past week. The higher the total score, the more severe is the depression. In the study by Chon et al. [28], Cronbach’s alpha was .91, while it was .88 in this study.
5) Anxiety
The State Anxiety Inventory and Trait Anxiety Inventory developed by Spielberger [29] were used to measure anxiety, which included 20 items each on a scale of 1 to 4. The higher the total score, the more severe was the anxiety. Cronbach’s alpha was .89 in the study by Spielberg [30]. In this study, Cronbach’s alphas for state anxiety and trait anxiety were .93 and .90, respectively.
6) Fatigue
The fatigue severity scale developed by Krupp et al. [31] was used to assess fatigue. It had a total of nine items, where participants were asked to answer how fatigued they have been in the past week on a scale of 1 to 7 points. The score is calculated as the average score of nine items, and the higher the score, the more severe the fatigue. In a previous study [32], Cronbach’s alpha was .94, whereas in this study, it was .84.
4. Data collection
To recruit research participants, flyers were posted on the undergraduate and graduate bulletin boards on the website of a university. Healthy adults who voluntarily decided to participate in the study visited the research office. The consent to participate was obtained after the research assistants explained the purpose and procedure, precautions for participating in the study, and the use of personal information and data.
Participants were asked to wear an Actigraph and perform daily activities. They were informed to take off the device during a shower or bath but wear it again immediately and record the time of not wearing it and the reason thereof. The device was set to measure at 30 Hz at 1-minute intervals, and participants wore it on the wrist of their non-dominant hand for three days. ActiLife 6.13.4 software (ActiGraph LLC) was used to analyze the collected data. Participants who had worn the device for three days then returned to the research assistants’ office and were asked to complete surveys about their demographics, IPAQ, anxiety, depression, and fatigue.
5. Data analysis
SPSS Statistics ver. 25 for Windows (IBM Corp., Armonk, NY, USA) was used for the data analysis. Participants’ characteristics, depression, anxiety, and fatigue were represented as counts, percentages, and averages, respectively. Spearman’s rho correlation was used to analyze the association between the amount of PA using IPAQ and Actigraph, and the Bland–Altman plot was used to evaluate their agreement. Each measurement was transformed to a z-value for the Bland–Altman plot, as the units of measurement by the two methods were different. Spearman’s rho correlation was employed to analyze the association of PA, using the two methods, with anxiety, depression, and fatigue.
6. Ethical considerations
This study was approved by the Institutional Review Board of I University (IRB approval number: 190515-1A). Adults who voluntarily decided to participate in the study were included in the study. Before participating in the study, they were also informed about potential discomfort while wearing the Actigraph. Participants were allowed to drop out any time if they wished to stop participating in the study. To protect the subject’s personal information, the collected data were anonymized, encrypted, and stored in a computer locked in the laboratory of the principal researcher.
RESULTS
1. General characteristics of subjects
The mean age of the participants was 22.5 years, and 17 (47.2%) were female (Table 1). Of the 36 participants, 33 (91.7%) answered that their subjective financial stability level was moderate, but none chose above moderate. Only four (11.1%) participants were current active smokers, although most of them (30, 83.3%) drank regularly. Their mean depression score was 10.78, state and trait anxiety were 38.31 and 37.56, respectively, and fatigue was 3.52.
General Characteristics and Physical Activity of Participants (N=36)
2. Physical activity levels
Total PA that was obtained by the self-report IPAQ survey was 1,891.69 MET-min/week; the mean amounts of light, moderate, and vigorous PA was 737.24 MET-min/week, 183.33 MET-min/week, and 971.11 MET-min/week, respectively (Table 1). The result of PA obtained through the Actigraph objectively demonstrated that the MET rate was 1.31, the mean daily calorie burn was 664.13 kcal, and the number of steps per minute was 7.74. The mean amounts of daily light and moderate physical activities were 517.45 MET/day and 152.50 MET/day, respectively, while vigorous PA was low. The total amount of PA per day was 669.96 MET.
3. The association between the results of PA measured subjectively through the IPAQ and objectively through the Actigraph
The total PA from the IPAQ did not indicate any statistically significant association with the MET rate, daily calorie burn, and steps per minute in the Actigraph (Table 2). However, moderate PA from the IPAQ and light PA from the Actigraph showed a statistically significant positive association (p=.043).
Correlation of Physical Activity between Subjective Measurement and Objective Measurement (N=36)
4. The agreement between the results of PA measured subjectively through the IPAQ and objectively through the Actigraph
The agreement between PA obtained through the IPAQ and Actigraph was analyzed using the Bland–Altman plot. For moderate and light physical activities, agreement between pairs of measurements using both methods were not obtained. As the average of the two paired measurements increased, their difference increased in the negative direction, as shown in Figure 1.
Bland-Altman plot of physical activity on IPAQ and actigraph. (A) Bland-Altman plot of light physical activity on IPAQ and actigraph, (B) Bland-Altman plot of moderate physical activity on IPAQ and actigraph.
For total PA, the pattern of agreement between the total PA measured subjectively using the IPAQ and the average daily calorie burn measured objectively using the Actigraph illustrated that the larger the average of the pair measured through the two methods, the greater the difference between them, indicating an inconsistent pattern with increasing fluctuations (Figure 2). Similarly, the overall activity measured using the IPAQ demonstrated that the MET rate and the number of steps per minute measured using the Actigraph also had a similar pattern, indicating inconsistency.
Bland-Altman plot of total physical activity on IPAQ and on actigraph. (A) Bland-Altman plot of total physical activity on IPAQ and average kcals per day on actigraphy. (B) Bland-Altman plot of total physical activity on IPAQ and MET rate on actigraph. (C) Bland-Altman plot of total physical activity on IPAQ and step per minute on actigraph.
5. Association with demographics, depression, anxiety, and fatigue
The subjectively measured PA using the IPAQ revealed that moderate intensity of PA was significantly higher in males (p=.031) and those with low BMI (p=.033). Vigorous PA was also greater in male participants (p=.010), although it was not significantly associated with BMI (Table 3). The PA measured using the IPAQ had a significant positive correlation with fatigue (p=.025) but not with depression or anxiety.
Correlation of Demographic Characteristics, Depression, Anxiety, Fatigue and Physical Activity (N=36)
For the objectively measured PA using the Actigraph, the MET rate, and the daily calorie burn, which represents the total amount of PA, indicated a significant positive correlation with male sex (p=.019, p=.033) and high BMI (p=.010, p=.008), although PA of each intensity was not associated with sex or BMI. The correlation of the PA measured using the Actigraph with depression or anxiety was significant; high depression (p=.032, p=.023) and state anxiety (p=.013, p=.031) were significantly associated with a low MET rate and steps per minute. Moderate PA had significant negatively correlations with high depression (p=.006) and state anxiety (p=.030). However, PA measured through the Actigraph did not show any significant association with trait anxiety or fatigue.
DISCUSSION
We conducted this study to confirm the degree of agreement between the subjectively measured PA using the self-reporting questionnaire and the objectively measured PA using the Actigraph, as well as their relationship with emotional states. Furthermore, we wanted to suggest how to select the most appropriate measurement method when assessing PA and what should be considered in future studies when using each measurement method.
We found that moderate PA measured using the IPAQ and light PA measured using the Actigraph were significantly correlated. This means that there is a tendency to respond by recognizing the intensity of activity somewhat excessively when subjectively measuring PA, which has also been reported in previous studies [33]. In contrast, low and moderate PA did not demonstrate a significant association between the two methods. From the analysis of the Bland–Altman plot pattern, as the average of the two paired measurements increased, the difference between them increased in the negative direction in both light and moderate physical activities. In other words, there is no agreement between pairs of measurements, which is consistent with previous studies [14, 34]. Upon further analysis using a regression model according to the size of the disagreement between two paired values of moderate PA, the regression formula was significant and there was a proportional bias. This means that measurement of moderate-intensity PA may cause a difference between the subjective and objective methods. Taken together, moderate-intensity PA in the questionnaire could objectively be low-intensity PA, thus, caution is needed when measuring moderate PA using self-report questionnaires.
Even when vigorous PA was reported in the self-reporting questionnaire, there was almost no vigorous PA measured using the Actigraph. This might be due to an inaccurate perception of the vigorous and moderate intensity of PA, or the criteria for distinguishing the intensity of PA may not have exactly matched between the questionnaire and the Actigraph. This can be interpreted as the Actigraph not measuring high-intensity PA sensitively. Alternatively, the participants may have detached their Actigraphs during vigorous PA despite the instructions. Shiroma et al. [34] also found a discrepancy between the range of moderate and vigorous physical activities that Actigraph measured, suggesting that the Actigraph might be inadequate to measure the PA of individuals who engage mainly in vigorous activities. However, Domingos et al. [35] found that vigorous physical activities measured using the Actigraph were higher than those measured using the self-reported questionnaire and concluded that disagreement with vigorous PA might have a systematic bias and, in part, it was inconsistent with the results of this study. For Actigraph, detecting vigorous PA may differ depending on the model of the Actigraph; thus, when selecting an Actigraph device to measure PA of the subjects who engage in considerable high-intensity activity, the sensitivity of the device to vigorous PA should be checked.
There was no significant correlation between the two measurement methods in terms of the overall PA. The Bland–Altman plot demonstrated a pattern of disagreement in the form of increasing variation in the difference between the two values of PA as the average of the two values increased, which was similar to the results of previous studies [36, 37]. Makarewicz et al. [37] concluded that the results obtained through subjective and objective methods were not consistent, although the agreement between the two methods in participants with low PA was relatively high. Huang et al. [38], who found similar results, suggested that those who undertake relatively fewer physical activities displayed a significant correlation, and those who have a lot of physical activities tended to perceive and report their activities as more vigorous. Participants in this study also reported a higher amount of PA subjectively when compared with a similar age group in previous studies [39], which might have caused the disagreement. In addition, Siebeling et al. [40] reported the low reliability of a self-reporting questionnaire from highly active participants. This study also illustrated the higher reliability of those who were less active and had a higher agreement with the self-reporting questionnaire and Actigraph. This suggests that in subjects with degraded memory or recall and with low PA, using an Actigraph may be a more accurate measurement. Recently, various tools that measure PA have developed rapidly with information technology, such as wearable devices and smartphone applications. Conduct of studies comparing PA using these various methods have been suggested in the era of digital health care.
The correlation between PA and other indicators (demographics, depression, anxiety, and fatigue) showed different results depending on the method of measurement. PA measured using the self-report questionnaire and depression or anxiety were not correlated, although the total amount and light physical activities measured through the Actigraph showed negative correlations with depression and state anxiety. Previous studies have found increased levels of depression or anxiety in sedentary lifestyles [41,42]. It is significant that in this study, in PA measured by Actigraph, depression and anxiety showed similar results. Previous studies have also demonstrated that the results measured using the Actigraph indicated a significant correlation with other health-related indicators [13,43]. Lee et al. [43] pointed out that potential errors could be involved in the measurement of light and moderate PA using a self-reporting questionnaire, which could be a reason for the differences in their results and those in our study.
Although a recent study illustrated that depression was related more to vigorous PA than light PA [44], in this study, light PA had a significant correlation with depression, thereby showing different results. Considering the small sample size of this study, it is necessary to study the relationship between depression and PA intensity using Actigraphs in the future with a larger sample size. When measuring PA using an Actigraph, research that provides evidence for an appropriate number of subjects is also needed. In contrast, in this study, subjective fatigue was positively correlated with the total amount of PA using the IPAQ questionnaire, but the PA based on the Actigraph did not show a significant correlation with fatigue. A recent study demonstrated that PA and fatigue are not significantly related in subjects with low fatigue [45]. Considering that the fatigue level of the participants in this study was not high (not greater than 4 points), the possibility that fatigue and the IPAQ response content influenced each other may not be ruled out. Therefore, exploring the causality of PA and fatigue in a large-scale or experimental study in the future is strongly suggested.
Meanwhile, the amount of PA measured using the self-report questionnaire differed significantly by gender. By intensity, the amount of moderate PA was slightly higher in females, whereas the amount of vigorous PA was higher in males. This was consistent with the results of previous studies that found different types and purposes of PA preferred by each gender [46]. Gender has been reported to influence the measurement of PA [14]. From the additional analysis in this study, males had a significantly higher amount of PA, and the degree of agreement between the measurement methods was different in case of a large amount of PA. Considering these results, a future study to analyze the agreement between measurements according to gender is proposed, which may provide a detailed rationale for considering the characteristics of the participants in the selection of the most suitable measurement method.
This study has certain limitations. First, the small sample size restricted the paramettric tests in the correlation analysis, which limits the generalization of the results of this study. Although we conducted nonparametric tests, significant results were obtained and they can be used in future research. Second, only one type of Actigraph was used to measure PA in this study. As there are many types of Actigraphs and Actiwatch devices, a comparative study among devices can help in choosing the most suitable Actigraph for PA research. Regardless of the limitations, the value of this study lies in providing statistical evidence that could increase the accuracy of measurement in future research by comparing relationship and agreement using the Bland–Altman plot after collecting data from the same population using the two popular methods, self-report IPAQ and the Actigraph. We suggest selecting different methods depending on the psychological status or characteristics of the subjects. The results of this study can be used as a basis for selecting a method for measuring PA in evaluation or intervention research in the future.
CONCLUSION
Total levels of PA did not show a significant correlation between the two measurement methodologies, self-reporting questionnaires (subjective), and Actigraphs (objective) among healthy young adults. The Bland–Altman plot did not reveal any agreement either. However, moderate-intensity PA obtained through the IPAQ showed a statistically positive correlation with light intensity PA recorded by the Actigraph, and the intensity of physical activities obtained through the self-reporting questionnaire tended to be higher than that of the Actigraph. Moreover, the two indicators measured through the Actigraph (MET rate and steps per minute) decreased with high levels of depression and state anxiety; moderate PA was also statistically low with high levels of depression or state anxiety, whereas PA measured using the IPAQ showed a significant positive correlation with fatigue. Using the Actigraph is recommended to find associations between PA and other health indicators, such as depression or anxiety.
Based on the results of this study, future research could be suggested as follows: First, studies with Actigraph that provide statistical evidence for the appropriate number of participants are needed. Future research should investigate the relationship between depression and PA intensity through the Actigraph; additionally, future studies that measure and compare PA using various types of wearable devices or smartphone apps are also suggested.
CONFLICT OF INTEREST
The authors declare no conflict of interest.
AUTHORSHIPS
SMH & SKS contributed to the conception, design of the study; interpreted the result; drafted the manuscript; critically reviewed and revised the manuscript; SMH & JMO conducted the data collection, analyzed; SMH supervised the whole study process; All authors read and approved the final manuscript.