Quality of Spine Surgery Information on Social Media: A DISCERN Analysis of TikTok Videos
Article information
Abstract
Objective
The use of social media applications to disseminate information has substantially risen in recent decades. Spine and back pain-related hashtags have garnered several billion views on TikTok. As such, these videos, which share experiences, offer entertainment, and educate users about spinal surgery, have become increasingly influential. Herein, we assess the quality of spine surgery content TikTok from providers and patients.
Methods
Fifty hashtags encompassing spine surgery (“#spinalfusion,” “#scoliosissurgery,” and “#spinaldecompression”) were searched using TikTok’s algorithm and included. Two independent reviewers rated the quality of each video via the DISCERN questionnaire. Video metadata (likes, shares, comments, views, length) were all collected; type of content creator (musculoskeletal, layperson) and content category (educational, patient experience, entertainment) were determined.
Results
The overall DISCERN score was, on average, 24.4. #Spinalfusion videos demonstrated greater engagement, higher average likes (p = 0.02), and more comments (p < 0.001) compared to #spinaldecompression and #scoliosissurgery. #Spinaldecompression had the highest DISCERN score (p < 0.001), likely explained by the higher percentage of videos that were educational (p < 0.001) and created by musculoskeletal (MSK) professionals (p < 0.001). Compared to laypersons, MSK professionals had significantly higher quality videos (p < 0.001). Similarly, the educational category demonstrated higher quality videos (p < 0.001). Video interaction trended lower with MSK videos and educational videos had the lowest interaction of the content categories (likes: p = 0.023, comments: p = 0.005).
Conclusion
The quality of spine surgery videos on TikTok is low. As the influence of the new social media landscape governs how the average person consumes information, MSK providers should participate in disseminating high-quality content.
INTRODUCTION
The advent of video-hosting social media platforms has resulted in a proliferation of medical information across the general public with studies reporting up to 96% of patients seek medical information on social media networks [1]. Tik Tok has amassed over 800 million users, establishing a wide array of virtual communities where individuals are free to disseminate information and has been considered the next social media frontier for medicine [2-7]. Thus, TikTok has become an increasingly influential platform for patients looking to broaden their understanding of spine pathologies, surgical treatment modalities, and postoperative experiences resulting in spine and back pain-related hashtags garnering billions of views across the platform. However, given the platform’s open access and unrestricted nature, content creators can release medical information irrespective of professional credibility. While enhanced accessibility offers wide dissemination of information, some spine-related content may be inaccurate resulting in potential harm in the form of false expectations and poor recovery advice. The large audience on the platform does however offer providers the unique opportunity to accurately inform and educate patients who seek spine-related content on TikTok. The value of accurate medical information on social media has been studied. Oser et al. [8] showed that type 1 diabetic patients actively utilizing social media for health information actually had lower levels of hemoglobin A1c. As such, high-quality open access information on social media has the power to truly impact the lives of patients. Therefore, the purpose of this study is to analyze the quality of spine surgery related information on TikTok and to better understand the characteristics of higher quality content. To achieve this, here we employed the DISCERN instrument to rate the quality of 150 videos on spine surgery related content from providers and patients on TikTok.
MATERIALS AND METHODS
1. Video Selection and Data Collection
Using TikTok’s search algorithm, we selected a sample of 50 videos for each hashtag utilized, including “#spinalfusion,” “#scoliosissurgery,” and “#spinaldecompression.” These hashtags were chosen as they provided a representative sample of spine surgery [9-17]. The 3 hashtags additionally had the most views of spine surgery hashtags and thus provide a good sample of content available on TikTok. Videos were excluded if they had less than 1,000 views, were duplicates, non-English language, and if they lacked audio or visual content. Metadata for each post was recorded, including content creator, views, likes, comments, shares, publication date, and video length. Furthermore, descriptive information such as the content category (educational, patient experience, entertainment) and creator type (layperson, musculoskeletal [MSK] provider) were recorded. Two independent reviewers analyzed video content and descriptions to categorize descriptive information and scored the quality of the consumer health information in each video using the previously validated DISCERN tool [18]. With the increasing focus on evidence based medicine, the DISCERN instrument (Supplementary Table 1) was developed as an objective method to measure the quality of medical and treatment information provided in a given source. As an instrument, the DISCERN tool has been extensively validated, tested for reproducibility, and utilized in the literature to analyze quality of information on various platforms. The tool includes 15 questions to evaluate the clarity, reliability, and biases. Each question is graded on a 5-point Likert scale with higher cumulative scores indicating higher source quality. The cumulative DISCERN score can be categorized into high quality (score 63–75), good quality (score 51–62), fair quality (39–50), poor quality (score 27–38), and very poor quality (score 15–26) [19]. Reviewer DISCERN scores were averaged between the 2 reviewers. Any discrepancies in categorical information or DISCERN score (difference in score > 15%) between the 2 reviewers were independently reviewed by a third observer.
2. Statistical Analysis
Shapiro-Wilk normality test was performed for all continuous variables to determine distribution. Analysis of variance (ANOVA) was used to compare the averages in normally distributed groups. For nonnormally distributed groups, a Kruskal Wallis test was utilized. For those variables significant on the ANOVA test, an ad-hoc Pearson t-test was performed. A chi-square or Fisher exact test was used to compare categorical variables as indicated. Multivariable linear regression was performed for those significant factors with DISCERN score as the dependent variable and different independent predictor variables. A multivariate linear regression was also performed with total views as the dependent variable. Statistical significance was considered with a p-value < 0.05. All statistical tests were done using R ver. 4.2.1 (R foundation for statistical computing. Vienna, Austria).
RESULTS
Of the 3 hashtags utilized, #SpinalFusion videos were the most popular (88.7 million views); #SpinalDecompression and #ScoliosisSurgery each had 8.8 million views. The average DISCERN score amongst the 150 videos analyzed was 24.4, ranging from 15 to 46.5. Two videos (1.3%) fell within the “fair quality” bucket, 35 (23.3%) fell within the poor-quality bucket, and a majority, 113 (75.3%) fell within the very poor-quality bucket.
#SpinalFusion videos had the most likes (p = 0.02) and comments (p < 0.001) (Table 1). In addition, #SpinalDecompression videos had the highest DISCERN scores (p < 0.001), which mostly were educational videos (p < 0.001) created by MSK creators (p < 0.001); in contrast, #SpinalFusion and #ScoliosisSurgery videos were created mainly by laypersons (p < 0.001) and focused on the patient experience (p < 0.001). Post hoc analysis of each question of the DISCERN score comparing the 3 hashtags showed #SpinalDecompression videos achieved significantly higher scores for aims, treatment options, and treatment description than #SpinalFusion and #ScoliosisSurgery (Supplementary Table 2).
Video metadata, quality, and type by TikTok hashtag
Of the 150 videos analyzed, fewer were created by MSK professionals than laypersons (38 vs. 112) (Table 2). On average, these videos tended to receive fewer views, likes, and comments, but this did not achieve statistical significance. However, the DISCERN score (p < 0.001) and video length (p = 0.004) for MSK-created videos were significantly higher than those created by laypersons (Table 2). MSK-created videos had significantly higher DISCERN scores for all questions except questions 5, 11, and 13, in which there was no statistical significance (Supplementary Table 3).
Video popularity and quality by content creator type
Patient experience videos were the most common, followed by educational and entertainment videos (Table 3). These patient experience videos received significantly more likes (p = 0.023), and comments (p = 0.005) compared to educational and entertainment videos; however, the less popular educational videos were significantly higher in quality (p < 0.001) and video length (p = 0.025).
Video popularity and quality by content category
In the multivariate analysis with DISCERN score as the dependent variable, total views, video length, creator type, and content category were found to be significant independent predictors of DISCERN scores. Videos with a higher number of views were more likely to have lower DISCERN scores (p = 0.011) (Table 4). Longer videos (p < 0.001) were a significant independent predictor of high DISCERN scores. With entertainment videos as the reference, educational videos (p = 0.001) and patient experience videos (p = 0.004) were both independent predictors of higher quality videos, with education videos having a larger B coefficient than patient experience videos (4.68 vs. 3.84). Similarly, compared to layperson videos, MSK-created videos were independent predictors of higher DISCERN scores (p = 0.013).
Independent predictors of TikTok DISCERN scores
In the multivariate analysis with views as the dependent variable, video length, creator type, and DISCERN score were found to be a significant independent predictor of views. Longer videos (p < 0.001) were a significant independent predictor of more views (Table 5). Higher DISCERN scores (p = 0.006) were a negative independent predictor of views. With entertainment videos as the reference, educational and patient experience videos had no significant impact on the number of views. However, compared to layperson videos, MSK-created videos were a negative independent predictor of views (p = 0.046).
Independent predictors of views
DISCUSSION
Medical information on social media is unique from other sources in that it can be rapidly shared, reaching a wide audience and this has led to a growing proportion of patients utilizing platforms such as TikTok to seek spine surgery information. With identification hashtags utilized on TikTok, the platform enables patients to view multiple related posts in succession [2]. In this analysis, the top 50 videos of 3 spine surgery related hashtags were reviewed for quality, metadata, and popularity. As the first study to explore the quality of information on spine surgery on TikTok, we show here that the overall quality is very poor.
Increasing the quality and quantity of high-quality spine surgery information on social media platforms like TikTok requires understanding the current state of information. The overall quality of videos as measured by the DISCERN instrument was 24.4 which is rated “very poor quality” in the previously established DISCERN scale [19]. Interestingly, another analysis of diabetes related information on TikTok found averages DISCERN scores between 40 and 50, much higher quality than the information on spine surgery we have found [20]. Other studies on quality of information on TikTok prostate cancer [21], acne [22], and chronic obstructive pulmonary disease [23] all have found low to moderate quality of information. The quality of information likely varies greatly by the specialty considered and within the sphere of spine surgery, the overall quality was very poor. Patients and practitioners alike should be therefore wary of spine surgery information available on the TikTok platform.
Of the 3 hashtags utilized here, #SpinalDecompression had the highest average DISCERN score. However, these videos were significantly less interacted with (fewer likes and comments). A majority of the videos among the #SpinalDecompression group were educational and created by MSK professionals. #SpinalFusion and #ScoliosisSurgery were on average of worse quality though they experienced greater popularity and interactions. These were mostly patient experience videos created by laypeople. The differences in demographics between creators likely mediates the differences in quality. Importantly though, even though #SpinalDecompression had the highest average quality score, the average of 27.7 would however be considered a poor-quality video. In the subgroup analysis performed, while MSK professionals create statistically significantly higher quality videos as rated by the DISCERN questionnaire, they also tended to be less popular than those created by laypersons. Educational videos similarly were of the highest quality but were significantly less popular. This finding may be explained by the significantly longer video length of MSK and educational videos. Although TikTok has recently increased the maximum video length to 10 minutes [24], shorter videos remain more likely to be seen by most people. Interestingly however, in our multivariate analysis, longer videos were independent predictors of more views. It is possible that MSK content creators should consider condensing or splitting up their videos to increase the likelihood that this high-quality information will be viewed by those in search of better information; however, the impact on popularity is not fully clear. However, MSK or spine surgeon creators shouldn’t shy away from making long videos.
Many of the trends described in our univariate analysis held true in the multivariate analysis of independent predictors of higher quality information. We found that number of views was an independent predictor of worse quality; the videos reaching the most people were the worse quality ones. Similarly, lower DISCERN scores were independent predictors of more views. It is also noteworthy that MSK videos, which were of higher quality, were independent predictors of fewer views as well. This may pose an inherent flaw with social media platforms such as TikTok as those videos with poor or even misinformation can certainly garner popularity and spread low-quality content. Video length was an independent predictor of better quality. Those trying to create quality, educational content likely require more time to do so (i.e., provide source citations - DISCERN question 7 or provide multiple perspective/options - DISCERN question 10, 11, 14). MSK-created videos that were independent predictors of higher quality, as were educational videos. It makes sense that videos created by professionals in the field would be of higher quality and more educational focused. However, of the 150 videos, only 38 (25.3%) were created by MSK professionals. Many of the top videos given from the TikTok algorithm were created by laypeople, likely a result of fewer MSK creators. Even within the MSK subcategory, even fewer were spine surgeons. There exists a clear lack of high-quality information and this could be filed with MSK professionals and spine surgeons who wish to grow their platform educating those around spinal pathologies. Given the low-quality information that currently exists on TikTok regarding and the growing use of the platform for medical education, this might serve as an important means to improve patient education and limit misinformation in the field. Unfortunately, our results do not point to specific things MSK professionals can do to garner more views, but we do support the notion that their activity on social media will be necessary to improve health information on the application. We do understand having experts create or even edit videos to ensure higher quality content is somewhat unrealistic, and so our primary takeaway is highlighting the prevalence of misinformation on social media platforms like TikTok and therefore the caution patients and providers alike should take when watching such content.
There are several limitations inherent to analyzing videos on TikTok. First, unlike some social media platforms, TikTok does not offer a validated method to establish content creator credentials; therefore, it is challenging to identify creator credibility accurately. We utilized 2 independent observers for even this categorical information in hopes of increasing the reliability. Second, we acknowledge a limitation in generalizability with the use of only 3 hashtags and the small sample size relative to all the possible videos on the platform. Additionally, we are unable to predict changes in TikTok’s proprietary search algorithm which may be different between different users. However, we utilized a new account with no prior search history to help assuage this concern. We also sought to include 3 hashtags that include a broad range of spinal pathologies and surgeries. Therefore, we believe that this analysis not only provides a good sample of TikTok videos within spine surgery, but our methods have also minimized the possible generalizability concerns. Additionally, while we have shown that the quality of information on TikTok is poor, the preset study was not designed to understand the impact of this on clinical care.
CONCLUSION
This sample of 3 spine surgery hashtags (“#spinalfusion,” “#scoliosissurgery,” and “#spinaldecompression”) have collectively garnered over 100 million views representing an active and growing body of spine surgery related information on TikTok. As rated by the DISCERN score, these videos were of very poor quality. Spine care providers should be aware that patients are utilizing TikTok as a source of medical information. Patients using TikTok to obtain spine-related information should be aware of the varied quality of videos and seek to verify all information by consulting with their healthcare providers. Future studies analyzing social media’s impact on patient treatment, outcomes, and clinical care would be of great interest.
Supplementary Material
Supplementary Tables 1-3 can be found via https://doi.org/10.14245/ns.2346700.350.
DISCERN instrument
DISCERN score for each hashtag by question on the DISCERN instrument
DISCERN score for each content creator by question on the DISCERN instrument
Notes
Conflict of Interest
Sheeraz Qureshi: Tissue Differentiation, HS2 LLC, Globus Medical, Stryker K2M, Simplify Medical Sravisht Iyer: Healthgrades, Globus Medical, Eliquence, Innovasis. All other authors have no conflicts of interest.
Funding/Support
This study received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Author Contribution
Conceptualization: TS, KA, IA, OT, AH, AL, SI, SQ; Data curation: KA, IA, OT, SQ; Formal analysis: TS, RM, OM; Methodology: TS, KA, IA, OT, AH, AL, AP, PS, RM, OM, ES, JD, SI, SQ; Project administration: TS, KA, IA, OT, AH, AL, AP, PS, RM, OM, ES, JD, SI, SQ; Visualization: ES, SI, SQ; Writing - original draft: TS, KA, IA, OT, AH, AL, AP, PS, RM, OM, SQ; Writing - review & editing: TS, KA, IA, OT, AH, AL, AP, PS, OM, ES, JD, SI, SQ.