Development of a Novel Retrospective Instrument to Gauge the General Well-Being Index and Lumbar-Specific Disability Index Post-intervention in Lumbar Degenerative Disease

Article information

Asian Spine J. 2023;17(3):538-548

Publication date (electronic) : 2023 February 6

doi : https://doi.org/10.31616/asj.2022.0293

Song Ying Chong¹, Mahmoud Danaee², Dharmendra Ganesan³

¹Faculty of Medicine, University Malaya, Kuala Lumpur, Malaysia

²Social and Preventive Medicine, Faculty of Medicine, University Malaya, Kuala Lumpur, Malaysia

³Neurosurgical Unit, Department of Surgery, Faculty of Medicine, University Malaya, Kuala Lumpur, Malaysia

Corresponding author: Dharmendra Ganesan, Neurosurgical Unit, Department of Surgery, Faculty of Medicine, University Malaya, Kuala Lumpur, Malaysia, Tel: +60-123457831, Fax: +60-37958636, E-mail: dharmendra@um.edu.my

Received 2022 August 17; Revised 2022 September 19; Accepted 2022 September 20.

Abstract

Introduction

Research concerning degenerative spinal disorders relies on the patient’s perspective of their quality of life (QOL) to assess treatment effects [1]. Multiple questionnaires were available in the field, which evaluated their spine-related disability index, including the Oswestry Disability Index (ODI) [2], Roland-Morris Questionnaire (RMQ) [3], and Quebec classification score [4], whereas other questionnaires were used to assess the patient’s QOL, such as Short Form-36 (SF-36) [5] and EuroQoL 5-Dimension questionnaires [6]. These are comprehensive questionnaires that serve as a reliable tool to gage patients’ specific and general functional state, at one point in time; considering the symptoms and their effects on daily activities and QOL. If this assessment is repeated at various timelines, it can be used to assess the response to a specific therapeutic modality.

Various outcome instruments are available, and the researcher determines the most appropriate instrument [7]. To assess the progress of the condition or response to treatment, commonly used questionnaires, including ODI [2], RMQ [3], and SF-36 [5], require prospectively recorded data at various points. As it is time-consuming, infrastructure and manpower would be required to conduct such research. In several parts of the developing world, this sort of facility and support is lacking; even if it were occasionally available in a busy clinical practice, such evaluation could not be performed on every patient.

Lumbar degenerative disease is a common disorder. A cohort of patients would benefit from surgical intervention. The surgical options for degenerative spinal disorders are varied, ranging from simple neural decompression using a microscope or endoscope to more intricate approaches involving fusion surgery. The surgical options that the surgeon advocates are based on their training, experience, and locality. To date, the literature on this topic is predominantly from developed countries. The surgical strategies employed in the developed world to manage lumbar degenerative disease may differ in developing or less developed countries. There is a paucity of literature on the techniques used and the outcomes regarding degenerative spinal disease in the developing world. Owing to its limited resources, these countries tend to practice a more basic neural decompression surgery rather than fusion surgery. However, owing to the limitation in obtaining prospective data in these countries due to various reasons, including the lack of funding, heavy clinic load, logistics, and manpower, these outcome data are not widely available. Therefore, these data sets do not get into mainstream literature but are possibly presented only in local meetings in the format of the surgeons’ experience. We devised a truncated questionnaire that permits a broader retrospective and comparative evaluation of general well-being and spine-specific disabilities for patients with a lumbar degenerative spinal disorder who have had an intervention to capture the outcome data from this cohort of patients. The outcome measures would permit the researcher to appraise the benefit of a particular intervention regarding functional domains.

The questionnaire has the following two large domains: general well-being and the lumbar-specific spinal disability index. Each domain has five questions, and the responses are objective with either one of three to four responses. The choice of questions in each domain is general (pain intensity, personal care, sleeping, social life engagement/recreation, and work) and specific (walking, sitting, change in position, bladder dysfunction, and lower limb numbness). These parameters were chosen to reflect a more holistic appraisal of the functional state despite being a short questionnaire. This assessment can be retrospectively performed either in person or through the telephone, which brings out the advantages of a greater response rate, relatively inexpensive but quick data collection, and the ability for interviewers to probe for incomplete or ambiguous answers [8]. Furthermore, this method is relevant in the current pandemic situation. We are presenting our assessment tool and the validation process.

Content validation certifies whether an instrument measures what is intended and verifies whether the items properly reflect the content domain of interest and whether the scale dimensions are consistent with each proposed item and provide clarity to respondents [9,10]. Content validity aspects include the relevance of the item and its clarity, which are determined by an evaluation of the instrument items by a group of experts with previous experience or currently recognized in the field of study, such as practicing surgeons, physicians, or experts [11]. This study aims to validate the content of a developed instrument to assess the general well-being and QOL of individuals with lumbar degenerative diseases who underwent lumbar spine surgeries (intervention).

Materials and Methods

The development and validation of an instrument encompass the following five distinct phases: planning, construction, quantitative analysis, validation, and a pilot study among patients [11]. The instrument planning and construction phases comprised an extensive literature review and expert opinions. Content validation and the pilot study are described in the present study. This project was approved according to protocol number 2020819-8994 of the Medical Research Ethics Committee of University of Malaya Medical Centre, Kuala Lumpur. Informed consent was waived owing to the retrospective nature of the study.

Content validation (phase I) was performed in two rounds. In the first round, the instrument was initially subjected to content validation by a committee of 20 medical experts with interest in the field of degenerative spinal disease. The content validation questionnaire was developed in English and was evaluated for the relevance and clarity of each item developed. The second round of content validation was repeated among 15 experts after the initial responses and comments were considered and relevant changes made to fine-tune the instruments. Once the content validations by the experts were concluded, a pilot study was conducted to evaluate the construct validity. A flow chart in the development of a new questionnaire is demonstrated in Fig. 1.

Fig. 1

Flow chart of development of new questionnaire.

Results

1. Phase I

1) The first round of content validation

In the first round of content validation, the domains were considered satisfactory (CVI between 0.70 and 1.00) for evaluating the general disability and the region-specific domains. As shown in Table 1, all items in the General Characteristics Domain obtained satisfactory scores; therefore, all the items in this domain were maintained in the second version of the instrument. Item 3, which assessed sleeping although achieved acceptable scores in relevancy (CVI=0.75, kappa=0.746) and clarity (CVI=0.70, kappa=0.746), was modified to ascertain whether the sleeping problems were due to spinal problems instead of other issues as pointed out by one of the experts. Moreover, the items that assessed pain intensity, social life engagement, and recreation had their wordings changed for improved understanding; for example, in the pain intensity section, the options of “improved, maintained, and worsened” were changed to “reduced, persistent, and worsened.” Items 6–10 presented the common scores of the lumbar-specific items. All items achieved a CVI of >0.85, except for relevancies in item 8, which assessed the change in positions of patients (CVI=0.75), and clarity in item 9 patient’s bladder dysfunction (CVI=0.75). To keep the questions more focused and maintain clarity to obtain a consistent understanding of the question either by reading or listening to the question by the respondent, we added the phrase “After the spinal intervention, …” to each of the questions. Moreover, wordings were improved while the experts highlighted some grammatical errors to keep the questions all in the present tense.

Table 1

The first phase of content validation of the general disability domain and lumbar specific disability domain

2) The second round of content validation

The content validation of the general disability and lumbar-specific domains in the second round of the content validation process is presented in Table 2. The second version of the instrument was resubmitted to a similar expert committee for re-evaluation of all items. Fifteen experts responded and participated in the second phase of content validation. All items achieved a CVI of >0.85 for relevancy and clarity. Thus, all items were successfully validated after appropriate corrections were made before the second validation phase.

Table 2

The second phase of content validation of the general disability domain and lumbar specific disability domain

2. Parallel analysis

In this section, the PA results are presented for the adapted instrument, including 10 items. The PA indicated that only the eigenvalue of the first extracted factor was more than the eigenvalue of simulated data (λ=1.646), suggesting that a single factor can be extracted from the data; therefore, in the next step for EFA, the number of extracted components is considered as one factor (Fig. 2).

Fig. 2

Scree plot and parallel analysis for items related to knowledge.

3. Exploratory factor analysis of impact scale

EFA using varimax rotation was applied to determine the factor structure among 10 items related to the general well-being scale. The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.863, which was above the suggested value of 0.6, and Bartlett’s test of sphericity was significant (χ²[45]=420.581, p<0.05).

In the current study, all initial communalities were above the threshold. The results of EFA on all 10 items were forced into one component based on the PA results. The results showed that this component explained 46.6% of the variance. According to these results, as shown in Table 3, all items showed a loading factor of above 0.5, except items 9 and 10, which had a loading factor of <0.5.

Table 3

Factor loadings based principal component analysis with varimax rotation for 10 items related to the general well-being scale

To investigate the effect of time on the outcomes of construct validity, the sample was divided into four categories, and EFA was performed on each subset of data because the current study is based on a retrospective study that took place from 2008 to 2020 (Table 4). The results revealed that, except for Q9 and Q10, which had low-loading factors in the pooled sample, the remainder of the items had acceptable loading factors across different subgroups, indicating that the passage of time did not affect the EFA results.

Table 4

Exploratory factor analysis based on time (loading factors)

Discussion

Lumbar degenerative spinal disease is a “wear and tear” condition that affects the more mobile parts of the spine, mainly the cervical and lumbar regions [14]. Most patients manifest symptoms of pain or numbness as their primary problem. The threshold to offer surgical intervention also hinges on the severity of the symptoms leading to functional disability in executing the activities of daily living and the QOL in terms of vocation and recreation. Therefore, the success of a particular intervention rendered to a patient with a degenerative spinal disease can be gauged by assessing the resolution or improvement of the presenting symptoms [15], wherein a return to the normal or usual QOL for a given age and medical condition is the ideal outcome of treatment [16]. This provides a good indicator of the functional benefit that the patient had derived from a particular intervention/treatment within his environment. Therefore, in cases of degenerative lumbar spinal diseases, symptomatology and functional disability play a key role in making management decisions and assessing the outcomes of these patients before and after the intervention.

Wilson and Cleary [17] proposed a taxonomy for different health outcome measures, which divides these outcomes into the following five levels: biological and physiological factors, symptoms, functioning, general health perceptions, and overall QOL. Consequently, various forms of lumbar spine-specific and general well-being questionnaires have been used to this end [2–4]. However, the key features of these questionnaires are that they are conducted in a prospective manner, indicating the symptoms and functional disability at that one point in time. Some of these lengthy questionnaires have multiple stems for one particular symptom that is being assessed, such as the ODI [2]; consequently, it can extract finer details of a symptom or activity. Although it is a more detailed assessment tool for acquiring these data in real-time from the participants, it needs patients’ cooperation and compliance. These questionnaires have undergone vigorous validation and are a standard means of furnishing outcome data for degenerative spinal disorders in several works of literature.

Given the busy service and lack of research assistants, prospective functional data are not routinely collected for most degenerative spinal disease cases that sought treatment in our facility. Based on the clinical follow-up, it is apparent that most of the patients who have undergone surgical intervention seem to have improved in functional ability. We wanted to study our surgical outcomes more scientifically; however, we realized that we lacked the prospective data. This initiated us to devise an assessment tool to perform a retrospective comparative assessment in a more standardized manner to allow a more systematic assessment of the outcome. The retrospective assessment tool will allow a comparison of the perceived degree of improvement in symptoms before and after the intervention. This comparison can be made at the various timelines of the intervention. However, the further the timeline is from the index surgery, there is the possibility that the patient may not be able to make a reliable comparison. Subsequently, we keep the questions more functional regarding their overall experience in a particular item of assessment rather than asking the patient to score the specific symptoms in a retrospective manner to minimize this sort of bias. Scoring symptoms retrospectively may be full of errors. However, evaluating the patient’s change experience in various functional items after an intervention in a broader sense rather than specific scoring would minimize the errors in terms of recall biasness. Without a retrospective instrument, we cannot measure the result more objectively in a standardized manner due to the lack of prospective functional data.

An obvious limitation of these existing questionnaires, including the ODI, RMQ, and SF-36, is that the data must be prospectively collected at two different time points, such as before and after the intervention, to compare the outcome after a particular intervention. This would be a more sensitive and specific way of presenting the outcome data; however, obtaining such data would require resources and patient commitment to answer these detailed questionnaires. To achieve a similar endpoint of outcome assessment after a particular intervention in a patient with a degenerative spinal disease, we have validated a single assessment tool using a comparative questionnaire to gage the difference the patient experienced after the intervention. The questionnaire examines a few common functional parameters to surrogate the patient’s overall functional ability. Since the answers expected are more of a reflection of the difference that the patient experiences after the intervention, the questions do not interrogate details of the symptom or activities like the ODI. Therefore, the options within each parameter are kept broader although distinct (Appendix 1). This assessment tool is more usable despite losing on the finer symptomatology and functional analysis. By indicating the usefulness of certain interventions in patients with a lumbar degenerative disease, particularly in resource-limited countries, this assessment tool can derive reasonable outcome assessments.

The questionnaire underwent content validation and reliability tests. Factor loadings are correlation coefficients between observed variables and latent common factors [18]. Factor loadings are part of the outcome of factor analysis, which serves as a data reduction method designed to explain the correlations between observed variables using a smaller number of factors. A factor loading that falls outside the interval bounded by (±cutoff value) is large and thus retained. Conversely, a factor loading that does not meet the criterion indicates that the corresponding observed variable should not load on the corresponding common factor.

Pain intensity, personal care, sleeping, social life engagement/recreation, and work were the general well-being domains that we focused on. The first question was about pain intensity. To encompass the overall pain and not specific to low back pain and lower limb pain, we kept this question broad. We assessed the overall pain syndrome related to the degenerative spinal disorder and hoped that the secondary pain-related issues will resolve or diminish by tackling the primary pain source. However, there were limitations to this question, as addressed by the experts, wherein patients would occasionally confuse non-spine-related pain simultaneously, resulting in differing expected results. However, this question has a factor loading of 0.776, which makes it a strong indicator that this question is highly important to assess the patient’s general well-being outcome. The second question included in the questionnaire was about personal care and activity of daily living (ADL). Options with concepts of being fully ADL independent, partially dependent, and total dependent were implemented in this section. Like the first question, we recognize that there may be other factors affecting ADL independence apart from the spine issue. This question has a factor loading of 0.700, which makes it a relatively strong contributor to a patient’s QOL.

As the following functions had specific relation to the lumbar spine, walking, sitting, changing position, bladder dysfunction, and lower limb numbness were the lumbar spine-specific domains that we focused on. For example, in the first question, which focuses on walking, we emphasized any presence of back or leg pain and if it causes any restriction in walking as a result of the back pain. This question has an acceptable factor loading of 0.796. The domains of bladder dysfunction (0.396) and lower limb numbness (0.390) were the only two items that had a factor loading of <0.400. This could be explained as the incidence of bladder dysfunction due to the degenerative spine disease was relatively uncommon even before surgery in our pool of patients (14.4%), thereby making this question irrelevant in most of the patients in the pool. Meanwhile, in the domain of lower limb numbness, patients may perceive numbness in different ways; while some patients felt the residual numbness was acceptable and negligible, some still felt that the numbness in the lower limbs affected their daily activities. Most studies considered loading factors of >0.5 and any lower than that should be excluded and removed from the questionnaire. However, according to [18–20], the cutoff value is arbitrarily selected depending on the field of study, and (±0.4) seems to be preferred by several researchers. Therefore, as the values of questions 9 and 10 were close to the cutoff point of 0.4, it is not excluded in the current study. However, to determine their value and decide on whether these two items should be excluded in the future, further evaluation should be made in future studies.

The intervention and assessment dates are recorded in the questionnaire to determine the timeline. The same instrument can be used to record outcome measures at various timelines. Therefore, analyzing the benefit of that index intervention at various time points is important. The confounding factors that would affect the outcome apart from the intervention would have a similar effect on the participant’s assessment of symptomatology in both the prospective and retrospective questionnaire should be taken into account.

To obtain an overall impression of the outcome after an intervention, the outcome for each item in the questionnaire has been denoted a score. We propose a summative score for the general well-being index and lumbar-specific disability index, respectively. A scale is used, wherein the general well-being index can be scored from 5 to 16, and the lumbar-specific disability index, which can be scored from 5 to 15. These scores are summated and subsequently grouped randomly into four categories to indicate the overall outcome (excellent, good, moderate, and poor) (Appendix 2). Once in clinical use, further clinical data will be employed to evaluate the discriminative and predictive validity as well as optimize the cutoff point for the scale.

This study had some limitations inherent to a retrospective questionnaire assessing functional outcomes. One of the key elements is the ability to remember the severity of the symptomatology at presentation. The risk of this biasness has been alleviated by focusing on the experiential change as perceived by the participant of the symptoms in functional terms rather than grading the severity of the symptom at presentation in a retrospective manner. The analysis of the effect of time on the outcomes of construct validity indicated that the passage of time did not affect the results of the EFA (Table 4). The formulation of a generic retrospective clinical symptomatology questionnaire for a lumbar degenerative disease has to encompass most of the common symptoms of that condition. However, not all the participants will suffer from all those symptoms. This is one of the limitations of a questionnaire-based assessment. Nevertheless, in this novel retrospective instrument, the absence of the symptom that is being assessed before and after the intervention would not have a negative impact on the outcome. However, if the patient develops a new symptom that was not present before the intervention, this will result in a worse outcome score. Another limitation inherent to a questionnaire-based functional assessment is the inability to tease other co-existing contributory medical conditions, such as knee osteoarthritis and peripheral neuropathy, which may also have relevance to the symptoms that are being assessed, which may impact the outcome assessment. However, this limitation exists in both prospective and retrospective disease-specific outcome assessments performed using a questionnaire [2,4]. However, further research that considers the premorbid history and medical characteristics, as well as their temporal relationship as potential confounders in determining the lumbar-specific disability outcomes, can be conducted. This is a novel instrument to permit a more standardized means of measuring outcomes for the cohort of patients with a lumbar degenerative disease who do not have prospective clinical data. More developing countries would be able to describe the outcomes in reference to the time of intervention of various sorts of treatments that were performed for patients with a lumbar degenerative disease using such instrument.

Conclusions

The content validation of the general well-being index post-intervention for spinal disorders and post-intervention lumbar-specific spinal disability index shows that it is a valid and reliable instrument to provide an impression of the outcomes following an intervention in a patient with a degenerative lumbar spinal disease. The functional parameters that were gauged were relevant to the disease condition. This tool will afford a standardized means of performing the retrospective comparative assessment of the effectiveness of a particular intervention. This is a valuable tool to describe outcome measures in a cohort of patients who do not have prospective pre-intervention data.

Article information Continued