Exploring acupuncture outcomes in a college clinic

 Jackie Shaw, Penelope Bidgood and Nasrollah Saebi

European Journal of Oriental Medicine (EJOM) Volume 5(4), 2006: pp55-63

First Published 2007

 Download PDF of this article

 

The College of Integrated Chinese Medicine (CICM) in Reading, UK, has developed research awareness and capability through a descriptive outcome study exploring patient profile and perception of change after acupuncture treatment in the student clinic. Data was collected from 495 patients treated by students during 12 clinical programmes from 1996-2004. Musculo-skeletal (30.5%), psychological (18.6%), and general (14.8%) conditions were the most predominant complaints. Most (81.4%) patients had chronic conditions and nearly half had been unwell for more than five years. Patients’ initial health status was worse than those registering chronic disease or disability in the general population and comparable to hospital outpatients. 74.3% of patients presented two symptoms or more and 41% of patients wanted help with general factors such as energy levels and emotional state. The study showed a clear association between acupuncture treatment and positive health outcome as assessed by patients. The majority (82.2%) of responding patients reported a large or moderate change in their main complaint and 88% a large or moderate change in their general health. These results, from a wide range of predominantly chronic patients, indicate high levels of patient satisfaction.

Keywords

Acupuncture, patient profile, outcome study, health-status, SF36, association, effectiveness, musculo-skeletal, psychological, chronic.

Introduction

This ambitious project was initiated ten years ago at the College of Integrated Chinese Medicine (CICM). So why, as a predominantly teaching institution, get involved in research?  The initial motivation was to develop research awareness and capability in the college community. The project also aimed to contribute, through clinical research, to the emerging collection of acupuncture outcomes evidence (MacPherson and Fitter 1998; Chapman, Norton and Paterson 2001; Andrews and Booth 2004; Bovey, Horner, Shaw et al. 2005).

Research is not a value free activity but produces different kinds of knowledge depending on the beliefs of those asking the questions (Gray 2004:15-33). Lewith, Walach and Jonas (2002) suggest that Complementary and Alternative Medicine (CAM) research should build up a rigorous and relevant evidence ‘house’ through six types of knowledge that address the needs and perspectives of different audiences. The next question was how could the college, with limited resources, contribute to the acupuncture evidence profile?

Observational methodologies such as audit and clinical outcomes studies are appropriate when attempting to uncover associations between intervention and outcome. They can also be suitable research designs for practitioner-based research. They need relatively low levels of resourcing and can be carried out in reallife settings with minimal changes to usual practice. Involving practitioners actively in research as practitioner-researchers can be an important way of nurturing critical thinking which in turn contributes to individual professional development and to the vitality of debate within an evolving profession (Thomas and Fitter 2002).

Against this background CICM began a descriptive clinical audit and outcome study to explore patient profile and perception of change after acupuncture treatment in the college student clinic.  The purpose was to increase understanding of the benefits of acupuncture in a clinical setting. The college clinic was thought to offer the opportunity to gather information on a greater number of patients than would be possible in private practice. In addition piloting practice audit processes within the clinic provided the setting to develop methodology as well as to give students direct experience of collecting outcomes data.

This paper and a follow up paper (to be published in the next issue of EJOM) describe the main findings of the study. Data was collected from 495 patients treated during 12 clinical programmes from 1996 to 2004. All the clinical groups provided the same patient profile data and assessed the overall impact of treatment using the same measures. This data is presented in this paper.

Materials and Methods

This project followed an earlier outcomes study involving seven practitioners and 58 patients (MacPherson and Fitter 1998) which investigated the factors influencing outcomes after treatment with acupuncture. There were a number of adaptations, however, in an attempt to reflect more fully the acupuncture treatment process.

Study process

Patients filled in three different types of questionnaires at different stages of the study:

The Short Form 36-Item Health Status Survey (SF-36)

The anglicised version of the SF-36 was used as the baseline measure of health status at the commencement of treatment as it is validated with a wide range of populations and shown to be appropriate with patients with a relatively low level of ill health (Brazier, Harper, Jones et al. 1992). New patients at the clinic filled in the SF-36 alone before the first session.

Initial assessment form (before treatment 1) and treatment progress forms (before treatment 5 and 10)

Student-practitioners filled in an initial assessment form in consultation with the patient during diagnosis. As well as collecting data on patient profile simple subjective measures were used to assess health status. Patients attending clinics 1-6 (CICM cohort) and 7-12 (MYMOP cohort) piloted different repeated measures that were assessed again on the treatment progress forms at defined

points in treatment. (This data is presented in a follow-up paper.)

Final feedback form

The patient filled in the final feedback form alone after treatment 10 or the final treatment whichever was the sooner. This gathered information on the patients’ evaluation of treatment outcomes and benefits. Patients assessed the overall effect of acupuncture on their original main complaint and general health using a 9-point scale.

Practitioners and patient sample

Practitioners were college students practising under the supervision of clinical tutors as part of their clinical training programme. All new patients at the student clinical received a letter explaining the purpose of the study. A consent form was signed by those patients agreeing to take part. Patients were excluded who declined or did not complete a consent form as well as those who provided no data, no baseline questionnaire or attended for diagnosis only.

Data collection and analysis

Reception staff at the college handled recruitment of patients and co-ordination of questionnaire timing. They also kept records of patients who stopped treatment during the course of the study with reasons where possible. Most of the analysis was performed using the software package SAS 9.1 with graphs produced in Minitab 14 for Windows.

Results

Participation rate

Usable data was collected from 495 patients (70.7% of patients attending the clinic). Of the remaining 205 patients, 102 (49.7%) either declined to take part in the study or, in most cases, no permission form was signed. The other 103 were excluded from the study, 41(20%) as no data was collected from them, 8 (3.9%) as they received no treatment (i.e. attended for diagnosis only), and 54 (26.3%) as they did not complete an initial assessment form (i.e. no baseline data available). It should be noted that not all 495 patients responded to all questions.

Response rates and attendance patterns

Possible reasons for missing questionnaires are due to procedural problems (practitioners forgetting or running out of time, patients not returning final form or administration problems) or due to unexpected patterns of attendance. The final questionnaire was the most difficult to collect as it was often not known that patients were stopping treatment until they failed to rebook or cancelled from home.

Despite attempts to improve collection rates the percentage of final feedback form returns were lower for clinics 7-12 (29.3%) than 1-6 (60%). Other studies have found that patients using the MYMOP questionnaire are more likely to complete a final feedback form (e.g. Bovey, Horner et al. 2005) but in this study they were less likely to. In this case it may indicate some research fatigue.

There was some retrospective analysis of data and patient records to ascertain what happened to patients who did not provide final feedback data.

Of 495 patients included in the analysis:

214 did provide a final feedback form

• 32 stopped treatment before the end of the clinical
• 33 stopped at the end of the clinical
• 13 carried on treatment after the end of the clinical
• 16 do not know, as no data available for these

281 did not provide a final feedback form

• 60 stopped treatment before the end of the clinical
• 111 stopped at the end of the clinical
• 61 carried on treatment after the end of the clinical
• 49 do not know.

Tables 1 and 2 below compare the attendance patterns and reasons for stopping treatment before the end of the programme for all patients and those who did not supply a final feedback form.

 

	All patient	Patients with no final feedback form
	n=345	n=197
Mean no. of sessions	9.95	9.4
Mode no. of sessions	10	8
Min no. of sessions	2	2
Max no. of sessions	32	32
% having
2-4 sessions	15.1	20.3
5-9 sessions	34.5	33.5
10-14 sessions	31.6	30.0
15-19 sessions	14.2	11.1
more than 20 sessions	4.6	5.1

Table 1 - Comparison of attendance patterns

 

 

	All patient	Patients with no final feedback form
	(n=52)	(n=20)
Lack of time	12	4
Changed circumstances	10	7
Feeling better	7	1
No (further) benefit	6	0
Changing practitioner	5	0
Does not like needles	4	4
Financial reasons	3	1
Break to assess changes	3	1
Starting other treatment	1	1
Died	1	1

Table 2 - Reasons for stopping before the end of the programme

 

Patient profile

Profile data on age, gender, main complaint, its duration, and number of other complaints were collected from the 495 patients in the study. Figure 1 shows the distribution of age by gender for all 12 clinics.

 

Figure 1 – Distribution of age by gender

Although ages range from 13 to 83, the vast majority (89%) of patients were aged between 21 and 60. Over 70% were women; ages for these ranged from 13 to 79, with a mean age of 42.6 years. Men, whose ages ranged from 14 to 83, were, on average, slightly older with a mean age of 43.8 years.

Presenting conditions

The patients’ main presenting complaints at diagnosis were categorised using the International Classification of Primary Care (ICPC) (Lamberts, Wood and Hofman-Okkes 1993).

The most predominant disease categories in the cohort were musculo-skeletal (30.2%), psychological conditions (18.8%),  including stress, depression, insomnia, disturbed memory or concentration and substance abuse, and general conditions (14.9%), including complaints such as general pain, general weakness/tiredness, allergies, general viral and infectious diseases, and other general/unspecific symptoms.

Other complaints with 5-7% of the total include neurological, gynaecological, digestive, respiratory and skin. There are small groups of blood, eye, ear, circulatory, metabolic, urinary and social problems, which have been categorised under ‘Other’ in the following table. There is very little difference in proportions between the genders in presenting conditions, although a slightly higher percentage of men (35%) than women (28%) complain of musculo-skeletal problems and about twice as many women as men have digestive problems. 

Patients were also asked about how long they had suffered from their main complaint (duration), which was divided into six categories, namely, ‘more than 15 years’, ‘5 to 15 years’, ‘1 to 5 years’, ‘6 months to 1 year’, ‘1 to 6 months’ and ‘less than 1 month’. Only 90 (20.4%) of the 487 respondents to this question were in any of the last three categories so these have been amalgamated as ‘less than 1 year’ in Table 3, which shows the duration of the suffering from each of the main presenting conditions.

 

	Duration (years)
Main Presenting Condition	Chronic			Total Number   (% complaint)	Acute  (% complaint)	Total Number (% of total)
	>15	5-15	1-5		<1
Musculoskeletal	25	35	61	121 (81.8%)	27 (18.2%)	148 (30.5%)
Psycological	22	22	31	75 (83.3%)	15 (16.7%)	90 (18.6%)
General	15	12	27	54 (75%)	18 (25%)	72 (14.8%)
Neurological	16	8	3	27 (79.4%)	7 (20.6%)	34 (7.0%)
Respiratory	8	9	5	22 (78.6)	6 (21.4%)	28 (5.8%)
Digestive	5	14	4	23 (88.5%)	3 (11.5%)	26 (5.4%)
Skin	8	9	2	19 (82.6%)	4 (17.4%)	23 (4.7%)
Other	7	8	13	28 (77.8%)	8 (22.2%)	36 (7.4%)
All Number (%of Total)	117 (24.1%)	125 (25.8%)	153 (31.5%)	395 (81.4%)	90	485 (18.6%)

Table 3 – Duration of problem for each main complaint overall

Following Chapman et al (2001), a condition is deemed to be ‘chronic’ if the duration is for at least one year; otherwise the condition is termed ‘acute’. It is evident that for most patients their condition is chronic rather than acute (81.4%). About 50% of patients had had their condition for more than five years and nearly a quarter for over 15 years. Almost all neurological, gynaecological, respiratory, digestive and skin complaints were chronic. Almost one third of acute conditions were musculoskeletal and a further fifth psychological.

Patients were also asked to specify the other complaints which they wanted acupuncture to help. Almost three quarters of patients presented with more than one complaint at diagnosis. 40.8% had two and a further 23.7% three complaints; the maximum number recorded was eight conditions at diagnosis (by one patient). There was an overall mean of 2.25 complaints per patient. Although the mean number of complaints varies between clinics, the mode (most frequent) number of complaints is two for all the clinics. The largest main complaint group, musculo-skeletal, presented with one to four conditions, with a mean of 2.0.

What patients hope to gain from treatment

Patients had differing expectations from their treatments and can be categorised into ‘health seekers’, who are looking for wider health benefits from alternative and complementary healthcare, or ‘health improvers’ who are looking for the resolution of specific complaints. About two thirds of the 420 patients categorised in this way were health improvers. When asked what they hoped to gain from their treatment, 45.2% specified that they wanted to improve their main complaint and 13.8% to gain more control over their symptoms or pain. The remaining patients (41%) hoped for other benefits from treatment; 23.1% to improve their feeling of wellbeing, emotional state or stress levels; 7.4% to increase their energy; and the remaining 10.5% for other specified benefits.

Initial health status

The anglicised version of the SF-36 was used as the baseline measure of health status at the commencement of treatment. The SF-36 measures health status in eight categories: physical functioning, social functioning, role-limitation physical, rolelimitation emotional, bodily pain, mental health, vitality and general health. Following MacPherson and Fitter (1998), Table 4 below compares the average scores for patients in this study with those surveyed in the MacPherson and Fitter study and with UK population data (Brazier, Harper et al. 1992).

Those patients attending for acupuncture in this study had worse health (scored lower) than those patients who had been for a GP consultation in the last three weeks in the Brazier study. They also had worse health in every SF-36 category than patients (3698/8815 people) registering chronic disease or disability in a 1997 community survey of health status (Knight, Stewart-Brown and Fletcher 2001). This is in line with a 2002 survey (Ong, Petersen, Bodeker et al.) where CAM users scored lower than non-users in most SF-36 categories.

Compared with those who had had an outpatient attendance in the last three months in the Brazier study, patients in this study scored higher (better health) on the physical functioning and lower (worse health) in the social functioning, vitality and, particularly, role-limitation emotional categories. Patients in the current study had a similar profile to those in the MacPherson and Fitter study for social functioning, role-limitation emotional, mental health, general health and vitality.

 

SF-36 Category	Current Study	M&F Study 1Out-patient		GP visit
		(58)	(212)	(290)
Physical functioning	77.6 (348)	67	74	81
Social functioning	67.9 (437)	67	75	76
Role-limitation physical	62.7 (356)	48	63	67
Role-limitation emotional	61.3 (428)	63	72	73
Bodily pain	62.6 (365)	53	64	68
Mental health	64.0 (352)	65	67	66
Vitality	44.0 (357)	44	53	52
General Health	59.2 (423)	62	59	63

 Scores range from 0-100 with a higher score indicating better health

Table 4 – Mean SF-36 scores in this study, the MacPherson and Fitter study and UK population data

 

Treatment outcomes – patients’ overall evaluation of treatment benefit

One of the aims of this study was to explore treatment outcomes from acupuncture through patients’ assessment of their changes in health. The final feedback form surveyed patients overall evaluation of treatment benefit using two outcomes: ‘overall effect (‘of treatment’) on main complaint’ and ‘overall effect (‘of treatment’) on general health’. Each of these is recorded (by the patients themselves) as being ‘full recovery’, ‘major improvement’, ‘moderate improvement’ ‘little better’, ‘no change’, ‘little deterioration’, ‘moderate deterioration’ or ‘major deterioration’.

Out of 495 patients in the study, final feedback was obtained from 214 patients (43.2%) on the overall effect of treatment on their main complaint and 190 (38%) on the effects of treatment on their general health. Only one person recorded that the treatment had been detrimental to the main complaint, and none for general health.

Of the 214 respondents on the effect on main complaint, 21 patients recorded a full recovery, a further 104 a major improvement and 51 a moderate improvement. Thus, over 80% of respondents had registered at least a moderate improvement in their main complaint and over half a major improvement or full recovery. A further 22 registered being a little better and 15 no change.

Almost all patients (88% of respondents) reported at least a moderate improvement in their general health, with six registering full recovery, 94 major improvement and 67 moderate improvement. Of the remaining 23, 14 registered their general health being a little better and nine reported no change. The distribution of the overall effects (excluding the one respondent who recorded that treatment had been detrimental) of treatment on both the main complaint and general health are shown in Figure 2, which compares the proportions in each category. Generally patients felt that the treatment had more effect on their general health than on their main complaint, although, in each case, over 80% had recorded at least a moderate improvement.

Figure 2 – Patients’ assessment of overall effect of treatment

 

Treatment progress in relation to patient profile

One of the project aims was to assess treatment effect in relation to the patient profile to see whether there were any patient characteristics that were a factor in the outcome of treatment. The two outcome variables mentioned above, namely the effect of treatment on the main complaint and on general health, are considered here, to see if there is an association with the patient profile measures, including initial health scores.

Statistical tests were carried out to test whether any of these outcomes was associated with age group, gender, main complaint, its duration, the number of complaints, bodily pain score and general health score on the SF-36 scale. As the data is categorical, being the counts for each patient, analysis consisted of contingency tables and X2 tests of association between two factors. In all cases a 5% significance level is chosen.

As less than half of the patients replied to the overall effects questions, the categories often become too small for the chisquared analysis to be valid; hence in some cases categories have been amalgamated before analysis ’40 and over’; in main complaint the categories are musculo-skeletal, psychological, general and other. As before, duration of complaint is either one of the chronic categories or acute (less than one year). The number of complaints is divided into three categories – one, two and three or more. Following MacPherson and Fitter(1998), bodily pain and general health are selected as baseline measures and the SF-36 scores are categorised into ‘more severe’ (score of less than or equal to 50) or ‘less severe’ (score of more than 50).

The results of the statistical tests concluded that in no case was there found to be a significant association between the treatment outcomes and any of the baseline patient characteristics. There was however a highly significant (p < 0.001) association between the two outcome variables. Patients who recorded full or major improvement in their main complaint generally also recorded full or major improvement in their general health, as might be expected.

Discussion

Participation in this study has increased the research capacity within the College of Integrated Chinese Medicine and lessons have been learnt that can be applied in future research activity. Here the research questions that focused the study are addressed:

What is the profile of patients who attend the college clinic for acupuncture treatment and what are their reasons for attending?

Musculo-skeletal (30.5%), psychological (18.6%), and general (14.8%) conditions were the most predominant complaints in the college over the 12 clinics, followed by neurological (7%) conditions. This is a similar pattern to the patient profile in other recent outcome studies (Chapman, Norton et al. 2001; Andrews and Booth 2004; Bovey, Horner et al. 2005; MacPherson, Sinclair-Lian and Thomas 2006). Acupuncture is relatively well known as a treatment for musculo-skeletal conditions and the proportion of patients in this category has been found higher in other studies (e.g. MacPherson and Fitter 1998; Thomas, Nicholl and Coleman 2001; MacPherson 2005). MacPherson et al (2006) suggest that the proportion of musculo-skeletal patients may be declining over time as patients become more aware about the range of conditions treated by acupuncture.

The general and psychological category includes complaints that are relatively undifferentiated by western medicine, and it is possible that patients turn to alternative medicine when they have not had much success in getting either a diagnosis or effective help through their general practitioner.  The spread of conditions presented to the clinic, as in the Chapman study, is quite broad supporting the suggestion that patient profile studies can be used to educate the public about the range of conditions treated by acupuncture (Chapman, Norton et al. 2001). They can also highlight conditions not commonly presented that may be treated.

Patients’ initial health status, as measured by the SF-36, had a comparable profile to hospital outpatient patients (Brazier, Harper et al. 1992) and worse health than those registering chronic disease or disability in the general population (Knight, Stewart-Brown et al. 2001). Chronic conditions predominated, in line with other studies (Wadlow and Peringer 1996; Bovey, Horner et al. 2005), with the majority of patients (81.4%) having had their condition for more than a year, and about half (49.9%) for more than five years. Patients such as these with a long history of ill health, and with conditions that are not predicted to regress spontaneously, are good subjects for this association research design where the question is practical effectiveness (Walach, Jonas and Lewith 2002).

Most (74.3%) patients present at least two symptoms and some considerably more at diagnosis. In addition 41% of the patients in this study are coming to treatment with expectations of what acupuncture may offer beyond help with a specific complaint. This raises the question of whether symptom-based outcomes measures adequately focus on assessing treatment outcomes of relevance and importance to patients. Research is also needed to explore the possible wider benefits of treatment.

What are the overall effects of acupuncture treatment (as assessed by patients)?

The majority (82.2%) of the 214 patients who returned the final questionnaire reported a large or moderate change in their main condition after acupuncture treatment and 88% a large or moderate change in their general health. No definitive inference that acupuncture caused the changes can be drawn from a study of this kind. Experience has also shown that effectiveness can be high in observational studies but not in controlled trials. However this may indicate a difference in treatment context between real life and less externally valid trial setting (Walach, Jonas et al. 2002) and can support the use of outcome studies as part of a multi-methods approach.

In addition a response rate of 43.2% caused by patient attrition and missing forms affects the associations that are implied by these results, through non-response bias. This is a common problem in outcomes studies (Walach, Jonas et al. 2002; Bovey, Horner et al. 2005). It has been suggested that sensitivity analysis (Briggs and Sculpher 1995; in Richardson 2001) could be employed to increase confidence in results by making various assumptions about nonresponders to estimate different effects (Walach, Jonas et al. 2002; Bovey, Horner et al. 2005)1 as used by Richardson(2001). This was beyond the scope of this paper but patient profile factors (age, age group, gender, main complaint, duration and frequency of main complaint) were compared for responders and non-responders and there was little difference with percentages within 1%-2%. Further analysis of patient records showed that non-responders had attended for a similar number of treatments with a slightly higher percentage having less than three treatments. Sixty non-responders stopped treatment before the end of the clinic with reasons collected from twenty. For these patients the reasons do not indicate positive or negative bias. Sixty-one patients (21.7%) who did not complete a final feedback form were known to continue with treatment after the clinic finished which may indicate satisfaction for these patients but analysis of non-response bias is inconclusive with respect to the remaining patients.

These caveats notwithstanding, the results from this study provide positive evidence of association between acupuncture treatment and health outcomes as assessed by these patients. They also support the findings of other acupuncture outcome studies (e.g.Bovey, Horner et al. 2005) and add weight to the overall evidence profile given the number of patients involved, the high proportion of chronic conditions presented (less likelihood of spontaneous remission), and the range of practitioners involved.

Are any of the patient profile factors associated with the effects of treatment?

MacPherson and Fitter (1998) found more rapid improvements in patients who had had their condition for a shorter time, as well as those with more severe initial conditions. It was surprising in this study that no statistically significant association was found between any patient profile factors and the treatment outcomes given the patient numbers. Does this mean that all patients are as likely to respond to treatment or does this raise further questions?

What has been learnt about the research process?

Higher response rates would improve the robustness of results. Final forms could be after a defined time period rather than a specified number of treatments. However, collection rates are unlikely to improve greatly in this context where busy staff and student-practitioners are responsible for forms. Better collection rates would necessitate extra research administration support on-site on each clinic day. This was beyond the funding available for this project.

Fitter and Thomas (2005) in discussing participation of acupuncturists in national studies suggested that motivation of practitioners is important to response rates. Training was given to student-practitioners to engage them and motivate them. Many of the students reported that they found the study useful in giving them tools to discuss treatment progress more formally with patients. Set against the pressures of course requirements, greater dedication would be hard to encourage unless students receive credits or other incentives.

Conclusion

This study has shown a clear pattern of association between acupuncture treatment and positive health outcome. Results from a wide range of predominantly chronic patients indicate that the majority of patients responding perceive definite benefit from acupuncture. Patient-assessed measures of subjective health status can show patient satisfaction with treatment even if they are not backed up by objective measurement (Bowling 1995:11) and the results from this study show high levels of satisfaction for these patients.

Outcome studies such as this can be used by practitioners to assess patients’ progress and reflect on practice when evidence is needed of general effectiveness. Outcomes research using patient-assessed evidence of effectiveness can also answer the wider community’s desire for knowledge about the benefits of treatment particularly when a high proportion of patients have chronic conditions and thus stable baseline health status, and when evidence is put together with that from other studies in a range of settings. This can be practically relevant for local decision making as it can inform practitioners making clinical decisions on acupunctures suitability (Lewith, Walach et al. 2002). It can also initiate debate with conventional practitioners and the wider public and strengthen the case for further research work.

Acknowledgements

The study has been funded by CICM and the Oriental Medicine Research Trust (OMRT). With thanks for support from Mark Bovey and Alison Gould at ARRC, from Hugh MacPherson and Mike Fitter at the Foundation for Traditional Chinese Medicine, from Claire Stephenson, Jane Roberts, Karen Proudfoot, Fiona Barry, Peter Linthwaite, Carol Horner, Robert Pedley, Jessica Kennedy, Peter Mole and the other OMRT advisors and trustees, from Chi-Keong Paul Ong at Oxford University, to Wanda Glowacka and the CICM admin. team and all the student practitioners who gathered data.

Note

1 Pointed out by Mark Bovey in personal communication January 2003.

References

Andrews, H. and J. Booth (2004). ‘The first step to auditing our practice.’ European Journal of Oriental Medicine 4(6): 58-63.

Bovey, M., C. Horner, J. Shaw, et al. (2005). ‘Engaging in the audit of acupuncture practice.’ The Journal of Alternative and Complementary

Medicine 11(2): 293-298.

Bowling, A. (1995). Measuring disease. Bury St Edmunds, Open University Press.

Brazier, J., R. Harper, N. Jones, et al. (1992). ‘Validating the SF-36 health survey questionnaire: a new outcome measure for primary care.’ British

Medical Journal 305: 160-4.

Briggs, A. and M. Sculpher (1995). ‘Sensitivity analysis in economic evaluation: a review of published studies.’ Health Econ 4: 355-371.

Chapman, R., R. Norton and C. Paterson (2001). ‘A descriptive outcome study of 291 acupuncture patients.’ European Journal of Oriental Medicine

3(5): 48-53.

Fitter, M. and K. Thomas (2005). ‘Duty, curiosity, and enlightened selfinterest: What makes acupuncture practitioners participate in national

research studies?’ The Journal of Alternative and Complementary Medicine 11(2): 227-228.

Gray, D. (2004). Doing Research in the Real World. London, Sage publications.

Knight, M., S. Stewart-Brown and L. Fletcher (2001). ‘Estimating health needs: the impact of a checklist of conditions and quality of life

measurement on health information derived from community surveys.’ Journal of Public Health Medicine 23(3): 179-186.

Lamberts, H., M. Wood and I. Hofman-Okkes (1993). The international classification of primary care in the European community, Oxford University

Press.

Lewith, G., H. Walach and W. Jonas (2002). Balanced research strategies for complementary and alternative medicine. Clinical research in

complementary therapies: principles, problems and solutions. G. Lewith, W. Jonas and H. Walach. London, Churchill Livingstone: 3-27.

MacPherson, H. (2005). ‘Strengthening the case that acupuncture is safe in competent hands.’ European Journal of Oriental Medicine 5(1): 60-73.

MacPherson, H. and M. Fitter (1998). ‘Factors that influence acupuncture: an evaluation of change with acupuncture.’ Acupuncture and Medicine

(16): 33-39.

MacPherson, H., N. Sinclair-Lian and K. Thomas (2006). ‘Patients seeking care from acupuncture practitioners in the UK: a national survey.’

Complementary Therapies in Medicine 14(1): 20-30.

Ong, C.-K., S. Petersen, G. Bodeker, et al. (2002). ‘Health status of people using complementary and alternative medical practitioner services in

four counties in England (UK).’ American Journal of Public Health (92): 1653-6.

Richardson, J. (2001). ‘Developing and evaluating complementary therapy services: Part 2. Examining the effects of treatment on health status.’

Journal of Alternative Complementary Medicine 7(4): 315-328.

Thomas, K. and M. Fitter (2002). Possible research strategies for evaluating CAM interventions. Clinical research in complementary therapies:

principles, problems and solutions. G. Lewith, W. Jonas and H. Walach. London, Churchill Livingstone: 59-91.

Thomas, K. J., J. P. Nicholl and P. Coleman (2001). ‘Use and expenditure on complementary medicine in England: a population based survey.’

Complementary therapies in Medicine 9: 2-11.

Wadlow, G. and E. Peringer (1996). ‘Retrospective survey of patients and practitioners of traditional Chinese medicine in the UK.’ Complementary

therapies in Medicine 4:1-7.

Walach, H., W. Jonas and G. Lewith (2002). The role of outcomes research in evaluating complementary and alternative medicine. Clinical

research in complementary medicines. G. Lewith, W. Jonas and H. Walach. London, Churchill Livingstone: 29-45.