How to Design and Conduct Scientific Research
Guidelines for WISE Researchers

When doing medical, scientific, and other research, it is important that all researchers follow the highest standards of the research protocol, and that they design studies and prepare articles for journals in accordance with those high standards. In order to assist you in the proper design of research studies and in writing high-quality research articles for journals, we are providing you with a detailed outline for designing and conducting clinical biomedical research, since many of you will be engaged in this type of research.

Following that detailed outline, we have listed a number of excellent Internet websites that provide detailed information on designing and conducting research of all kinds.

How to Design & Conduct Clinical Biomedical Research

The following detailed research guidelines may be used for conducting research involving medical or other types of interventions for the treatment of diseases or disorders, or for enhancing one’s health in any way. These guidelines are particularly useful in designing research studies of complementary and alternative medicine interventions since they were written up by the World Health Organization (WHO) to be used in researching traditional ethnic medicine practices around the world. These guidelines came from Chapter 3 of the WHO publication, General Guidelines for Methodologies on Research and Evaluation of Traditional Medicine, WHO, 2000, (74 pages), which may be accessed in its entirety at: http://www.paho.org/Spanish/AD/THS/EV/PM-WHOTraditional-medicines-research-evaluation.pdf

3. Clinical Research

3.1 General considerations

Normally, clinical research of all types of conventional and traditional medicine considers both efficacy and safety and is conducted according to WHO’s guidelines for good clinical practice and the Declaration of Helsinki (see Annex VI). Safety evaluation, however, may not be the main focus of clinical research in traditional medicine, because of the long history of traditional medicine. The information here provides further details to the relevant sections dealing with clinical trials in the assessment of herbal medicines (Part 1) and traditional procedure-based therapies (Part 2).

In addition to evaluating the safety and efficacy of traditional medicine through clinical trials, there may be a number of different objectives when evaluating traditional medicine through clinical research, as when using clinical research to evaluate conventional medicine. Some of the objectives specific to the assessment of traditional medicine through clinical research are to:

• Evaluate traditional medicine in its theoretical framework (e.g. mechanistic studies);
• Devalue traditional medicine in the theoretical framework of conventional medicine (e.g. mechanistic studies);
• Compare the efficacy of different systems of traditional medicine and/or conventional medicine; and
• Compare the efficacy of different traditional practices within a system of traditional medicine.

3.2 Literature Review

The starting point in the design of a research protocol is a complete literature review, including the traditional use of the proposed practice and existing scientific research in the field. Where little or no literature exists, the oral tradition and the source of this tradition need to be clearly stated.

A review of the literature should identify the current level of evidence of efficacy and safety for the proposed intervention. Evaluation of the literature should follow well-established and accepted guidelines (see Part 1, section 1.3). However, meta-analysis in traditional medicine may be difficult, mainly due to the lack of large clinical trials of good quality. In addition, the efficacy of a particular treatment may also vary according to the skill and experience of the practitioner. These issues must be considered and kept in mind.

3.3 Selection of Study Design

Clinical research aimed at evaluating traditional medicine should incorporate the conventional concepts of research design, such as randomized controlled trials or other types of clinical studies, such as observational studies. The USA Food and Drug Administration guidelines Guidance for industry: significant scientific agreement in the review of health claims to conventional foods and dietary supplements, which introduce several types of clinical studies, could be consulted (see Annex VII). The Guideline for Good Clinical Practice produced by the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (see Annex VIII), as well as official guidelines from other governmental agencies (such as those in Annex V) may also be a good reference source for clinical research design.

Conventional concepts of clinical research design may be difficult to apply when using clinical research to evaluate various systems and practices of traditional medicine, depending on the goal of the assessment (see Part 3, section 3.1). In such circumstances, the choice of study design should be discussed on a case-by-case basis with experienced traditional medical practitioners. The study design may be chosen from a whole spectrum of clinical research designs that are suitable for assessing traditional medicine (see Annex VII), including:

Single-case design
Single-case designs have the advantage of being adaptable to the clinical needs of the patient and the therapeutic approach of the practitioner, but have limitations due to their lack of generalization to other patients. Such designs are appropriate for the development of research hypotheses, testing those hypotheses in daily clinical practice and refining clinical techniques. Single-case designs using a common protocol—if the protocol can be systematically followed—should be advocated for collaborative research among practitioners from different backgrounds.

For example, single-case designs can evaluate the effectiveness of various specialized acupuncture methods in patients with a variety of individual differences. In a single-case design, the patient is his or her control. Treatment can be randomized for a patient, rather than the patient being randomized for a treatment.

Black-box design
The study of traditional medicine can also be undertaken in a “black-box” manner. This means that the treatment and all of its components are delivered as they would be in the usual clinical situation. In this type of study, no component of the treatment “package” is isolated and studied independently. This allows the effectiveness of traditional medicine to be determined either within its theoretical framework or within that of conventional medicine.

Ethnographic design
Ethnographic studies that document the social and cultural context in which a traditional practice emanates may be appropriate in situations where there is no available scientific literature or other documentation. These and other qualitative studies can provide baseline information from which hypotheses may be generated and can lead to further research.

Observational design
Observational studies collect findings on a therapeutic or prophylactic treatment under routine conditions. The special feature of these studies is that they seek, as far as possible, not to influence the individual doctor–patient relationship with respect to indications, and the selection of and carrying out the treatment. These studies may be conducted with or without a control group. The specific details of the study (e.g. the time and extent of examination for each individual patient, the number of patients involved) and the envisaged methods (e.g. data recording and evaluation) must be adapted to the question investigated in the study (e.g. safety or appropriate posology). Observational studies have specific advantages in studying aspects of clinical safety. The use of such studies to prove efficacy is limited because bias in patient selection may occur. Nevertheless, the level of evidence on the efficacy of traditional medicine can be significantly increased by well-designed observational studies.

Study outcome measures
It is essential that the outcome measures chosen be appropriate to the research question. Appropriate outcomes may include quantitative and qualitative outcomes; primary and/or secondary outcomes; and generic and/or highly specific outcomes.

Selection of patients
The sample must represent the target population of patients to which the results would be generalized. Publication of the study requires a clear description of the patients using both traditional and conventional terms. The reliability of the categorization/diagnostic criteria used in the study should be considered and stated. The source of the patients under study should be comprehensively described along with details of the recruitment process. The inclusion and exclusion criteria should be completely described and rationalized. Any potential bias in patient selection, recruitment, and enrollment should be excluded. Investigators should be aware of any potential errors that may occur when studying traditional medicine out of context and without reference to its traditional theories and concepts.

When the research involves techniques that depend on skills that may differ between practitioners, such research should be conducted by more than one practitioner to increase the generalizability of the results.

Sample size
The number of patients in a study needs to be adequate, in order to be able to determine any clinically important differences between the study groups. Concerning the study design, the statistical methods used should be appropriate to the proposed analysis of the study’s outcome.

Control groups
A well-conducted and controlled clinical trial could provide sufficient evidence to establish a relationship between the use of herbal medicine or traditional procedure-based therapy and the prevention, diagnosis, improvement or treatment of an illness, provided there is a supporting body of evidence from observational or mechanistic studies.

Randomized controlled trials require one or more control groups for purposes of comparison. The selection of control groups depends on the objectives of the study. In the evaluation of traditional medicine, a concurrent control group should be used. The control groups may involve (not in order of priority):

• Well established treatment
• Non-treatment
• Different doses of the same treatment
• Sham or placebo treatment
• Full-scale treatment
• Minimal treatment
• Alternative treatment

Different controls can be used in clinical trials to answer different questions. The use of a placebo, when possible, is desirable because it generates evidence of better quality. Placebo-controlled trials are intended to establish whether treatment is valuable over and above what might be achieved by a control treatment, and not whether treatment is valuable at all. Thus, it allows researchers to distinguish specific from non-specific effects of treatment to determine whether the additional cost, risk and effort of a specific treatment are worthwhile. It is also important to understand the mechanism of a treatment. This is true for the evaluation of all drugs. It is not only of academic interest but is also of practical value, especially for developing new treatments from traditional ones. However, in some cases, placebo-controlled trials may not be possible (see section on clinical trials in Part 1).

It is preferable to compare herbal medicine with both a well-established treatment and another control group (from the list of control groups) to determine whether the herbal medicine is useful in the context of current best practice. One specific problem in clinical research of traditional medicine is the simultaneous conventional treatment of patients (e.g. cancer patients) in a study. It may not be ethically possible to withdraw the conventional treatment. Therefore, in such cases, the focus of research may be on the additional or supportive effects of traditional medicine. Research on combinations of traditional and conventional medicine should always consider potential therapeutic interactions and side effects (see section on black-box design in Part 3).

Randomization
Randomization has been a tremendous advance in developing comparable groups to assess therapeutic interventions. It is essential to control various known, and even unknown, biases. Nevertheless, there are many situations where randomization can be impossible or unethical. The best way to solve this problem is probably by the proper selection of control treatments.

Blind assessment
Blind assessment is a critical component of conventional evaluation of therapeutic interventions. However, in the evaluation of efficacy of traditional procedure-based therapies (such as physical therapy, surgery, acupuncture, and manual therapy), it can be difficult, impractical or impossible for the practitioner to be kept ignorant of what treatment the patients are receiving. It is essential that this be noted in the evaluation of the validity of a study and that the judgment on its validity be applied consistently across all systems of conventional and traditional medicine.

Treatment blinding in the evaluation of herbal medicines should adopt the approach of conventional medicines, e.g. using active and control formulations with similar color, taste, and weight. However, if the herbal medicine cannot be administered in a predetermined standardized formulation, it will be impossible to keep the treatment blinded. Treatment blinding is also difficult to implement in most types of traditional procedure-based therapies. It is important, however, to reduce any bias introduced by non-blinded treatment by carrying out a blinded assessment of the primary outcomes of the study.

Evaluation of quality of life
Traditional medicine is used not only to prevent, diagnose, improve, and treat illness but also to maintain health and improve the quality of life. For example, traditional medicine may not cure patients with certain illnesses, such as cancer and AIDS, but may help improve their quality of life. The WHO QOL user manual, developed by the WHO Programme on Mental Health, can be used to help evaluate the results of clinical research on herbal medicines and traditional procedure-based therapies (see Annex IX)

Other issues related to therapeutic interventions
In both the development of a study protocol to assess traditional medicine and in its submission for publication or health-authority approval, the following information regarding study outcomes should be clearly provided:

• Description of the therapeutic intervention;
• Description of the reasons for the selection of the therapeutic intervention;
• Description of the rationale for the choice of the study outcomes;
• Description of the outcome measurements, including a review of the validity and reliability of the measurements;
• A comprehensive protocol for taking the measurements (including how and when the measurements were taken); and
• A clear statement of which expected outcomes the statistical method was based on:

          The following issues should also be considered:

• The type of intervention must be clearly defined. In treatment using herbal medicines, this should also include, for example, information on the composition and manufacturing of finished herbal products. In traditional procedure-based therapy, this should include, for example, information on the tools and equipment used.
• The training, skills, and experience of the traditional medical practitioner should be taken into account. Issues concerning the variability of treatment by a single practitioner (intra-practitioner variability) and groups of practitioners (inter-practitioner variability) should be addressed. Ideally, the practitioner’s diagnostic ability should be reliable.
• If the setting is an important component of a treatment, its essential features must be described.
• The dose, frequency, and duration of treatment must be described completely. “Dose” in traditional procedure-based therapies refers to a variety of attributes related to each episode of the therapy, which may vary markedly between different systems of traditional medicine. In acupuncture, for example, “dose” includes the force of physical manipulation, the duration of each episode of therapy, the duration of needled manipulation, the number of repetitions of a procedure, the number of needles used, the depth of stimulation, the needle sensation if elicited, the details of any electrical stimulation including stimulus, frequency, intensity, etc. The “dose” used in any study should be based on the relevant literature and experience of traditional medical practitioners.
• The duration of follow-up should be clearly stated. Its length needs to be appropriate to the treatment carried out. In patients with acute pain, follow-up should be carried out within a 24-hour period. In patients with chronic pain, follow-up of a minimum of several months (e.g. 3–6 months) is desirable.
• Temporal considerations need to be assessed and noted. The study design should take into account seasonal variations that are important to some traditional medicine systems. It should also contain an appropriate time course to allow the treatment to demonstrate its effectiveness. The number of treatments in a finite period of time needs to be clearly stated.

The information in the ICH Harmonized tripartite guideline: guideline for Good Clinical Practice issued by the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use is a useful example of the information required (see Annex VIII).

Additional Internet Resources for Research Design and Methodology

World Health Organization Research Methodology Guidelines
http://www.paho.org/Spanish/AD/THS/EV/PM-WHOTraditional-medicines-research-evaluation.pdf

Quantitative & Qualitative Research
http://writing.colostate.edu/guides/
One of the best websites on research methodology, it covers a range of subjects, including an introduction to statistics, experimental method and design, ethnography, observational research, narrative inquiry, survey research, and content analysis.

Investigating the Social World: The Process and Practice of Research
http://www.pineforge.com/isw4/weblinks.htm
This website has a large number of research design resources that are useful in researching social, biomedical, and other natural sciences.

The Web Center for Social Research Methods
http://pirun.ku.ac.th/~g4986069/research.pdf
This website has excellent resources.

Engage in Research
http://www.york.ac.uk/library/subjectresources/researchmethods/#gen
A new, interactive website for biosciences undergraduates. It covers the basic areas of research, including how and where to search for scientific papers, how to structure a literature review, basic statistics, and much more.

Free Resources for Program Evaluation and Social Research Methods
http://www.york.ac.uk/library/subjectresources/researchmethods/#gen
This page lists free resources for methods in program evaluation and social research. The focus is on “how-to” do evaluation research and the methods used: surveys, focus groups, sampling, interviews, and other methods.

Introduction to Qualitative Research
http://www.york.ac.uk/library/subjectresources/researchmethods/#gen
This useful guide was designed as an introductory text for teachers, health professionals and business consultants studying master’s level courses. Its purpose is “to begin the process of people thinking about doing their own research for their purposes”. Topics that are covered in the guide include a comparison of qualitative and quantitative research, action research and evaluation research.