OR-george schmid

7/29/2019 OR-george schmid

1/50

How to Do Operational

Research

George Schmid, M.D., M.Sc.

World Health Organization/Centers for

Disease Control and Prevention

[email protected]


2/50

Format Informal

Practicalif you want to hear about modeling (alegitimate, important part of OR), run out the door now

The session is for you, and us

You will forget 90% of what you hear today. So, we've

given you materials on four CD-ROMs: Designing HIV/AIDS Intervention Studies (and other materials)

(Pop Council);

Training course materials in operations research (Pop Council)

Framework for Operations and Implementation Research in Health

and Disease Control Programmes (and other materials) (GlobalFund and WHO)

Other materials (note the FHI guide to qualitative research)


3/50

Doing Operations andImplementation Research


4/50

Research Techniques

Systematic data collection

Qualitative techniques

Quantitative techniques Surveys, experiments, focus groups.


5/50

The OR Process

Problem identification

Solution generation

Solution testing Results dissemination

Results utilization


6/50

OR Requires Collaboration

Managers:

responsible for decision parameters anddesired outcome

Researchers:

responsible for recommending andimplementing research techniques


7/50

Taking My Problem Into

Operational Research


8/50

Q1.What Proportion of Individuals

Potentially Helped by Bed Nets are

Using Them?1. 2%

2. 7%

3. 14%

4. 24%

Morel CM et al. BMJ 2005


9/50

Taking My Problem Into

Operational Research


10/50

First, You Must Recognize a Problem

Wanting to know

Wanting to act

Having the ability to act


11/50

Q2. Who Typically Identifies Problems

for OR? (more than one answer

possible)1. The patient

2. The programme staff

3. The programme manager

4. The District manager

5. National staff


12/50

Zambia Example

Problem: TB patients are not being

tested for HIV

Observation: "TB Corners" where TBpatients are seen are in open hallways

Intervention: Provide counselling with

privacySome issues simply need some common

sense, and initiative


13/50

Identifying Problems

The role of anyone who works in the

programme!

But, primarily, the role of the manager Identifying problems requires a team

approach

Managers must be present in their programmes Managers must have good relationships with

their staff


14/50

You've Identified a Potential Problem

Patients may not be taking their ART

People may not be using condoms

Patients may not be returning forfollow-up visits

Your health care workers seem to

have high rates of absenteeism .


15/50

The OR Process

1. Problem identification

2. Solution generation

3. Solution testing4. Results dissemination

5. Results utilization

Diagnostic step

Intervention step


16/50

There are Lots of Ways to Approach

OR

Some are simple, others more formal


17/50

Simple? Complex?

Attendance at the ART clinic is low

Patients do not have transport

Injection drug users are not usingclean needles

Clean needles are not available


18/50

There are Lots of Ways to Approach

OR

Some are simple, others more formal

Flexibility needed and desired, to

meet the objectives of what you needto do

But, let's talk about more formal

approaches now.


19/50

The Menu of the WHO/GF OR Book

1. Research team

2. Determine Issues

3. Develop proposal

4. Ethical clearance

5. Funding

6. Budget

7. Capacity building

8. Monitor project

9. Pre-test

10. Quality control

11. Stakeholder

discussions12. Dissemination plan

13. Disseminate results

14. Document changes

15. Monitor changes

16. Consider furtherimprovement

WHO (TDR)/GF (Jane Kengeya-Kayondo, George Shakarishvili, Serge Xueref)

and Bill Brieger and Amy Ellis of Johns Hopkins School of Public Health


20/50

Three Phases to OR

1. Planning

2. Implementation

3. Follow Through


21/50

Planning

1. Organize the Research Group

What disciplines do you need? Epidemiology, behavioural science?

Statistics?

Economics (It is difficult to retroactively do the economicspart of a study)?

Structure

Assign tasks, with deadlines

Meet regularly

Give credit!!!!!!

Who had the research idea? Builds teamwork among departments/persons and avoids

hard feelings

Pays off in the future

Pl i


22/50

Planning

2. Determine the research questions

(and objectives) Be clear, be specific

Research questionWhat do you want to know?

Observation

"Our health care workers seem to have high rates of

absenteeism"

Research question

"What is the rate of absenteeism among our nurses in the

Kiev city hospital, and the reasons for it?"

Objective

Not: To study adherence

But: To determine rates of adherence among attendees at the

XX hospital and reasons for adherence and nonadherence

Pl i


23/50

Planning

3. Develop a research

proposal/protocol

A document that details:

1. What the research is about and why it

is important

2. How the study will be conducted

3. How the results will be used

Planning


24/50

Planning

3. Develop a research

proposal/protocol"standard parts" Research objectives and questions

Background Literature review

Local context

Research team

Methods (study design, study population,sampling details, how data will be collected, datamanagement, quality assurance)

Data collection instruments Plans for use of data

Budget


25/50

Planning

3. Develop a research protocol

Does every "formal" OR project need

a research protocol?

Yes

Makes you clearly state what you will do

(and helps to improve your thinking)

Gives you the plan that you will follow, so

everyone knows what will be done, andshould be done


26/50

Q3. Does Every OR Proposal need

to go through Ethical Review?

1. Yes

2. No


27/50

Planning

4. Obtain ethical clearance


28/50

Q4. What is the most common thing

wrong with consent forms?

1. Do not contain the contact

information for the local ethics board

2. Cannot be understood by potentialparticipants

3. Do not mention the word "research"

4. All risks are not revealed5. Give the patient too much information


29/50

Planning

5. Identify funding

A variety of sources

Requests for small amounts of money arealmost always preferred

Follow closely the guidelines of fundingagencies

Ensure projects are well-written

Having money from one agency for aproject enhances the ability to get furtherfunds from any agency, i.e., "a proven trackrecord"


30/50

Planning

6. Establish a budget

Should be comprehensive, even if nota large budgetensure you havecosted everything, including

dissemination costs A good budget is a sign of careful

planning

? Include a financial officer


31/50

Planning

7. Technical capability/capacity

building

Ensure that staff know their roles and

responsibilities

Ensure that staff have the training and

capability to perform their jobs

Short term training (courses, TA)

Long term training (building capacity)

Help staff gain skills to advance


32/50

Implementation

8. Quality Assurance of the project

There must be a protocol!

There almost always is a work plan, that includes: All activities to be implemented, before and during the

study

Starting date, completion date, intermediate deadlines

Persons who are responsible for each activity

Milestones for each activity

An advisory group that meets regularly may assistin maintaining quality and the work plan

Monitoring should be part of regular team meetings

A quality assurance plan should be part of theprotocol

If you do not perform the study exactly as you

said you would, you cannot trust the results


33/50

Implementation

9. Pre-test all research materials

Valid? (Do data collection instrumentscapture the desired information reliably?)

Reliable? (Do data collection instrumentscapture the desired information

consistently?)


34/50

Implementation

9. Pre-test all research materials

Questionnaires Is the wording clear? Do people understand what

you want? Do the questions ask what they aresupposed to?

Test in 5-20 persons

Focus groups Are the guides clear?

Hold one or two focus groups.

Consent forms

Do people understand them? Language level?(have your child read it) (sometimes, to checkunderstanding during study, develop a series of 5or so questionspatient must answer 4 or 5 toenter study)

Dissemination


35/50

Dissemination

10. Establish and maintain data

management and QA

Forms should be easy to use

Observe data collectors regularly,supervisors review all data collection

instruments daily for completeness andaccuracy, have regular meetings of the datacollectors (daily, likely)

Data entry into computers preferably asstudy is ongoing so errors by datacollectors can be quickly caught and fixed

Double-entry of data?

Dissemination


36/50

Dissemination

11. Explore with stakeholders

interpretation of results

Review preliminary results with theteam early, e.g., within a week afterdata entry is complete, to identifyreasons for unusual findings

Subsequently, review data with theteam and stakeholders, e.g.,

community leaders or project advisorygroup, to identify key findings andinterpretations

Follow-through


37/50

Follow-through

12 & 13. Develop dissemination plan

and disseminate the results

Audience #1 ("internal"). Those who can benefitfrom, and act on, the information locally Local meetings

Meetings with decision makers

Audience #2 ("external"). Those who can adaptthe information for their own context, or use it forfurther research National or international meetings

Journals Create the right documents/strategy/medium for

the right audience


38/50

Q5. Who Need Not Be Part of the

Pre-Test Evaluation of Materials?

1. Potential participants

2. Staff who use the materials, e.g.,interviewers, focus group leaders

3. Data management staff4. The members of the research team

developing the materials

5. All must be

Dissemination


39/50

Dissemination

14. Document changes as a result of

the research

Were the results implemented? If not,

why not?

Dissemination


40/50

Dissemination

15. Monitor changes in the revised

program

If the findings were valid, andappropriately implemented, thenprogram changes should occur whenthe results are implemented

More of a monitoring (and evaluation)function than OR, but gets into the

area of Translation Research.

Dissemination


41/50

Dissemination

16. Consider ways of further

improving the program

Q6 Which Format of Information


42/50

Q6. Which Format of Information

Sharing Do Decision-makers Like

Least (Industrialized Country)1. Internet/e-mail

2. Meetings/conferences

3. Colleagues

4. Journal articles

5. Short summaries

Dobbins M et al. Implementation Sci 2007


43/50

Q7. Who Typically Acts on Identified

Problems?

1. The patient




5. The National staff

6. Too few people act

Q1 What Proportion of Individuals


44/50

Q1.What Proportion of Individuals

Potentially Helped by Bed Nets are

Using Them?1. 2%

2. 7%

3. 14%

4. 24%

Morel CM et al. BMJ 2005

Q2 Who Typically Identifies Problems


45/50

Q2. Who Typically Identifies Problems

for OR? (more than one answer

possible)1. The patient




5. National staff

Q3 D E OR P l d


46/50

Q3. Does Every OR Proposal need

to go through Ethical Review?

1. Yes

2. No

Q4 Wh t i th t thi


47/50

Q4. What is the most common thing

wrong with consent forms?

1. Do not contain the contact

information for the local ethics board

2. Cannot be understood by potentialparticipants

3. Do not mention the word "research"

4. All risks are not revealed5. Give the patient too much information

Q5 Wh N d N t B P t f th


48/50

Q5. Who Need Not Be Part of the

Pre-Test Evaluation of Materials?

1. Potential participants

2. Staff who use the materials, e.g.,interviewers, focus group leaders

3. Data management staff4. The members of the research team

developing the materials

5. All must be

Q6 Which Format of Information


49/50

Q6. Which Format of Information

Sharing Do Decision-makers Like

Least (Industrialized Country)1. Internet/e-mail

2. Meetings/conferences

3. Colleagues

4. Journal articles

5. Short summaries

Dobbins M et al. Implementation Sci 2007

Q7 Wh T i ll A t Id tifi d


50/50

Q7. Who Typically Acts on Identified

Problems?

1. The patient




5. The National staff

6. Too few people act

OR-george schmid

Documents

Transcript of OR-george schmid