Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

96
Evidence-Based Practice Lunchbijeenkomst 14 maart 2013

Transcript of Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Page 1: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Evidence-Based Practice

Lunchbijeenkomst

14 maart 2013

Page 2: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.
Page 3: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

HBO Universiteit

Post Grad

(E)MBA

Business Schools

Page 4: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

1. Wat moet een student in de praktijk straks kunnen:

- zelf onderzoek doen?

- onderzoek kunnen beoordelen + toepassen?

2. Wat betekent dat voor het curriculum?

3. Wat betekent dat voor de accreditatie?

Aanleiding

Page 5: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

“In onze moderne samenleving is het cruciaal dat hbo-

bachelors over een onderzoekend vermogen beschikken

dat leidt tot reflectie, tot evidence-based practice, en tot

innovatie.”

HBO standaard (2009)

Page 6: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Evidence-Based Practice ?

Page 7: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Wat is het?

Waar komt het vandaan?

Hoe ziet het er uit in een opleiding?

Hoe zit dat bij 4e jaars studenten?

Page 8: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Evidence based practice:

Wat is het?

Page 9: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Uitgangspunt bij evidence-based practice is dat beslissingen gebaseerd dienen te zijn op een combinatie van wetenschappelijk denken en de best beschikbare 'evidence'.

Evidence-based practice

Page 10: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Met het begrip 'evidence' wordt niet meer bedoeld dan 'informatie'.

Dit kan informatie zijn afkomstig uit wetenschappelijk onderzoek, maar ook interne bedrijfsinformatie en zelfs persoonlijke ervaring geldt als 'evidence’.

Evidence based practice

Page 11: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

In principe neemt iedere manager dus beslissingen op basis van 'evidence'.

De meeste managers besteden echter nauwelijks aandacht aan de kwaliteit van de 'evidence' waarop ze hun beslissingen baseren.

Evidence based practice

Page 12: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Evidence-based practice:

kritisch en wetenschappelijk denken

van verschillende informatiebronnen gebruik maken

de beschikbare evidence kritisch tegen het licht houden

denken in termen van waarschijnlijkheid in plaats van 'golden bullets'.

Page 13: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Professional expertise and judgement

Best available organizational

evidence

Stakeholders’ values and concerns

Best available scientific evidence

Evidence-based decision

Evidence based practice

Page 14: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Evidence based practice:

Waar komt het vandaan?

Page 15: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

“there is a large research-user gap”

“practitioners do not read academic journals”

“the findings of research into what is an effective intervention

are not being translated into actual practice”

“academics not practitioners are driving the research agenda”

“the relevance, quality and applicability of research is

questionable”

“practice is being driven more by fads and fashions than

research”

“many practices are doing more harm than good”

What field is this?

Page 16: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

McMaster University Medical School, Canada

Medicine: Founding fathers

David Sackett Gordon Guyatt

Page 17: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

How it all started

Page 18: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

if you’re hyperventilating

breathe into a bag

Problem I: persistent convictions

Page 19: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

elderly people who have an irregular heartbeat are much more likely to die of

coronary disease

give them a drug that reduces the number of

irregular beats

Problem I: persistent convictions

Page 20: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

How 40,000 cardiologists can be wrong

In the early 1980s newly introduced anti-arrhythmic drugs were found to be

highly successful at suppressing arrhythmias.

Not until a RCT was performed was it realized that, although these drugs suppressed arrhythmias, they actually increased mortality.

By the time the results of this trial were published, at least 100,000 such patients had been taking these drugs.

Page 21: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

David Sackett

Half of what you learn in medical school will be

shown to be either dead wrong or out-of-date

within 5 years of your graduation; the trouble is that

nobody can tell you which half.

The most important thing to learn is how to learn

on your own: search for the evidence!

(Remember that your teachers are as full of bullshit

as your parents)

Page 22: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

More than 1 million articles in 40,000 medical journals per

year (= 1995; now probably more than 2 million). For a

specialist to keep up this means reading 25 articles every

day (for a GP more than 100!)

Problem II: too much information

Page 23: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Problem II: too much information

HRM: 1,350 articles in 2010 (ABI/INFORM). For an HR

manager to keep up this means reading 3 to 4 articles

every day (for a ‘general’ manager more than 50!)

BTW: most of the research is seriously

flawed or irrelevant for practice

Page 24: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

The 5 steps EBP

1. Formulate a focused question (Ask)

2. Search for the best available evidence (Acquire)

3. Critically appraise the evidence (Appraise)

4. Integrate the evidence with your professional

expertise and apply (Apply)

5. Monitor the outcome (Assess)

Zelf onderzoek doen?

Page 25: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Evidence-Based Practice

1991Medicine

1998Education

1999Social care, public policy

2000Nursing

2000Criminal justice

????Management?

Page 26: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Evidence based practice:

Hoe ziet het er uit in de opleiding?

Page 27: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Fase 1: Het ontwikkelen van kritisch en wetenschappelijk denken dat leidt tot een professioneel-kritische attitude met betrekking tot organisatievraagstukken

Fase 2: Het kunnen formuleren van een expliciete vraag en op basis van deze vraag kunnen zoeken in online databases naar uitkomst van relevant wetenschappelijk onderzoek.

Fase 3: Het kritisch kunnen beoordelen van wetenschappelijke en organizational evidence (critical appraisal)

Fase 4: Uitkomst van wetenschappelijk onderzoek kunnen toepassen in de eigen beroepspraktijk (application of science)

Evidence-based practice

Page 28: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Professional expertise and judgment

Best available organizational

evidence

Stakeholders’ values and concerns

Best available scientific evidence

Evidence-based decision

Evidence based practice

Page 29: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Waarom (wetenschappelijk) onderzoek?

Page 30: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Trust me, 20 years of experience

Page 31: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Bounded rationality

Page 32: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

“The first principle is that you must not fool yourself - and you are the easiest person to fool”.

Richard Feynman

Page 33: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Bounded rationality

Page 34: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Het feilbare brein

System 1

Snel, actie Intuitief, associatief shortcuts & biasses

System 2

Langzaam (lui!) Rationeel Nadenken

Page 35: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Systeem 1

Page 36: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Seeing order in randomness Mental corner cutting Misinterpretation of incomplete data Halo effect False consensus effect Group think Self serving bias Sunk cost fallacy Cognitive dissonance reduction

Confirmation bias Authority bias Small numbers fallacy In-group bias Recall bias Anchoring bias Inaccurate covariation detection Distortions due to plausibility

Systeem 1: het feilbare brein

Page 37: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Het feilbare brein- Meningen

- Assumpties (aannames)

- Overtuigingen

- Persoonlijke ervaringen

Wetenschappelijk onderzoek- Feiten

- Evidence: bewijs / aanwijzingen

Waarom onderzoek?

BIAS

Page 38: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

1. Denkfouten

2. Informatiebronnen

3. Mythbusting

4. Assumpties

Fase 1: Kritisch & wetenschappelijk denken

Page 39: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

1. Seeing order in randomness

2. Confirmation bias

3. Small numbers fallacy

4. Outcome bias

5. Halo effect

6. Authority bias

7. Group think

7 Denkfouten (die u beter aan anderen kunt overlaten)

Page 40: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Een Type I fout of een vals positief: denken dat er een patroon / verband is terwijl dat er in het echt niet is.

Een Type II fout of een vals negatief: denken dat er geen patroon / verband is terwijl dat er in het echt wel is

Dr. Michael Shermer (Director of the Skeptics Society)

Seeing order in randomness

Page 41: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Een Type I fout of een vals positief: denken dat het geritsel in de bosjes een gevaarlijk roofdier is, terwijl het gewoon de wind is (goedkoop foutje)

Het feilbare brein: patern recognition

Page 42: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Een Type II fout of een vals negatief: denken dat het geritsel in de bosjes gewoon de wind is, terwijl het een gevaarlijk roofdier is (duur foutje)

Het feilbare brein: patern recognition

Page 43: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Het probleem met patroon herkenning:

Het kritisch beoordelen of er sprake is van een

Type I of een Type II fout is best moeilijk,

(vooral in ‘split second life and death’ situaties),

dus de default positie is om aan te nemen dat

alle patronen echt zijn.

Het feilbare brein: patern recognition

Page 44: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Een Type I fout of een vals positief: denken dat het geritsel in de bosjes een gevaarlijk roofdier is, terwijl het gewoon de wind is (goedkoop foutje)

DEFAULT

Een Type II fout of een vals negatief: denken dat het geritsel in de bosjes gewoon de wind is, terwijl het een gevaarlijk roofdier is (duur foutje)

Het feilbare brein: patern recognition

Page 45: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Ook ervaren mensen en experts zien patronen en verbanden waar ze niet zijn.

stress & lifestyle peptic ulcer

Het feilbare brein: patern recognition

Page 46: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Peptic ulcer – an infectious disease!

This year's Nobel Prize in Physiology or Medicine goes to Barry Marshall and Robin Warren, who with tenacity and a prepared mind challenged prevailing dogmas. By using technologies generally available (fibre endoscopy, silver staining of histological sections and culture techniques for microaerophilic bacteria), they made an irrefutable case that the bacterium Helicobacter pylori is causing disease. By culturing the bacteria they made them amenable to scientific study.

In 1982, when this bacterium was discovered by Marshall and Warren, stress and lifestyle were considered the major causes of peptic ulcer disease. It is now

firmly established that Helicobacter pylori causes more then 90% of duodenal ulcers. The link between Helicobacter pylori infection and peptic ulcer disease has been established through studies of human volunteers, antibiotic treatment studies and epidemiological studies.

Oct 2005

Page 47: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Ook slimme mensen houden er verkeerde ideeën op

na, niet omdat ze dom of eigenwijs zijn, maar omdat het

de meest logische conclusie is op basis van hun eigen

ervaringen.

Het feilbare brein: patern recognition

(systeem 1 doet altijd mee!)

Page 48: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Fase 1: Het ontwikkelen van kritisch en wetenschappelijk denken dat leidt tot een professioneel-kritische attitude met betrekking tot organisatievraagstukken

Fase 2: Het kunnen formuleren van een expliciete vraag en op basis van deze vraag kunnen zoeken in online databases naar uitkomst van relevant wetenschappelijk onderzoek.

Fase 3: Het kritisch kunnen beoordelen van wetenschappelijke en organizational evidence (critical appraisal)

Fase 4: Uitkomst van wetenschappelijk onderzoek kunnen toepassen in de eigen beroepspraktijk (application of science)

Evidence-based practice

Page 49: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.
Page 50: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

5-step approach

EBMgt is a 5-step approach

1. Formulate an answerable question (PICOC)

2. Search for the best available evidence

3. Critically appraise the quality of the found

evidence

4. Integrate the evidence with managerial

expertise and organizational concerns and apply

5. Monitor and evaluate the results

Page 51: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Formulate a focused question

Page 52: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Focused question?

Does team-building work?

What are the costs and benefits of self-steering teams?

What are the success factors for culture change?

Does management development improve the

performance of managers?

Does employee participation prevent resistance to

change?

How do employees feel about 360 degree feedback?

Page 53: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

What is a ‘team’?

What kind of teams?

In what contexts/settings?

What counts as ‘team-building’?

What does ‘work’ mean?

What outcomes are relevant?

Over what time periods?

Foreground question?

Does team-building work?

Page 54: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

P = Population

I = Intervention or success factor

C = Comparison

O = Outcome

C = Context

Answerable question: PICOC

Page 55: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

2. Finding the best available evidence

Page 56: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Searching evidence

What do we search?

Page 57: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Current Information

Overview of a subject

General background

Academic Information

Statistical Information

Theories about a subject

What do we search?

Company information

Page 58: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Peer reviewed journals

Page 59: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Searching evidence

Where do we search?

Page 60: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Searching for evidence

Page 61: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Databases

ABI/INFORM

Business Source Elite

PsycINFO

Web of Knowledge

ERIC

Google Scholar

Page 62: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Searching for evidence

Page 63: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Searching evidence

How do we search?

Search Strategy

Page 64: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Two types of search strategies

Search strategy

Building blocks methodSnowball method

Page 65: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Search strategy

Page 66: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Search in ABI/Inform:

1. How many articles has Stephen Covey published in peer reviewed journals?

2. How many of these articles are based on scientific research?

3. Are there articles (by other authors) that are critical of Covey’s 7 Habits?

4. How many of these critical articles are based on scientific research?

Exercise: Search for evidence

Page 67: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Search in ABI/Inform or BSE:

Search for peer reviewed research articles to answer the following question: What is the long term effect of a hostile take-over on the financial performance of the acquired organization? Use the following search terms:

“hostile takeovers”, “financial performance”, “long term”

How many studies did you find?

Exercise: Search for evidence

Page 68: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Fase 1: Het ontwikkelen van kritisch en wetenschappelijk denken dat leidt tot een professioneel-kritische attitude met betrekking tot organisatievraagstukken

Fase 2: Het kunnen formuleren van een expliciete vraag en op basis van deze vraag kunnen zoeken in online databases naar uitkomst van relevant wetenschappelijk onderzoek.

Fase 3: Het kritisch kunnen beoordelen van wetenschappelijke en organizational evidence (critical appraisal)

Fase 4: Uitkomst van wetenschappelijk onderzoek kunnen toepassen in de eigen beroepspraktijk (application of science)

Evidence-based practice

Page 69: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Which design for which question?

Research designs

Page 70: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Randomized controlled study? Grounded theory approach? Cohort / panel study? Qualitative field research? Longitudinal study? Post-test only study? Survey? Action research? Case study?

What is the best design?

Page 71: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

What is the BEST car?

Page 72: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

What is the best design?

quants vs quallies, positivists vs post structuralist, etc

Page 73: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

on the research question

Page 74: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Effect vs Non-effect

Page 75: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Types of questions

Does it work?

Does it work better than ....?

Does it have an effect on ....?

What is the success factor for ....?

What is required to make it work ...?

Will it do more good than harm?

Effect

Page 76: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Types of questions: non-effect

Needs: What do people want or need?

Attitude: What do people think or feel?

Experience: What are peoples’ experiences?

Prevalence: How many / often do people / organizations ...?

Procedure: How can we implement ...?

Process: How does it work?

Explanation: Why does it work?

Economics: How much does it cost?

Page 77: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Internal validity

Page 78: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

internal validity = indicates to what extent the

results of the research may be biased and is thus

a comment on the degree to which alternative

explanations for the outcome found are possible.

Internal validity

Page 79: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.
Page 80: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

We are pattern seeking primates:

we are predisposed to see order

and causal relations in the world

Causal relations

Page 81: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

1. Are the "cause" and the "effect” related?

2. Does the "cause" precede the "effect" in time?

3. Are there no plausible alternative explanations for the

observed effect?

effect size

before and after measurement

randomization, blinding, control group, measurements

Considerations for research:

Causality

Page 82: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Bias & Confounding

Research shows:

Shoe size > quality of handwriting

Smoking youngsters > better lung function

Page 83: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Levels of internal validity

Page 84: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Explanation

Which design for which question?

Page 85: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Different types of research questions

require different types of research designs,

but ...

Page 86: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Best research design?

But 1: feasibility

Page 87: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

But 1: feasibility

Page 88: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Step 3: Critical appraisal of studies

Page 89: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Intermezzo

How to read a research article?

Page 90: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Critical appraisal: quick and dirty

Is the study design appropriate to the stated aims?

Are the measurements likely to be valid and reliable?

Was there a relevant effect size?

Is the outcome (population, type of organization)

generalizable to your situation?

Page 91: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Levels of internal validity

1. Were there enough subjects in the study?

2. Was a control group used?

3. Were the subjects randomly assigned?

4. Was a pretest used?

5. Was the study started prior to the intervention or event?

6. Was the outcome measured in an objective and reliable way?

6x yes = very high (A)

5x yes = high (A)

4-3x yes = limited (B)

2x yes = low (C)

1-0x yes = very low (D)

Page 92: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

Appraisal

Critical appraisal questionnaires

www.cebma.org/ebp-tools

Page 93: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

CAT: Critically Appraised Topic

Page 94: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

CAT: Critically Appraised Topic

A critically appraised topic (or CAT) is a structured, short (3

pages max) summary of evidence on a topic of interest,

usually focused around a practical problem or question. A CAT

is like a “quick and dirty” version of a systematic review,

summarizing the best available research evidence on a topic.

Usually more than one study is included in a CAT.

Page 95: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

CAT: structure

1) Background / context

2) Question (PICOC)

3) Search strategy

4) Results / evidence summary

5) Findings

6) Limitations

7) Recommendation

Page 96: Evidence-Based Practice Lunchbijeenkomst 14 maart 2013.

CAT-walk