Informatie Analyse

34
© 2011 IBM Corporation Informatie Analyse Laila Fettah – Associate Sales Engineer SPSS 27 January 2011

description

Laila Fettah – Associate Sales Engineer SPSS 27 January 2011. Informatie Analyse. Agenda. Government – Challenges Data mining CRISP-DM Example Application. Ongoing Budget Pressures. Lack of Decision-Quality Information. Ongoing Improvement, Less Resources. - PowerPoint PPT Presentation

Transcript of Informatie Analyse

Page 1: Informatie Analyse

© 2011 IBM Corporation

Informatie Analyse

Laila Fettah – Associate Sales Engineer SPSS

27 January 2011

Page 2: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Agenda

Government – Challenges Data mining CRISP-DM Example Application

2

Page 3: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Government faces challenges everyday…

Demonstrate Effective Public

Policy

Ongoing Budget Pressures

Lack of Decision-Quality

Information

Transparency & Accountability

Ongoing Improvement,

Less Resources

3

Page 4: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

How satisfied

do citizens feel?

Have job creation

programs helped curb

benefits applications?

Have new crime

fighting tactics been

effective?

What fraud patterns

are emerging?

How have collection strategies impacted budgets?

What is likely to

happen in the long-

term?

…and must answer critical questions everyday...

4

Page 5: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Budgeting & Finance

Program Execution

Services Delivery

Workforce/ HR

Executive Leaders

Operations/ Readiness

Information Technology

Supply Chain

…and silos often persist that impact outcomes...

5

Page 6: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Information Technology

Budgeting & Finance

Management

Operations/ Readiness Program

Execution

Services Delivery

Supply Chain

Public Safety Staff

Communities

…analytics can tear down silos

6

Page 7: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

What is data mining?

7

Finding patterns in your data that you can use to do your business better

Business-oriented discovery of patterns producing insight and a predictive capabilitywhich can be deployed widely

Process of autonomously retrieving useful information or knowledge (“actionable assets”) from large data stores or set

“Predictive analysis helps connect data to effective action by drawing reliable conclusionsabout current conditions and future events.”

Gareth Herschel, Research Director, Gartner Group

Page 8: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

What’s in a name?

Data Mining is not a great metaphor– Would mean people who dig for gold are “rock miners”!

Other early candidates:– Knowledge Discovery in Databases (KDD)– “Torturing the data until it confesses”

• “…and if you torture it long enough, it’ll confess to anything!”

8

Page 9: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Traditional analyses

First by gender or offender?

By count of crime typeBy time of

offence

What do I do NOW???

What is the profile of the repeat offenders in my

district?

Give me the number of males and females

within the repeat offenders

Give me the times

that crimes where

committed

Give me a count of the types

of crimesReport 1

Report 2

Report 3

Page 10: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Make individual profiles

A descriptive question

I know from my understanding of

crime that gender, time, place, type of crime, age can be

important

Youth gangs from cities A and B that are mostly active on

Thursday night in the center.Addicts that are mostly active around the central station as

pick pockets………..

There are several profiles for repeat offenders. The most

important are….

Data Mining

What is the profile of the repeat offenders in my district?

Let me think….

Data Mining Technology

Create profiles of repeat offendersbased on gender, time, location,type of crime…

Ok, so I need to talk with the railway and with local

authorities in city A and B….

10

Page 11: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Descriptive (KPI) Predictive (KPP) Prescriptive (Scenario)

Statistics

Profiling

Clustering

Associations

Classification

Scoring

Prediction

Forecasting

Prediction

Scoring

Forecasting

What If

Underlying analyses

11

Page 12: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

CRISP-DM

CRoss Industry Standard Process for Data Mining– Funding from European commission– Non-proprietary– Application/Industry neutral– Tool neutral– Focus on business issues as well as technical analysis– www.crisp-dm.org

Process framework for data mining projects– Process Standardization

12

Page 13: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

CRISP-DM phases

13

Page 14: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Example Application Areas:

Public Safety–Reduce crime–Improve border protection–Proactive disease surveillance–Intrusion and insider threat

detection Customs & Excise, Tax, Social

security–Predict & prevent fraud–Improve collections–Focus investigators &

inspectors

Defense–Increase battle readiness of

assets–Improve employee acquisition,

retention & growth Citizen satisfaction

–Implement continuous citizen feedback loop

–Improve operational processes ……

14

Page 15: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Johnny is arrested for breaking into a carHe is 15 years old and confesses that he wanted to belong to a group of friends

Will he become a repeat offender?

If YES: advise DA and later parole officer?

15

Page 16: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Johnny is arrested for breaking into a carHe is 15 years old and confesses that he wanted to belong to a group of friends

Will he become a repeat offender?

If YES: advise DA and later parole officer?

A citizen reports a burglaryReports that her house was burglarized while she was talking to a representative from the city council

Does this crime resemble others? Is it serial?

Do we have a team working on similar crimes that we can assign it to?

16

Page 17: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Johnny is arrested for breaking into a carHe is 15 years old and confesses that he wanted to belong to a group of friends

Will he become a repeat offender?

If YES: advise DA and later parole officer?

A citizen reports a burglaryReports that her house was burglarized while she was talking to a representative from the city council

Does this crime resemble others? Is it serial?

Do we have a team working on similar crimes that we can assign it to?

A Break-in into a shop is reportedThe perpetrators entered by breaking a window probably between 3am and 5am. Crime was discovered at 6 pm next day

Does it make sense to send out a CSI team?

Is it likely that they’ll find useful evidence?

17

Page 18: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

An organized crime unit wants to bust a drugs ringThe detectives are interested in identifying the central players within a narcotics network

Johnny is arrested for breaking into a carHe is 15 years old and confesses that he wanted to belong to a group of friends

Will he become a repeat offender?

If YES: advise DA and later parole officer?

A citizen reports a burglaryReports that her house was burglarized while she was talking to a representative from the city council

Does this crime resemble others? Is it serial?

Do we have a team working on similar crimes that we can assign it to?

A Break-in into a shop is reportedThe perpetrators entered by breaking a window probably between 3am and 5am. Crime was discovered at 6 pm next day

Does it make sense to send out a CSI team?

Is it likely that they’ll find useful evidence?

Who are the key persons? Who are the leaders?18

Page 19: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Johnny is arrested for breaking into a carHe is 15 years old and confesses that he wanted to belong to a group of friends

Will he become a repeat offender?

If YES: advise DA and later parole officer?

A citizen reports a burglaryReports that her house was burglarized while she was talking to a representative from the city council

Does this crime resemble others? Is it serial?

Do we have a team working on similar crimes that we can assign it to?

A Break-in into a shop is reportedThe perpetrators entered by breaking a window probably between 3am and 5am. Crime was discovered at 6 pm next day

PD uses predictive analytics to profile crimes & criminals to improve solved crime rates and optimize resource usage

Crime Data

Crime record notes and call logs

Surveillance Data

Communication Data

Financial Data

Predictive Modeling for Crime Pattern Detection

An organized crime unit wants to bust a drugs ringThe detectives are interested in identifying the central players within a narcotics network

Does it make sense to send out a CSI team?

Is it likely that they’ll find useful evidence?

Who are the key persons? Who are the leaders?19

Page 20: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Johnny is arrested for breaking into a carHe is 15 years old and confesses that he wanted to belong to a group of friends

Will he become a repeat offender?

If YES: advise DA and later parole officer?

A citizen reports a burglaryReports that her house was burglarized while she was talking to a representative from the city council

Does this crime resemble others? Is it serial?

Do we have a team working on similar crimes that we can assign it to?

A Break-in into a shop is reportedThe perpetrators entered by breaking a window probably between 3am and 5am. Crime was discovered at 6 pm next day

PD uses predictive analytics to profile crimes & criminals to improve solved crime rates and optimize resource usage

Crime Data

Crime record notes and call logs

Surveillance Data

Communication Data

Financial Data

Aspiring Repeat Offender profile…If maleAnd age 14-16And crime =‘car break in’And motive =‘peer pressure’Then repeat risk is HIGH ALERT DA…

Predictive Modeling for Crime Pattern Detection

An organized crime unit wants to bust a drugs ringThe detectives are interested in identifying the central players within a narcotics network

Does it make sense to send out a CSI team?

Is it likely that they’ll find useful evidence?

Who are the key persons? Who are the leaders?20

Page 21: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Johnny is arrested for breaking into a carHe is 15 years old and confesses that he wanted to belong to a group of friends

Will he become a repeat offender?

If YES: advise DA and later parole officer?

A citizen reports a burglaryReports that her house was burglarized while she was talking to a representative from the city council

Does this crime resemble others? Is it serial?

Do we have a team working on similar crimes that we can assign it to?

A Break-in into a shop is reportedThe perpetrators entered by breaking a window probably between 3am and 5am. Crime was discovered at 6 pm next day

PD uses predictive analytics to profile crimes & criminals to improve solved crime rates and optimize resource usage

Crime Data

Crime record notes and call logs

Surveillance Data

Communication Data

Financial Data

Aspiring Repeat Offender profile…If maleAnd age 14-16And crime =‘car break in’And motive =‘peer pressure’Then repeat risk is HIGH ALERT DA…

Crime profile Team 4Cluster ‘Bogus Official’ - Burglary, - Visit by city official, - Entry ‘Back door’, - Victim “Elderly’

Predictive Modeling for Crime Pattern Detection

An organized crime unit wants to bust a drugs ringThe detectives are interested in identifying the central players within a narcotics network

Does it make sense to send out a CSI team?

Is it likely that they’ll find useful evidence?

Who are the key persons? Who are the leaders?21

Page 22: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Johnny is arrested for breaking into a carHe is 15 years old and confesses that he wanted to belong to a group of friends

Will he become a repeat offender?

If YES: advise DA and later parole officer?

A citizen reports a burglaryReports that her house was burglarized while she was talking to a representative from the city council

Does this crime resemble others? Is it serial?

Do we have a team working on similar crimes that we can assign it to?

A Break-in into a shop is reportedThe perpetrators entered by breaking a window probably between 3am and 5am. Crime was discovered at 6 pm next day

PD uses predictive analytics to profile crimes & criminals to improve solved crime rates and optimize resource usage

Crime Data

Crime record notes and call logs

Surveillance Data

Communication Data

Financial Data

Aspiring Repeat Offender profile…If maleAnd age 14-16And crime =‘car break in’And motive =‘peer pressure’Then repeat risk is HIGH ALERT DA…

Crime profile Team 4Cluster ‘Bogus Official’ - Burglary, - Visit by city official, - Entry ‘Back door’, - Victim “Elderly’

CS profile No Deployment…If Break InAnd NightAnd report>12hrsAnd entry =‘broken window’And object=‘Commercial Property’Then probability evidence is 6%…

Predictive Modeling for Crime Pattern Detection

An organized crime unit wants to bust a drugs ringThe detectives are interested in identifying the central players within a narcotics network

Does it make sense to send out a CSI team?

Is it likely that they’ll find useful evidence?

Who are the key persons? Who are the leaders?22

Page 23: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

An organized crime unit wants to bust a drugs ringThe detectives are interested in identifying the central players within a narcotics network

Johnny is arrested for breaking into a carHe is 15 years old and confesses that he wanted to belong to a group of friends

Will he become a repeat offender?

If YES: advise DA and later parole officer?

A citizen reports a burglaryReports that her house was burglarized while she was talking to a representative from the city council

Does this crime resemble others? Is it serial?

Do we have a team working on similar crimes that we can assign it to?

A Break-in into a shop is reportedThe perpetrators entered by breaking a window probably between 3am and 5am. Crime was discovered at 6 pm next day

PD uses predictive analytics to profile crimes & criminals to improve solved crime rates and optimize resource usage

Crime Data

Crime record notes and call logs

Surveillance Data

Communication Data

Financial Data

Aspiring Repeat Offender profile…If maleAnd age 14-16And crime =‘car break in’And motive =‘peer pressure’Then repeat risk is HIGH ALERT DA…

Crime profile Team 4Cluster ‘Bogus Official’ - Burglary, - Visit by city official, - Entry ‘Back door’, - Victim “Elderly’

CS profile No Deployment…If Break InAnd NightAnd report>12hrsAnd entry =‘broken window’And object=‘Commercial Property’Then probability evidence is 6%…

Key PlayersFocus on:• Keith Patterson• Colin Wiertz• Markus Haffey

Predictive Modeling for Crime Pattern Detection

Does it make sense to send out a CSI team?

Is it likely that they’ll find useful evidence?

Who are the key persons? Who are the leaders?23

Page 24: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Johnny is arrested for breaking into a carHe is 15 years old and confesses that he wanted to belong to a group of friends

Will he become a repeat offender?

If YES: advise DA and later parole officer?

A citizen reports a burglaryReports that her house was burglarized while she was talking to a representative from the city council

Does this crime resemble others? Is it serial?

Do we have a team working on similar crimes that we can assign it to?

A Break-in into a shop is reportedThe perpetrators entered by breaking a window probably between 3am and 5am. Crime was discovered at 6 pm next day

PD uses predictive analytics to profile crimes & criminals to improve solved crime rates and optimize resource usage

Management Dashboard

Crime Data

Crime record notes and call logs

Surveillance Data

Communication Data

Financial Data

Predictive Modeling for Crime Pattern Detection

Aspiring Repeat Offender profile…If maleAnd age 14-16And crime =‘car break in’And motive =‘peer pressure’Then repeat risk is HIGH ALERT DA…

Crime profile Team 4Cluster ‘Bogus Official’ - Burglary, - Visit by city official, - Entry ‘Back door’, - Victim “Elderly’

CS profile No Deployment…If Break InAnd NightAnd report>12hrsAnd entry =‘broken window’And object=‘Commercial Property’Then probability evidence is 6%…

Key PlayersFocus on:• Keith Patterson• Colin Wiertz• Markus Haffey

An organized crime unit wants to bust a drugs ringThe detectives are interested in identifying the central players within a narcotics network

Does it make sense to send out a CSI team?

Is it likely that they’ll find useful evidence?

Who are the key persons? Who are the leaders?24

Page 25: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Capture Predict Act

Crime Data

Crime record notes and call logs

Surveillance Data

Communication Data

Financial Data

25

Page 26: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Capture Predict ActCapture Predict Act

Crime Data

Crime record notes and call logs

Surveillance Data

Communication Data

Financial Data

Crime Pattern & Hotspot Clustering

Automated Link AnalysisProfiles & Associations

Predictive Modeling for Crime Pattern Detection

26

Page 27: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Capture Predict ActCapture Predict Act

Crime Data

Crime record notes and call logs

Surveillance Data

Communication Data

Financial Data

Crime Pattern & Hotspot Clustering

Automated Link AnalysisProfiles & Associations

Criminal Career Scoring Model

MO Typology Model

Crime Scene Assessment Model

Predictive Modeling for Crime Pattern Detection

27

Page 28: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Capture Predict ActCapture Predict Act

Crime Data

Crime record notes and call logs

Surveillance Data

Communication Data

Financial Data

Crime Pattern & Hotspot Clustering

Automated Link AnalysisProfiles & Associations

Criminal Career Scoring Model

MO Typology Model

Crime Scene Assessment Model

Arresting Officer

Case AssignmentOfficer

CSI Resource Planner

Alert!Aspiring Repeat Offender Risk HIGHAdvise DA and inform parole officer

Alert!Serial Crime ProfileMO fits Team 4

Alert!Very Low Likelihood EvidenceProbability <10% No Deployment

Predictive Modeling for Crime Pattern Detection

28

Page 29: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Capture Predict ActCapture Predict Act

Crime Data

Crime record notes and call logs

Surveillance Data

Communication Data

Financial Data

Crime Pattern & Hotspot Clustering

Automated Link AnalysisProfiles & Associations

Criminal Career Scoring Model

MO Typology Model

Crime Scene Assessment Model

Investigative Model Template Repository

Arresting Officer

Case AssignmentOfficer

CSI Resource Planner

Investigating Officer

Predictive Modeling for Crime Pattern Detection

Feedback resultsFeedback loop of new data to improve and adapt predictions

Key PlayersFocus on:• Keith Patterson• Colin Wiertz• Markus Haffey

Alert!Aspiring Repeat Offender Risk HIGHAdvise DA and inform parole officer

Alert!Serial Crime ProfileMO fits Team 4

Alert!Very Low Likelihood EvidenceProbability <10% No Deployment

29

Page 30: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Capture Predict ActCapture Predict Act

Crime Data

Crime record notes and call logs

Surveillance Data

Communication Data

Financial Data

Crime Pattern & Hotspot Clustering

Automated Link AnalysisProfiles & Associations

Criminal Career Scoring Model

MO Typology Model

Crime Scene Assessment Model

Investigative Model Template Repository

Arresting Officer

Case AssignmentOfficer

CSI Resource Planner

Analytical Process Automation & OptimizationAutomate prediction & deployment process

Analytical Process Management & ControlMonitor & manage analytics process

Predictive Modeling for Crime Pattern Detection

Feedback resultsFeedback loop of new data to improve and adapt predictions

Investigating Officer

Key PlayersFocus on:• Keith Patterson• Colin Wiertz• Markus Haffey

Alert!Aspiring Repeat Offender Risk HIGHAdvise DA and inform parole officer

Alert!Serial Crime ProfileMO fits Team 4

Alert!Very Low Likelihood EvidenceProbability <10% No Deployment

30

Page 31: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

Capture Predict ActCapture Predict Act

Crime Data

Crime record notes and call logs

Surveillance Data

Communication Data

Financial Data

Crime Pattern & Hotspot Clustering

Automated Link AnalysisProfiles & Associations

Criminal Career Scoring Model

MO Typology Model

Crime Scene Assessment Model

Investigative Model Template Repository

Arresting Officer

Case AssignmentOfficer

CSI Resource Planner

Analytical Process Automation & OptimizationAutomate prediction & deployment process

Analytical Process Management & ControlMonitor & manage analytics process

Predictive Modeling for Crime Pattern Detection

Management Dashboard

Feedback resultsFeedback loop of new data to improve and adapt predictions

Investigating Officer

Key PlayersFocus on:• Keith Patterson• Colin Wiertz• Markus Haffey

Alert!Aspiring Repeat Offender Risk HIGHAdvise DA and inform parole officer

Alert!Serial Crime ProfileMO fits Team 4

Alert!Very Low Likelihood EvidenceProbability <10% No Deployment

31

Page 32: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

32

Start from business understanding… not from data or technique…

Page 33: Informatie Analyse

© 2011 IBM Corporation

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

33

…and use a methodology!

Page 34: Informatie Analyse

© 2011 IBM Corporation

Questions

Van informatie op Orde naar Informatie van Waarde – 27 januari 2011

34