IMPACT Framework en Evaluatie by Clemens Neudecker

15
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands. IMPACT Framework en Evaluatie Clemens Neudecker, Technisch Project Manager IMPACT

description

Presentation given on the KB IMPACT Demo Day on 16 February 2011 in The Hague.

Transcript of IMPACT Framework en Evaluatie by Clemens Neudecker

Page 1: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

IMPACT Framework en EvaluatieClemens Neudecker, Technisch Project Manager IMPACT

Page 2: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

Achtergrond: Een uiteenlopende set van tools

Verschillende tools van diverse ontwikkelaars,gericht op specifieke problemen

Verschillende platforms (Windows/Linux),executables, DLL’s, SDK’s, broncode

Prototypes of proof-of-concept,state-of-the-art commerciële producten

Gebruikers zonder of met weinig achtergrond in computers

Page 3: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

IMPACT Framework“Een uniform technisch framework waarmee eindgebruikers op een duidelijke en

consistente manier met IMPACT tools en applicaties kunnen werken”

Opgebouwd uit open source software componenten(Apache License 2.0)

Gebaseerd op open standaarden

Service oriented architecture,distributed processing

Page 4: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

Applicatie integratie Transformatie van tools naar web services met gebruik van een generieke wrapper

(Java-based)

Web-based implementatie van tools, platform-onafhankelijk Web services kunnen worden gecombineerd in workflows (ketens)

Page 5: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

Workflow integratie: Mashups OCR workflow =

data pipeline

Bouwstenen = stappen in het proces (knooppunten)

Integratie = interactie tussen knooppunten

Samenwerking myGrid

Page 6: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

Workflow management Web 2.0 style registry: myExperiment

Lokale client: Taverna Workbench

Web client: project website

Page 7: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

Voordelen

Flexibel

Modulair

Transparant

Mogelijkheid tot uitbreiding

Kan gebruikt worden als: productieplatform, evaluatie framework, systeem voor digitale duurzaamheid – en nog veel meer!

Page 8: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

Evaluatie Bibliotheken selecteren afbeeldingen voor datasets

(toevallig & met focus op een collectie)

Ground truth wordt geproduceerd(100% correcte tekst en lay-out)

Vergelijking van workflows(verschillende combinaties van tools)

Page 9: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

Image Datasets Collectie van datasets per institutie, status November 2010:

- 534,193 afbeeldingen in totaal (met uniek ID)- 3.2 TB aan ruimte- Metadata

Page 10: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

Ground Truth Ground truth = 100% correcte transformatie van tekst en lay-out (coordinaten)

PAGE formaat specificatie

Ground truthing tools

Ground truthing richlijnen

Grote handmatige inspanning,niet triviaal!

Page 11: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

IMPACT Framework

Modulaire en transparente methode voor evaluatie van specifieke workflows

Page 12: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

Evaluatie: OCR Nauwkeurigheid

Page 13: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

GedeeltelijkgemistGemist

Samen-smelten

Gesplitst

Ground Truth

Segmentatie Resultaat

Mis-classi-ficatie

Paragraaf

(Onder)titel

Evaluatie: Layout en Segmentatie

Page 14: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.

Live Demonstratie

Workflow registry

Lokale client

Web client

Page 15: IMPACT Framework en Evaluatie by Clemens Neudecker

IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.