MonetDB spatial Onderzoeksinstituut OTB 2007-10-16 1 MonetDB spatial Een open-source Spatial DBMS...

2007-10-16

MonetDB spatial

1

Onderzoeksinstituut OTB

MonetDB spatial

Een open-source Spatial DBMS

Media Plaza/Het gebruik van OS software in GEO-ICT

Wilko Quak

2007-10-16 2MonetDB spatial


Overzicht

• TUDelft / OTB / Sectie GISt / GeoInfoNed Project• De spatial DBMS markt.• Categorie-en van DBMS gebruikers• Intermezzo: spatial DBMS.• MonetDB en MonetDB (spatial)• De toekomst van MonetDB (spatial)



GIS technology



GISt visie

Spatial Information Infrastructure (SII), a vision…• Heading towards an information society with

varying impact on sectors of our economy• Impact on the geo-information sector is huge: geo-

info and processing can be transported via (wireless) networks and can be combined with positioning

• Engines of the SII nodes: geo-DBMSs (Data Base Management Systems) filled with geo-data

• Fair pricing of data and services, no more endless copying of data (and loss of quality/up-to-dateness)



GISt: scientific/technological goal

• Central research theme geo-DBMS as ‘glue’ between:

• 3D spatio-temporal modeling• Computational geometry• Distributed GI processing• Mobile GIS (LBS)• Knowledge engineering

Ambition: top 1 (3) of geo-DBMS Universities in the world



Geo Database Management Center• TU Delft open R&D center ‘geo-DBMS’• Partnerships with ICT industry: Oracle, ESRI,

Bentley• ‘Database lab’ and ‘3D GIS/VR lab’• Website http://www.gdmc.nl for

• Communication/documentation

• Internet GIS



Met welk onderzoek kan de Sectie GISt bijdragen?

• Onderzoek naar bruikbaarheid bestaande producten (benchmarken)

• Wat mist er nog• Meedenken met ontwikkelingen• Eigen DBMS ontwikkelen als research vehikel:

GeoInfoNed Project



GeoInfoNed – RGI-232



wie

• CWI – Leading DBMS experts with MonetDB

• TUDelft/OTB – Knowledge of spatial processes

• CycloMedia – Huge dataset and interesting problems

• RWS/AGI – Large and diverse datasets and interesting problems



Wat

• Is er nog wel ruimte op de markt? Een inventarisatie:• Aanbieders• Gebruikers

We gaan een spatial DBMS bouwen en daar onderzoek mee doen.



De spatial DBMS markt (aanbieders)

• Oracle – Marktleider commerciele segment• Informix/DB2 – Prima implemenatie. Niet erg

actief• PostgreSQL/PostGIS – ‘Marktleider’ open-source• SQLServer – komt binnenkort• MySQL – open-source, beperkte functionaliteit



Spatial DBMS markt (gebuikers)

4 classes:1. Server Builders: publish spatial data via web

server2. GIS User: Load various datasets and perform

complex analyses3. Data Maintainer: Maintain one core dataset4. Power Users: All of the above and more



Class 1: Web Server Builders

• Runs under GeoServer• You do not really need a DBMS for this (You

use a fraction of DBMS functionality)• Only one query counts: Find everything within

BBOX.• Multi-User• Small Queries



Class 2: GIS users

• Full fledges GIS is much more powerful but SQL experts might be tempted

• Main interest is functionality• Spend more time on loading data• Need a good query optimizer• Data mining



Class 3: Dataset Maintainers

• Limited number of queries.• 24/7 stability• Transactions.• Clustering of data after updates is interesting.



Class 4: Power users

• All of the above and more• Take time to understand software• Run into research problems:

• Point clouds from laser-scanning• Support 3d-volumes• Realtime dynamic data



Intermezzo: Spatial DBMS

• Spatial data types• Operaties op types• Indexeren op types



Spatial Data Types

• Basic types: Point Line Polygon

• More Complex: 3D, Geodetic Coordinates



Spatial Data types volgens OpenGIS

PointLineStringLinearRingPolygonMultiPointMultiLineStringMultiPolygon



Functions op spatial types



• R-tree puts MBR on object- MBR : minimum bounding rectangle- Object can be of type point, line or region- At lowest (leaf) level objects are grouped

to larger MBR’s, which can be grouped (and so on .....)

Indexeren spatial types

D

F

E

G

H

J

K

A

C

L

N

MB

I

A B C

D E F G H I J K L M N

Branching factor 4

Einde Intermezzo



MonetDB Introduction*

MonetDB is opens• Hardware trends• MonetDB design considerations• MonetDB architecture

*Slides borrowed from CWI

2007-10-16 MonetDB spatial


Hardware Trends50% p/year:

- cpu speed

- mem size

- mem bandwidth

- disk bandwidth1% p/year:

- mem latency

10% p/year:

- disk latency

2007-10-16 MonetDB spatial


Latency is the enemy!

• Commercial DBMS products (oracle, DB2, SQLserver) stem from OLTP roots

• focus on minimizing random I/Os => depend on latency!

• MonetDB: built for bulk access• optimize CPU and memory performance



MonetDB design considerations

• Multi-model database kernel support• Extensible data types, operators, accelerators• Database hot-set is memory resident• Simple data structures are better• Index management should be automatic• Do not replicate the operating system• Optimize when you know the situation• Cooperative transaction management



Monetkernels

MAPI protocol

JDBC

C-mapi lib

Perl

End-user application

ODBC PHP Python

SQL XQuery

MonetDB product family



SWOT

State-of-the-art SQL & XQuery DBMS; multi-model database support; decomposed storage model; query processing for decision support systems; extensible kernel; multiple APIs;…..

strengthTriggers and SQL procs missing; XQuery; some overhead in simple SQL queries; MIL scripting; missing GIS module

weakness

Commercial systems requirements; competition established market; floating interest; development skills and quality awareness

threats

Unique selling points for advanced applications in multimedia, streaming, covering both small and the large world

opportunities

GET INSPIRED BY ADVANCED APPLICATIONS



MonetDB - Physical data organization• Binary Association Tables

ID Day Discount10 4/4/98 0.19511 9/4/98 0.06512 1/2/98 0.17513 7/2/98 0

OID ID100 10101 11102 12103 13104 14

OID Day100 4/4/98101 9/4/98102 1/2/98103 7/2/98104 1/2/99

OID Discount100 0.195101 0.065102 0.175103 0104 0.065



Spatial MonetDB



Spatial MonetDB -- nu

• Eerste implementatie in volgende release (Q4).• OpenGIS implementatie gebaseerd op PostGIS

• OpenJUMP plugin• Shape2MonetDB converter • GeoTools Plugin (in ontwikkeling)



Spatial MonetDB – de toekomst

• MonetDB moet het hebben van zijn speciale architectuur.

• Uitdagingen:• Puntenwolken van Laserscanner.• Data-minen op grote spatio-temporele

datasets.• XML-Data.

• Saaier werk:• Testen huidige implementatie• debuggen



Vragen

Discussie

MonetDB spatial Onderzoeksinstituut OTB 2007-10-16 1 MonetDB spatial Een open-source Spatial DBMS...

Documents

Transcript of MonetDB spatial Onderzoeksinstituut OTB 2007-10-16 1 MonetDB spatial Een open-source Spatial DBMS...