Post on 31-Mar-2015
Edwin A. ValentijnProf Astronomical Information Technology
Target- OmegaCEN – KapteynUniversity of Groningen
e-science platform 29 October2013Target
DATA FEDERATIONS
Probing Big data for Answers using Data about Data
Target
Dit project wordt medegefinancierd door het Europees Fonds voor Regionale Ontwikkeling en door het ministerie van EZ, Pieken in de Delta. Het project wordt mede mogelijk gemaakt door de gemeente Groningen, de provincie Groningen, de provincie Drenthe en SNN en staat onder auspiciën van Sensor Universe.
Data federationsdatacentric approach
• Federated dataservers• Our own• GPFS• GRID
• Federated databases• Oracle• Open source
• Distributed computing
Target
AstroWise -> Wise“living archives”
• Modeling dependencies• 64 bit identifyer• Terabytes of pointers• Extreme datalineage ++• Distributed-acces-proces-
calibrate- QC –analysis• KiDs
• 11 nodes-260 users• 2Pb, 45M file• 10 Giga table entries
• Web + Python interface
Target
Experimental Astronomy Volume 35 januari 2013Topical issue 19 papers 389 pages
Leiden
Groningen
Bonn
Munich
Padua
Naples
PotsdamNijmegen
Toulouse
WFAU
CASU
VO
LyonZürich
Distributed communitiesacces-proces-calibrate-analyse
Present KiDSo 50 -100 researchers ~10 labs
Future Euclid:o 1200 registered members and growingo 700 researchers o 130 laboratories/departments o 13 European countries contributing: Austria,
Denmark, France, Finland, Germany, Italy, Netherlands, Norway, Spain,Switzerland, Portugal, Romania, UK
o NASA/US: provides the IR detectors.o Canada: in negociation
LOWISE – LOFAR LTA
Target
EGEE Julich – SARA GRID ftp
5 Petabyte/year
EGEE Julich – SARA GRID ftp
5 Petabyte/year
MuseWISE
Target
ESO VLTIFS 24 IFU100 Gb/night
National Archive
Target
Integ
ratio
n
Lifelines – Healthcare165.000 persons- 30 years
LifeWise
AstroInformatics 2011
METABASE
MOLGENIS
Molecular
Care
Physiology
GCC
Target
IJKDIJKTNO -Target Holding - Dijkdatacenter
Target
Euclid Mission ArchiveTarget – ESA
data centric approach
Target
Euclid Archive Systemdata centric approach
Target
Metadata
RepositoryESAC + RUG