B2B Integration in the Cloud

1

B2B-integrationin the ‘cloud’

[email protected]@i8c.be

2

Get acquainted withB2B integration

Overall presentation goal

3

AbstractGedreven door nieuwe business processen en door regulering neemt de behoefte voor gegevensuitwisseling tussen organisaties hand over hand toe. Als 'trading partners' wisselen bedrijven massaal gegevens uit onder de vorm van elektronische berichten of 'events'. De automatisering van deze B2B-communicatie kent zeer belangrijke verschillen ten overstaan van de integratie door middel van 'enterprise service buses' binnen een organisatie.Tijdens deze avondconferentie verkrijgt U inzicht in de verschillende protocollen en berichtformaten die terzake worden toegepast, gaande van file transfer en EDI tot AS2 met allerlei XML-dialecten. Ook beveiliging en integratie met 'eindsystemen' wordt uitvoerig behandeld.De markt van B2B-integratie-oplossingen is overigens in volle evolutie. En met de komst van 'cloud computing' zal de wereld van B2B-communicatie nog drastisch wijzigen. Communicatie via de cloud, en met partners in de cloud, vormt een belangrijk toekomstperspectief !De sprekers geven de nodige achtergrond en leiden U, in één avond, door de boeiende evoluerende wereld van B2B-integratie al dan niet via cloud computing.

4

• Integration consultancy• ESB, SOA, BPMS, B2B, EAI, Composite Apps• Vendor independent• 33 consultants• Part of the Cronos group• http://www.i8c.be

Who are we?

5

The “waves”• 1st: EDI and closed networks• 2nd: From 2000 onwards: Internet & XML• 3rd: 2010: Cloud & Integration-As-A-Service

Picture by Mila Zinkova

6

Who to integrate?

SUPPLIERS

CUSTOMERS

GOVERNMENT

FINANCIAL INSTITUTIONS

3rd Party Hub

7

B2Bi - B2B integration• Communication From tapes & floppies

– Bisync, X25 and ISDN– To the Internet: FTP, SFTP, EDIINT AS2, ...– (Almost) Always event-driven, asynchronous

• Via intermediary (ASP) or point-2-point– Intermediary: VAN, TDN (ASP, ISP) to Cloud

• Message format– From EDIFACT, ANSI X12, HL7, SWIFT, TRADACOM, ..– To XML: UBL, OAG, CIDX, ...

• Choreography: sequence of messages allowed+Technical (n)ack, Functional (n)ack

8

The 1st wave

OrganizationA

OrganizationB

ValueAdded

NetworkLOC+147+0090305::5’MEA+WT++KGM:22500’LOC+9+NLRTM’LOC+11+SGSIN’RFF+BM+933’EQD+CN+ABCU2334536’NAD+CA+ABC:172:20’

9

Value Added Network• Routing (but nor format translation)

– Each party has “mailbox”• Different transmission methods

– Phone line, ISDN, Bisync, X.25, FTP, OFTP, X.400 mailbox, FTP, WebSphereMQ (IBM), ..

• Trusted Third Party, e.g. audit information• Interconnect to other VANs• No translation or transformations

10

EDI Message Standards

Message standards• UN/EDIFACT• ANSI X12• TRADACOMS• ODETTE• Industry Subsets

Considerations• Every year new

versions of EDI messages

• EDI translator tools were quite expensive

11

X12 & EDIFACT

12EA

I

FTP AdapterVAN #2

VAN #2

X400X400

VAN setup

Back-endApplication

VANVAN

File (CSV)

VAN connector

MailDial up

Appl. adapter

Transform

FTPVPN

Back-endApplication

TranslatorEDI

13

EDI

FORTUNE 1000

EDI95%

5% EDI2%

98%

SME’s

• UN/EDIFACT & X.12 are a success!• Implementation is cumbersome (e.g. VAN)• OK for large organizations, not for SME’s• Rip-and-read-processing

14

EDI Learned Lessons• Business semantics – stable standards

– Standard transactions, messages and data elements e.g. UN/EDIFACT Directories (message building blocks)– Common, inter-changeable data elementBUT not specific enough:- Message Implementation Guidelines document exact usage of the

messages• Acknowledgement

– Technical ack: means syntactically OK (CONTRL)– Functional ack: receiver received and understood message (APERAK)

• Security: Value Added Network – Trusted 3rd party

15

EDI: Codes & Numbers• Unique identification

– Message: duplicate detection, electronic audit trail (message tracking)

– Party: e.g. DUNS (Dun & Bradstreet)

• Currency Codes• Measurment codes• Country codes• Location codes• ...

16

EDI: TPA & MIG• Trading Partner Agreements (TPA)

• Message Implementation Guidelines (MIG)

17

2nd Wave – Internet & XML• Decline of the VANs

– Direct communication– But also upcome of the TDN’s, ASP’s, ...

• XML as an alternative message format– Standardization of XML message formats

• New protocols (on top of TCP/IP)

– HTTP: Web Services, EDIINT AS2, POX, RosettaNet RNIF, ebXML ebMS, ...

– File transfer: FTP, SFTP, FTPS– Mail: SMTP, AS1

18

B2B Message Formats

19

And more B2B XML standards...

20

Too many B2B XML standards• “Long tail of B2B standards”...

– EDI handles diversity with EDI Message Implementation Guidelines

21

“New” B2B Protocols• Internet & TCP/IP de facto standard

– Decline of X.25, X.400, Bisync, ISDN, ...• HTTP

– XML over HTTP (Plain Old XML)– ebMS – ebXML Messaging, RosettaNet RNIF– AS2

• File Transfer: FTP, SFTP, FTPS, OFTP, OFTP2, AS3• Mail: SMTP, AS1• Very limited use of Web Services and SOAP!

22

EDIINT• EDI over the INTernet

– Initiative within IETF• Apply existing IETF Standards

– Adds retry, security (S/MIME), acknowledgements– Any payload: EDI, XML, binary

• Applicability Statement– EDIINT/AS1: over email– EDIINT/AS2: over HTTP– EDIINT/AS3: over FTP

RFC 1123 Requirements for Internet HostsRFC 1767 MIME Encapsulation of EDI ObjectsRFC 1847 Security Multiparts for MIMERFC 2045 MIME Format of Internet Message BodiesRFC 2046 MIME Media TypesRFC 2049 MIME Conformance Criteria and ExamplesRFC 2298 An Extensible Message Format for Message

Disposition NotificationsRFC 2616 Hypertext Transfer Protocol -- HTTP/1.1RFC 2821 Simple Mail Transfer Protocol (SMTP)RFC 2822 Standard for the Format of Internet Text MessagesRFC 3370 Cryptographic Message Syntax (CMS) AlgorithmsRFC 3798 Message Disposition NotificationRFC 3851 S/MIME Version 3.1 Message SpecificationRFC 3852 Cryptographic Message Syntax

23

AS2 & SFTP• EDIINT AS2

Message

AS2Server

AS2Server

Message

SFTPServer

SFTPClient

PUT File

GET File

Poll FilePoll File

• SFTP: file transfer over SSH (secure shell)FTPS: FTP over SSL/TLS not very popular

24

AS2 & SFTP Products

25

MDN

Message Level Security

Sender

Sign

Priv

ate

Receiver

Verif

y Publ

ic

Dec

rypt

Priv

ate

Verif

yPu

blic

Sign

Priv

ate

Trust Certificate

Trust Certificate

Encr

ypt

Publ

ic S/MIME(PKCS#7)

Authenticatesender

Over unsecure protocols: HTTP, SMTP, ...For non-repudiation: cannot deny sending/receiving the message

26

• VPN• Fixed links, e.g. MPLS

Network Security

• IP filtering – Fixed IP addresses

Rev Proxy

Rev.Proxy

101.43.55.26101.43.55.28

www.cronos.be/b2b113.62.15.55

• Via intermediary: TDN (VAN)

HubISP/TDN/VAN

Poll Poll

27

Upcome of the ISP • The VAN reinvented: ISP

– Integration Service Provider (ISP)– TDN: Transaction Delivery Network (~ ASP)

• Support for Internet protocols• ISP: platform & consultancy services

– Integration setup:• Analyse, design, develop, test integration scenario’s• You don’t get to touch “their” environment

– Partner Onboarding: Take responsibility connecting other parties

– Specific vertical solutions: closed groups, telco, VMI, ...

28

ISP - Return of the VAN

29

Market consolidation• Acquisitions

– Sterling Commerce, ViaCore IBM– ADX & SoftShare Liaison

• Mergers– Inovis/GXS merger

• Partnerships– IBM partners with Hubspan– Microsoft partners with GXS– Oracle partners with E2Open and SPS Commerce– SAP Partners with Crossgate

30

3rd wave: Cloud Computing

31

nfrastructureIntegration

What is cloud computing?

AASIAs-A-Service

Abstraction of the hardwareDon’t care where those servers are

32


AASlatform

PAs-A-ServiceAbstraction of the software

Don’t care: about OS, #servers, … Build or deploy my application on provided platform

33


AASoftwareS

As-A-Service

Storage

SAAS: Abstraction of everything Just want to use the application

34

The Cloud stack

Infrastructure

Platform

Software

Desktop

Storage

35

Some SAAS providers• Billing: • CRM:• Collaboration: • ERP:• HR:

• Product & Price configuration:• Business Intelligence:

36

3rd wave: the Cloud• Connecting to the cloud

• Connecting through the cloud

Integration

37

Salesforce to elsewhere• Outbound messaging

– Async– Secure

• HTTPS• Supports X509 Client

Certificates– Reliable, 24h retry– Only notifications

• Info about objects that have changed

Internet

Outbound Messaging

Account

Contact

Opportunity

NativeSalesforce Object

Web Service

Endpoint

SOAP

api

Enterprise customers: Salesforce connectors for SAP and OracleEarlier API version: only polling

38

Cloud specific adapters• Tibco Salesforce Plug-in

• WebMethods Salesforce Adapter

39

Linux

Google App Engine to on-premise

“SDCServer” SSL

Username/password ResourceRules

PublicResource

on Intranet

PrivateResource

on IntranetGoogleAppEngine

Application

“Reverse Invoke”SecureData

Connector

EnterpriseGoogle

SignedFetch

Fetch

40

Cloud as DMZ

• Use cloud infrastructure as “DMZ”

B2BServer

AS2

BizTalkOrchestration

& Adapter

RelayService

WS

• Azure AppFabric Connect

41

Cloud as DMZ• BizTalk Server 2010 AppFabric Connect for Services

• WSO2 Cloud Services Gateway

42

File & message exchange• File-Transfer-As-A-Service

– No focus on B2B but on either• Backup Single user/enterprise• Proprietary API’s or Tools

– No SFTP-As-A-Service– No AS2-As-A-Service

• Queuing As A Service: – Will AMQP standard be adopted?– Not ready for prime time

43

API’s• Web Services: SOAP, REST

44

3rd wave: the CloudOn premise Cloud

Com

pany

ACo

mpa

ny B

45

Babelway

• Belgian company, founded in 2007– http://www.babelway.com

• B2B Integration SaaS [multi-tenant]

• GOAL: simplify B2B data exchange– No system installation and maintenance– No B2B software installation and maintenance– Easy operations, easy conf. of connections– 1 to many.

46

Babelway – Solution Overview• 3 Main Functions

– Transport of electronic messages between 2 partners

– Transformation of messages from an input to an output format

– (Opt.) Storage of messages for a defined period of time.

• 2 Technical Parts

Message Processing Engine

Web GUI

47

Babelway – Configuration - Channel

• Channel = automatic data flow process between– Your system – Ext. System of B2B Partner

• • Incoming messages can be routed to a

particular channel in function of content or metadata

48

Babelway – Configuration - Catalogue

51

Babelway –Transformation

• Data validation• Data mapping

• Drag and drop (choices, loops, ...)• XSLT (+ create own functions)• Java code

• Lookup tables (+ auto. population)• Message signing• PDF creation

52

Babelway – Admin. & Operations• Testing - test cases can be created and run to

validate channel operations before deployment• Message Tracking• Tickets

– Generated by multiple events– Messages can be manually corrected and

resubmitted• Management – user account administration,

notifications,...• Rest API – data interface

53

Babelway – System Architecture• Amazon AWS EC2 and S3 Services (I-a-a-S)

54

Boomi – http://www.boomi.com• Founded in 2000, acquired in 2010 by Dell

"Integration is a key element in Dell's future transformation into software and services," says Ray Wang, principal Analyst and chief executive officer at Constellation Research. "Companies like Dell, HP, and Cisco will need to be in software in order to grow."

• Connects any combination of cloud, SaaS and on-premise applications with no appliances, no software, and no coding

• Software-as-a-Service platform– No hw/sw to install or maintain, auto. upgrades– Accessible via any internet browser– Multi-tenant architecture

55

Boomi – Cloud Integration• Salesforce CRM• NetSuite Leading vendor of cloud computing business management

software suites. (ERP, CRM, Accounting, ...)• Taleo Talent Management and Recruitment Software

• Peachtree Accounting and Financial Software

• QuickBook Small Business Accounting Software

• MS Dynamics GP ERP

• Zuora online recurring billing and payment solutions for SaaS and subscription businesses

• RightNow Cloud-Based Web Customer Services

• Freshbooks Online Invoicing, Time Tracking & Billing Software

• SuccessFactors Business Execution Software• ...

On premise• Oracle E-Business

Suite• PeopleSoft• SAP ERP• Siebel

56

Boomi - Connection ClassesADP Employease, ADP Perspective, ADP HRB, Aprimo Marketing Studio, AS400, Autotask, AtTask, Centive, Coupa, DB2, Disk, ExactTarget, FileMakerPro, FTP, FTPS, SFTP, Fishbowl, Google App Engine, Host Analytics, HTTP, HTTPS, Innotas, Intacct, Interspire, Intuit Partner Platform, JD Edwards v.7 Lotus Notes & Domino, MarketBright, Marketo MAS 200, MAS 500, MAS 90, Microsoft Dynamics CRM Versions 4x & higher, Microsoft Dynamics CRM Online, Microsoft Dynamics-Great Plains v.9, v.10, Microsoft SBF, MySQL, NetSuite Oco, Oracle CRM On Demand/ Siebel CRM On Demand, Oracle Database, OpSource Billing, Parature , Pivotlink, POP/SMTP, PostGres, QuickArrow, QuickBase , RedPraire On Demand WMS , RightNow, Salesforce-Pro/Ent/Unltd, Silverpop, SmartTurn, SpringCM, SQL Server, StrikeIron, SuccessFactors, Sugar CRM (On Site or On Demand) v.5.5, Taleo Business Edition, Twitter, Web Services, WorldAPP KeySurvey, Yahoo! Stores, Zuora

57

• Main component in Boomi integration• Represents an interface between 2+ systems on

business proc. level• Composed of shapes

connected like a flow chart

• Executions can be event-based or schedule driven– (complex) scheduler– API to incl. event-driven integr. into your process

Boomi AtomSphere – Process

58

• End-points of a process Get data in or send data out

• Enable communication with applications and data sources– On-premise applications - e.g. SAP– Cloud/Web-based applications – e.g. Salesforce.com– Data repositories – e.g. FTP, DB or an email server

• Accomplish integration by abstracting the technical details of APIs and transportation protocols.

Boomi AtomSphere - Connectors

59

Boomi – The Boomi Atom (1) ...• Lightweight dynamic runtime engine• Can be deployed

– On a local machine in your network (on-premise)• access to internal resources or applications (behind the

company’s firewall)• solves On-Premise-to-SaaS integration problem

– In the Boomi AtomSphere cloud • connect with resources or applications accessible via the

internet (e.g. web applications or FTP sites)• all computing is performed in the Boomi data center

> No software or hardware needs to be installed.• for SaaS-to-SaaS integration

60

• Contains all the components required to execute an integr. proc. from end to end (incl. connectors, transformation rules, …)

• Monitoring - Status and health of “all atoms” and integration processes Dashboard

• Provides secure access to on-premise application(s) when deployed behind firewall

Boomi - ... The Boomi Atom (2)

HTTPS

-128 bit encr.

61

1. Build: ...– Visual Integration Technology (point-and-click, drag-and-drop)

– Library of pre-built connectors and process maps

Boomi – Integrate in 3 steps – 1. Build

62

2. and Deploy integration process

Boomi – Integrate in 3 steps – 2. Deploy

Atom Name

Atom

63

3. Manage all integrations from the AtomSphere platform.

Boomi – Integrate in 3 steps – 3. Manage

Atom NameAtom Name

64

• Version control offers rollback of integration processes

• Concept of environments is provided

• Molecules - (Ent. Ed.) Atom version which provides– High-Availability - through Fault-Tolerance– High Performance - through intelligent Load-Balancing

Boomi – Extra Features

65

• • Founded in 2006 (co-founder and CEO G. Dhillon aka founder of

Informatica Corp.)

• Data integration platform, allows connectivity and robust transformations to web, SaaS, and enterprise data

• No programming•

SnapLogic – http://www.snaplogic.com

66

• SL Server – Execution engine (security, logging, runtime, ...)

• for complex data transfer and transformation tasks• pipelines are published

– Can be installed • Local

– Linux, Windows, Mac OS X• In the cloud

– Rackspace, Amazon Web Service, – Private cloud

SnapLogic – Server

67

• SL Designer– Browser-based visual configuration tool (allowing drag-

and-drop functionality)– Create and configure workflow pipelines connecting any

system (no need for custom code)

SnapLogic - Designer

68

• Orchestration of Snaps that solve an integration path from end to end

• Snaps are connected together in the Designer• Can be scheduled, triggered on-event or on-

demand• Addressable by a simple REST URL

SnapLogic - Pipeline

69

• SL SnapStore - http://store.snaplogic.com/– Online store for snaps ( sw delivery model)– Snap

• Certified connector plug-in for the SnapLogic Platform• Interfaces with applications to easily perform data

operations without any custom code• Created by developers ( > SnapLogic API (SnAPI) and SDK)

– for private use– to place for sale and download from the SnapStore

• Cost-effective alternative to custom hand-coding

SnapLogic – SnapStore

AND/OR

http://store.snaplogic.com/

70

• SL SnapCenter– Enterprise level clustering solution– Allows easy conf. for 1+ SL servers to scale up

parallel pipeline processing• Users build, manage and execute pipelines as on a

single SL server• Cluster manages execution of pipeline behind the

scenes in distributed env.• Higher throughput, higher # of integration tasks

– Console’s tabs: overview, pipeline & server, events and history

SnapLogic – Architecture - SnapCenter

71

SnapLogic – RESTful Architecture

72

http://sl***.dyn.snaplogic.com:8088/ __snap__/__static__/designer/SnapClient.html

SnapLogic – Live TrialFree trial Snaps

• Box.net• Google Analytics• Google Checkout• Jiggsaw• Netsuite• Peoplesoft• Quickbooks• Salesforce• Twitter• YouTube• ...

73

• Scalability– Handle data flows from any number of sources to any number of

destinations– Performance monitoring and load balancing (clustered SnapLogic Servers)

• Extensibility– Snaps and SnapStore => limitless connectors available

• Usability– Browser-based UI– No coding needed– Read and write from any hosted or on-premise data source

• Versatility– SL can be deployed on the desktop, in the cloud (public or private), or on-

premise. Deployment model supports developing on desktop, testing on inexpensive

computing resources and deploying on highly scalable public/private clouds

SnapLogic - Advantages

74

• - acquired by IBM • WebSphere Cast Iron Cloud Integration• Pre-configured integration solutions for

hundreds of leading enterprise and cloud applications

• “Configuration, not coding" approach

Cast Iron

75

Cast Iron: does 4 things1. Connectivity - Connect to ent. apps, on-

demand apps, databases, flat-files etc.2. Transformation - graphically map data

between source and target applications3. Business Logic (Workflow) - graphically define

the flow of data between source and target applications

4. Management- one console manages your integration, and have complete visibility to data flowing thru the appliance

76

• 2 deployment models:– in Cast Iron Cloud I-a-a-S– on Cast Iron integr. appliances

Integr. On-Premise• WebSphere DataPower Cast Iron Appliance XH40:

physical appliance to add to on-premise datacenter• WebSphere Cast Iron Hypervisor Edition: virtual

appliance that can be installed on own hardware

Cast Iron – Product Solutions

77

Cloud Apps and Platforms• BigMachines

Product Configurator, B2B eCommerce, Quoting and Proposal Software

• GoogleApps• MS Dynamics CRM• NetSuite• Oracle CRM On Demand• PROS

World Leader in Pricing and Revenue Management Software

• RightNow• Salesforce.com• SuccessFactors• SPS Commerce

Supply Chain Solutions

• Taleo

On Premise• Agile• CA Clarity• JD Edwards• Lawson• Microsoft Dynamics• Oracle EBS• PeopleSoft• Siebel• SAP

78

• Cast Iron Studio

Cast Iron - Development

79

Hubspan

• Example of Intergration Service Provider that pretends to be Integration As A Service

• Why not IAAS? No “self service”

80

Data Integration in the Cloud• Informatica

• Pervasive– Data Integrator available in the cloud

81

More integration in the cloud• RunMyProcess

– BPMS in the Cloud with integration options• Meddius

– Healthcase specific• Workday SAAS

– Enterprise Interface Builder (Based on CapeClear ESB which was acquired by Workday)

– Packaged Integrations• Celigo

– Focus on integration with NetSuite • ...

82

Microsoft IAAS announcedAlso this week, we disclosed an early peek into our strategy of “Integration as a Service” which begins to shed light on how we will be taking the integration workload to the cloud. This is a transition we have already made with Windows Server and SQL Server (as we have released Azure flavors of these server products); and we are committed to following this

same path with integration.

Our plans to deliver a true Integration service – a multi-tenant, highly scalable cloud service built on AppFabric and running on Windows Azure – will be an important and game changing step for BizTalk Server, giving customers a way to consume integration easily without having to deploy extensive infrastructure and systems integration.

83

Microsoft IAAS announced

84

Summary• Integration As A Service

– Very promising• Interesting new players + market turmoil

– Mix of in the cloud & on-premise• Software and Hardware appliances

– Self-service (but someone’s got to do it)

1st wave: EDI & VANOld technologies aren’t gone yet, e.g. EDI

2nd wave: XML over InternetAS2, SFTP, …With or without Integration Service ProviderMany XML languages, too many XML dialects

3rd wave: Integration As A Service

85

Thoughts...

Oracle8i , Oracle9i, Oracle10g, Oracle11g, Oracle12c ?

Cloud: re-invention of time-sharing?IAAS: re-invention of the VAN?

Chief Cloud Officer?

Integration-As-A-Service market: turmoil– Big players to enter the market: Microsoft, ... – Acquisitions, mergers, takeovers...– Which vendors (ISP, integration) will reinvent

themselves

86

Hype and reality

87

Questions

B2B Integration in the Cloud

Technology

Transcript of B2B Integration in the Cloud