Death of the batch job

Dennis van der Stelt

all your batch jobs are belong to us


Software Architect

http://dennis.bloggingabout.net/

[email protected]

Particular Software engineer

death of the batch job

@dvdstelt

#nservicebus

http://bloggingabout.net/blogs/dennis/


AGENDA


NServiceBusIn Particular


NServiceBus

It’s not like WCF, which does RPC

But closer to WCF than to BizTalk


BUS TOPOLOGY


MessagingWhat is it and why do I need it?


SpatialTemporalPlatform

coupling aspects

demo

Quick NServiceBus Demo


NServiceBus SagasA pattern by relation database community


PROCESS MANAGER

process managerInitiating message

demo

NServiceBus Sagas


Sagas Recap“What have you done” – Within Temptation


HANDLING MESSAGES

Behavior like normal message handlers

class MySaga : IHandleMessages<MyMessage>{public void Handle(MyMessage message){…

}}


STARTING SAGAS

Extends the default IHandleMessages<T>

class MySaga : IAmStartedByMessages<MyMessage>{public void Handle(MyMessage message){…

}}


STORING STATE

Extends the default IHandleMessages<T>

class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>

{public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;

}}


CORRELATING MESSAGES TO SAGA INSTANCE


{protected override void ConfigureHowToFindSaga(SagaPropertyMapper<MySagaData> mapper){mapper.ConfigureMapping<MyMessage>(m => m.MyProperty).ToSaga(s => s.MyStateProperty);

}

public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;

}}


REQUESTING TIMEOUTS

Reminders to the Saga itself

class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>,IHandleTimeouts<MyTimeout>

{public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;RequestTimeout<MyTimeout>(TimeSpan.FromSeconds(10));

}

public void Timeout(MyTimeout state){…

}}


SENDING MESSAGES

Reminders to the Saga itself


{public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;

this.Bus.Send(new MyCommand());this.Bus.Publish(new MyEvent());

}}


SAGA STATE

Memento

class MySagaData : IContainSagaData{public virtual Guid Id { get; set; }public virtual string Originator { get; set; }public virtual string OriginalMessageId { get; set; }

[Unique]public virtual Guid MySagaId { get; set; }

}

ALTER TABLE [dbo].[MySagaData] ADD UNIQUE NONCLUSTERED ([MySagaId] ASC) ON [PRIMARY]


SAGA STATE

Memento



public virtual IList<Product> Products { get; set; }}

public class Product{public virtual Guid ProductId { get; set; }

}


SAGA STATE

Memento



public virtual IList<Product> Products { get; set; }}

public class Product{public virtual Guid ProductId MyUniqueId { get; set; }

}


Death to the batch jobBecause we don’t want to depend on operations ;-)


scheduled tasks

Your CEO had insomnia and was using the system in the middle of the night. The batch job failed somewhere in

the middle of updating 74 million records…

You need to figure out which row it failed on (how?), why it failed, correct the issue, then start the job again from where it left off, because if you have to start from the beginning, it won't get done before peak hours in

the morning.

because that sounds better than batch job


BATCH JOB

All customers that ordered $5000 in the last year, get preferred status

DateTime cutoff = DateTime.Today.AddDays(-365);

foreach(var customer in customers){var orderTotal = customer.Orders.Where(o => o.OrderDate > cutoff).Sum(order => order.OrderValue);

customer.Prefered = orderTotal > 5000;}


Tromsø, Norway


what if

we can see things before they happen?


customer preferred status

Dev: Let's say Steve orders something for $100. At what point does that amount no longer count toward Steve's preferred status?

BA: That's easy, after 365 days!


-$300-$100

+$300

DURABLE TIMEOUTS

Our way to predict the future

2015 2016

+$100


DURABLE TIMEOUTS

public void Handle(OrderPlaced message){

this.Data.CustomerId = message.CustomerId;

this.Data.RunningTotal += message.Amount;this.RequestTimeout<OrderExpired>(TimeSpan.FromDays(365),

timeout => timeout.Amount = message.Amount);

CheckForPreferredStatusChange();}

public void Handle(OrderExpired message){

this.Data.RunningTotal -= message.Amount;CheckForPreferredStatusChange();

}


DURABLE TIMEOUTS

private void CheckForPreferredStatusChange(){

if(this.Data.PreferredStatus == false && this.Data.RunningTotal >= 5000){

this.Bus.Publish<CustomerHasBecomePreferred>(evt => evt.CustomerId = this.Data.CustomerId);

}else if(this.Data.PreferredStatus == true && this.Data.RunningTotal < 5000){

this.Bus.Publish<CustomerHasBecomeNonPreferred(evt => evt.CustomerId = this.Data.CustomerId);

}}


Best PracticesThe silver bullets?


Rule #1 : Don’t query data

Never ever, ever, ever query data

- From the saga to another data source

- Owned by saga with a 3rd party tool

race conditions do not exist


STARTING SAGAS

What business events can start a Saga?

class MySaga : Saga<MySagaData>,IAmStartedByMessages<MyMessage>,IAmStartedByMessages<YourMessage>,IAmStartedByMessages<AnotherMesssage>

{public void Handle(MyMessage message){…

if (VerifyState())MarkAsComplete();

}}

sagas bring agility


Please…stay in touch!http://dennis.bloggingabout.net

[email protected]

Death of the batch job

Technology

Transcript of Death of the batch job