Death of the batch job

Click here to load reader

  • date post

    15-Apr-2017
  • Category

    Technology

  • view

    298
  • download

    0

Embed Size (px)

Transcript of Death of the batch job

  • Dennis van der Stelt

    all your batch jobs are belong to us

    Dennis van der Stelt

    Software Architect

    http://dennis.bloggingabout.net/

    dennis@bloggingabout.net

    Particular Software engineer

    death of the batch job

    @dvdstelt

    #nservicebus

    http://bloggingabout.net/blogs/dennis/

  • Dennis van der Stelt

    AGENDA

  • Dennis van der Stelt

    NServiceBusIn Particular

  • Dennis van der Stelt

    NServiceBus

    Its not like WCF, which does RPC

    But closer to WCF than to BizTalk

  • Dennis van der Stelt

    BUS TOPOLOGY

  • Dennis van der Stelt

    MessagingWhat is it and why do I need it?

  • Dennis van der Stelt

    SpatialTemporalPlatform

    coupling aspects

  • demo

    Quick NServiceBus Demo

  • Dennis van der Stelt

    NServiceBus SagasA pattern by relation database community

  • Dennis van der Stelt

  • Dennis van der Stelt

    PROCESS MANAGER

    process managerInitiating message

  • demo

    NServiceBus Sagas

  • Dennis van der Stelt

    Sagas RecapWhat have you done Within Temptation

  • Dennis van der Stelt

    HANDLING MESSAGES

    Behavior like normal message handlers

    class MySaga : IHandleMessages{public void Handle(MyMessage message){

    }}

  • Dennis van der Stelt

    STARTING SAGAS

    Extends the default IHandleMessages

    class MySaga : IAmStartedByMessages{public void Handle(MyMessage message){

    }}

  • Dennis van der Stelt

    STORING STATE

    Extends the default IHandleMessages

    class MySaga : Saga,IAmStartedByMessages

    {public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;

    }}

  • Dennis van der Stelt

    CORRELATING MESSAGES TO SAGA INSTANCE

    class MySaga : Saga,IAmStartedByMessages

    {protected override void ConfigureHowToFindSaga(SagaPropertyMapper mapper){mapper.ConfigureMapping(m => m.MyProperty).ToSaga(s => s.MyStateProperty);

    }

    public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;

    }}

  • Dennis van der Stelt

    REQUESTING TIMEOUTS

    Reminders to the Saga itself

    class MySaga : Saga,IAmStartedByMessages,IHandleTimeouts

    {public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;RequestTimeout(TimeSpan.FromSeconds(10));

    }

    public void Timeout(MyTimeout state){

    }}

  • Dennis van der Stelt

    SENDING MESSAGES

    Reminders to the Saga itself

    class MySaga : Saga,IAmStartedByMessages

    {public void Handle(MyMessage message){this.Saga.Data.MyStateProperty = Message.MyProperty;

    this.Bus.Send(new MyCommand());this.Bus.Publish(new MyEvent());

    }}

  • Dennis van der Stelt

    SAGA STATE

    Memento

    class MySagaData : IContainSagaData{public virtual Guid Id { get; set; }public virtual string Originator { get; set; }public virtual string OriginalMessageId { get; set; }

    [Unique]public virtual Guid MySagaId { get; set; }

    }

    ALTER TABLE [dbo].[MySagaData] ADD UNIQUE NONCLUSTERED ([MySagaId] ASC) ON [PRIMARY]

  • Dennis van der Stelt

    SAGA STATE

    Memento

    class MySagaData : IContainSagaData{public virtual Guid Id { get; set; }public virtual string Originator { get; set; }public virtual string OriginalMessageId { get; set; }

    [Unique]public virtual Guid MySagaId { get; set; }

    public virtual IList Products { get; set; }}

    public class Product{public virtual Guid ProductId { get; set; }

    }

  • Dennis van der Stelt

    SAGA STATE

    Memento

    class MySagaData : IContainSagaData{public virtual Guid Id { get; set; }public virtual string Originator { get; set; }public virtual string OriginalMessageId { get; set; }

    [Unique]public virtual Guid MySagaId { get; set; }

    public virtual IList Products { get; set; }}

    public class Product{public virtual Guid ProductId MyUniqueId { get; set; }

    }

  • Dennis van der Stelt

    Death to the batch jobBecause we dont want to depend on operations ;-)

  • Dennis van der Stelt

  • Dennis van der Stelt

    scheduled tasks

    Your CEO had insomnia and was using the system in the middle of the night. The batch job failed somewhere in

    the middle of updating 74 million records

    You need to figure out which row it failed on (how?), why it failed, correct the issue, then start the job again from where it left off, because if you have to start from the beginning, it won't get done before peak hours in

    the morning.

    because that sounds better than batch job

  • Dennis van der Stelt

    BATCH JOB

    All customers that ordered $5000 in the last year, get preferred status

    DateTime cutoff = DateTime.Today.AddDays(-365);

    foreach(var customer in customers){var orderTotal = customer.Orders.Where(o => o.OrderDate > cutoff).Sum(order => order.OrderValue);

    customer.Prefered = orderTotal > 5000;}

  • Dennis van der Stelt

    Troms, Norway

  • Dennis van der Stelt

    what if

    we can see things before they happen?

  • Dennis van der Stelt

    customer preferred status

    Dev: Let's say Steve orders something for $100. At what point does that amount no longer count toward Steve's preferred status?

    BA: That's easy, after 365 days!

  • Dennis van der Stelt

    -$300-$100

    +$300

    DURABLE TIMEOUTS

    Our way to predict the future

    2015 2016

    +$100

  • Dennis van der Stelt

    DURABLE TIMEOUTS

    public void Handle(OrderPlaced message){

    this.Data.CustomerId = message.CustomerId;

    this.Data.RunningTotal += message.Amount;this.RequestTimeout(TimeSpan.FromDays(365),

    timeout => timeout.Amount = message.Amount);

    CheckForPreferredStatusChange();}

    public void Handle(OrderExpired message){

    this.Data.RunningTotal -= message.Amount;CheckForPreferredStatusChange();

    }

  • Dennis van der Stelt

    DURABLE TIMEOUTS

    private void CheckForPreferredStatusChange(){

    if(this.Data.PreferredStatus == false && this.Data.RunningTotal >= 5000){

    this.Bus.Publish(evt => evt.CustomerId = this.Data.CustomerId);

    }else if(this.Data.PreferredStatus == true && this.Data.RunningTotal < 5000){

    this.Bus.Publish evt.CustomerId = this.Data.CustomerId);

    }}

  • Dennis van der Stelt

    Best PracticesThe silver bullets?

  • Dennis van der Stelt

    Rule #1 : Dont query data

    Never ever, ever, ever query data

    - From the saga to another data source

    - Owned by saga with a 3rd party tool

  • race conditions do not exist

  • Dennis van der Stelt

    STARTING SAGAS

    What business events can start a Saga?

    class MySaga : Saga,IAmStartedByMessages,IAmStartedByMessages,IAmStartedByMessages

    {public void Handle(MyMessage message){

    if (VerifyState())MarkAsComplete();

    }}

  • sagas bring agility

  • Dennis van der Stelt

    Pleasestay in touch!http://dennis.bloggingabout.net

    dennis@bloggingabout.net