Skip to content

Episode 29 – Acting Like We Get The Message

2013 May 15
by Kerry Street
Share

Kerry and Rinat introduce the Actor in the Actor Model of Computation. They wonder if the Actor’s embodiment of communication (via messaging) may simplify the way that they reason about and implement their solutions. They discuss this potential use of the Actor Model in the context of their current usage of Domain-Driven Design (DDD), Application Services, Aggregates, and Event Sourcing.

Download (mp3): Episode 29 – Acting Like We Get The Message – (63 minutes)

Subscribe via RSS | Subscribe for free in iTunes

Episode References:

What do you think?

25 Responses Post a comment
  1. Daniel permalink
    May 16, 2013

    A great explanation of the actor model implementation in Akka was made in a recent interview with Victor Klang, the lead engineer of it:
    http://highscalability.com/blog/2013/5/8/typesafe-interview-scala-akka-is-an-iaas-for-your-process-ar.html

    • Kerry Street permalink*
      May 16, 2013

      Thanks Daniel, it looks good. I will check that out.

  2. May 16, 2013

    Really interesting episode again! I think I have to listen to it again to catch all of it. I made a very similar experience as Rinat when I played around with a toy implementation of view projections. Placing the projections them self in charge of loading events is a huge simplification, although it probably means a lot of duplicate reads (cheap anyway).

    • Kerry Street permalink*
      May 16, 2013

      I plan on listening to it a few more times myself to catch it all :)

  3. Daniel permalink
    May 16, 2013

    Regards,

    I made two concept maps of the CQRS+ES frameworks presented in this podcast. I am trying to contrast the new proposed concepts with old ones.

    https://dl.dropboxusercontent.com/u/23333599/OldConceptMapCQRSES.png
    https://dl.dropboxusercontent.com/u/23333599/NewConceptMapCQRSES.png

    Please tell me if there is any misconception.

    I have some doubts about the new framework:

    * From the conceptual point of view the EventStore can be viewed as an Actor that doesn’t project the events it receives, it simply stores them in some Stream.

    * If a Projection is an Actor, then it receives a EventBatches as Messages from an internal Event Publisher, is this correct?.

    * Why not treat an ApplicationService as an Actor?.

    * Correct me if I am wrong here: An Stream is like a linked-list of Events, where each Event is tagged with an StreamID (that is derived from the AggregateID) and some other properties like a timestamp or a sequence number. The tags can be used to filter the Events when a query is processed.

    * You talk about direct communication between Aggregates (they are Actors in the new framework). You mention an example in the Registration Process. In this example the Registration Actor and the User Actor interact with each other to complete the process. At the end of the process the Registration Actor sends a RegistrationCompletedEvent to the EventStore. This means there are some intermediate commands and events that the EventStore is unaware of. I see some benefits of using a Receptor to communicate Aggregates because it decouples them and it forces us to send all events through the pipeline (EventStore, Publisher, Receptor, AppService). It can be more tedious to wire up all, but this way the EventStore is aware of every significant Event in the system.

    I appreciate the replies!

    • Kerry Street permalink*
      May 16, 2013

      I haven’t read all of this yet Daniel, but just an FYI: Rinat is on vacation and traveling right now so it will be a few weeks before he is back and settled in with a chance to review this. I will point it out to him when he gets back so that he can respond. Thanks for the participation!

  4. May 16, 2013

    Found this one about actors on reddit today. I don’t know enough about actors to judge whether it’s accurate, but it’s pretty clever with the cartoons :-)

    http://adit.io/posts/2013-05-15-Locks,-Actors,-And-STM-In-Pictures.html

    • Daniel permalink
      May 16, 2013

      All people have to go to the critical section at least once at day. xD

      Nice link!

    • Kerry Street permalink*
      May 16, 2013

      What an awesome post that is. Great link! Thanks Kim.

  5. May 16, 2013

    Interesting and thought provoking as always. I thought I would throw my two cents in regards to resources around actors, one is an excellent read at http://c2.com/cgi/wiki?TransactionalActorModel and the other is Microsofts Actor work on the subject at http://actorfx.codeplex.com there you will find a link at channel 9 about their plans for this model on the cloud. For those who really enjoy reading code it’s worth the read. I can not wait to see what Rinat comes up with.

    Thanks,

    Jason Wyglendowski

  6. Johan permalink
    May 19, 2013

    Hi

    Thanks for a good episode. I’d be glad if someone could help clarify a few things though. If I’m not mistaken you mentioned that prior to actors the different steps (aggregates) involved in for example a user registration process published events. These were picked up by event handlers which published a command to other aggregates. Did I understand it correctly that you said that by using actors you no longer publish an event but rather send commands directly to another aggregate/actor? If so wouldn’t this imply that you loose track of potentially important business data (domain events) that you no longer publish since you’re only sending commands and thus loose the history? Or were the events you were describing of a pure technical nature? Also if you don’t publish events in a (long running) process how do you update projections to track the current state of the process (which could be useful to show as a progress bar in a UI for example)?

    Regards,
    /Johan

    • Kerry Street permalink*
      May 20, 2013

      Johan,

      Rinat is on vacation but there are several questions on this episode so I will be asking him to take a look and respond to them when he gets back in a couple weeks.

      Until then, my current understanding is that we would still publish the same types of Events that we do today from the Aggregates, but instead of needing to rely on additional classes that will listen for those Events just so that they can fire off a Command message to the “next aggregate in line”, we will now be able to have the Aggregate publish the Command directly to an Actor Id that it already knows about. The Actor/Agg may already know the next step in the process so that we don’t need intermediaries between the Actors/Aggs just so that they can send messages to each other. We may be able to reduce or eliminate these kinds of things in the new approach: https://github.com/Lokad/lokad-cqrs/tree/2013-04-24-btw-episode-04/SaaS.Domain/Processes

  7. May 19, 2013

    When you’re talking about actor implementations of aggregates calling each other directly, you mean one actor placing a message in the recipients mailbox? The little I know about actors tells me that all communication should be done via each actors mailbox (right?).

    Are outgoing messages to other actors being dispatched immediately or will some infrastructure dispatch any pending outgoing messages as the actor finish processing and goes to sleep?

    • Kerry Street permalink*
      May 20, 2013

      Using the other Agg’s mailbox destination to send the message to is also my understanding of what we should be doing.

      Not sure about the second question. Rinat is doing some research and having conversations with people do determine what the “first try” implementation will look like. I look forward to hearing about and sharing what he has discovered on his learning spike. :)

      • May 20, 2013

        Yeah, there are some things regarding consistency in case of failure that concerns me. I’m really looking forward to track the progress on this!

        Some constraints / ground rules might make things clearer

        – Messages to other actors are only sent after processing of a message (not during) (makes sense?)
        – Actors inboxes must be durable
        – An actor may not injure a human being or, through inaction, allow a human being to come to harm
        – An actor must obey the orders given to it by human beings, except where such orders would conflict with the First Law
        – An actor must protect its own existence as long as such protection does not conflict with the First or Second Laws

    • Daniel permalink
      May 20, 2013

      Hi,

      One important aspect of the actor model of computation is that it doesn’t guarantees message ordering in a communication between actors. It assumes that actors can be hosted on different nodes in a network, thus messages may be delivered out of order (due to packet switching mechanism in routers) or may be lost (due to network congestion routers can overflow their message buffers). Therefore, actors made his best effort to deliver a message.

      Akka seems to guarantee message ordering but only between a pair of actors. In a communication between two or more sources and one destination the incoming messages can be interleaved.

      http://doc.akka.io/docs/akka/current/general/message-delivery-guarantees.html

      To guarantee message delivery you must implement your own protocol (using some style of retry policy) or you can use a reliable proxy provided in Akka (a pair of actors that mediates the communication and hides the protocol they implement).

      http://doc.akka.io/docs/akka/current/contrib/reliable-proxy.html

    • Daniel permalink
      May 21, 2013

      Hi,

      - With respect to when the delivery of a message is made.

      The dispatch of a message is “scheduled” immediately to not block the actor while he is processing the message. Nevertheless, the actual delivery is not made immediately because the communication can take a finite time. Furthermore, the sender actor can “schedule” more messages to different targets during the delivery of one message. Thus, an actor needs some kind of message buffer to support asynchronous communication. This is called “buffered asynchronous communication” in the actor model.

      With respect to the guarantee of message delivery and message ordering.

      Let me clarify:
      The actor model assumes the guaranteed delivery of messages but not the message ordering.

      Akka does not guarantees message delivery (it provides “at-most-once” semantics) and guarantees message ordering in a limited form (only between a pair of actors).
      This is good if your application doesn’t need guaranteed delivery, but if you need it, you must implement it, make it explicit in the code, and assume (and deal with) the incurred overhead.

      The advantage of actors is that you can design your system assuming guaranteed delivery of messages and then introduce actor proxys responsible for assuring this. In this way, the system can be decoupled from the details of implementing guaranteed delivery, although the incurred performance cost remains the same.

      • May 21, 2013

        Thank you for taking the time to explain this.

        I figured that there had to be some mechanism like “buffered asynchronous communication” in place. It would be awkward if an actor crashed mid processing after having sent of a couple of commands.

        So can I assume that if actors are combined with cqrs + es the following will have to happen when an actor has processed a message from its inbox?

        1) Dispatch buffered commands to other actors
        2) Flush generated events to an event-store

        Which will involve 2pc unless the same transactional aware data structure is used to store both dispatched commands and generated events..?

    • Daniel permalink
      May 23, 2013

      Hi,

      I was reading the eventsourced extension of Akka to try to get an overview of how the actor model can be adapted to the CQRS+EventSourcing framework.

      - First, to clarify:

      The output messages are not dispatched after each input message is handled. The message handling and the message dispatching are asynchronous processes, although you can explicitly block message handling thread when you send a message (in Akka this is done with Futures) until you receive a reply from the target actor, which has a performance cost.

      - I continue with the problem of unifying actors with the CQRS+EventSourcing framework based on the eventsourced extension of Akka:

      Akka doesn’t guarantee by itself that a dispatched message is actually processed or even received. There are three possible scenarios:

      1. The message never arrives at the receiver mailbox.
      2. The message arrives at the receiver mailbox but the receiver actor dies before picking the message from the mailbox.
      3. The message arrives at the receiver mailbox, then the receiver actor picks the message from the mailbox but crashes afterwards.

      In 2 and 3 the actor can be restarted after it crashes, but its mailbox will be empty. Thus, any message stored in the mailbox before the crash and any message sent while the actor was restarting will be lost.

      To solve the problem of message delivery (scenario 1) you can create proxy actors that implement a retry policy and a protocol in which any message reception is acknowledged.

      To solve the problem of durability of messages (scenario 2) you can backup it with a persistence store (filesystem or database). Despite this, you have no guarantee that the message is actually processed when it is picked from the mailbox (scenario 3). To guarantee this, the receiver actor must copy the message from its persistent mailbox and only delete it after it was correctly processed. This way if the actor crashes in any intermediate step of the message handling procedure, it can retry to handle the message after it restarts. You can even be interested in never delete any message from persistent store (e.g. in an Event Sourced application). In this case you don’t delete the message from the persistent store, you only mark it as processed.

      Despite all this, there can be other kind of problems related to idempotency of message handling procedures (e.g. side effects during message handling). For example, an actor A1 can send a new message to other actor A2 while he is processing an input message. Suppose the actor A2 succesfully processes the sent message. What happen if the actor A1 crashes after he send the message to A2?. Well, according to the formulated solution the actor A1 will restart the message handling procedure, and consequently, it will send another message to A2. You see that this can be a problem if there is no idempotency at the actor A2 side.
      The solution again is with the use of proxy actors. In this case the proxy actor must keep track of the sent messages to avoid a duplicate message dispatch.
      Now what happend if the actor A2 crashes after he receives a message from actor A1?. If the actor A2 has a persistent mailbox he can restart without any problem as I said it before.

      - To answer the question of 2PC:

      All this solution seems to evit 2PC. Remember in a 2PC situation there are various resource managers and one transaction manager. The transaction manager coordinates a commit across resource managers, thus essetially it blocks until all resource managers commit or some of them abort.
      In the presented solution there is no central manager and anybody blocks (at least no implicitly). First, every actor can be itself a transaction manager if it has a persistent store that supports its mailbox and feeds the mailbox with the unprocessed messages on every reset. Further, there is no resource managers since the only resource he manages is its own private state. The only problem arises when there are internal side-effects (the idempotency problem that I mention before).

      • Daniel permalink
        May 23, 2013

        * “All this solution seems to avoid 2PC”

        • Daniel permalink
          May 23, 2013

          * “there is no central manager and nobody blocks”

  8. Kerry Street permalink*
    July 4, 2013

    Adding this link to convo. Was referenced in a tweet to us from @craigsmitham: Basic Actor Model using TPL Dataflow in C#

Leave a Reply

Note: You may use basic HTML in your comments. Your email address will not be published.

Subscribe to this comment feed via RSS