11/2 ---- The ordering of events in distributed systems

(Lamport, "Time, Clocks, and the ordering of events in a distributed
system", Comm. ACM, July 1978, Vol. 21, No 7, pp 558-565.)

- Observation: many events in a distributed system need to be ordered
in particular order.

- Example:  A fault tolerant NFS service.

   - Lets assume we replicate a server (and its state) on two
     machines

   - If we want to keep the two machines consistent we need to execute
     operations in the same order on both machines.  This may be hard
     in distributed system because message be delayed or reordered.
     Furthermore, certain operations might appear to happen
     concurrently.

   - Example of what happens if operations are not ordered:
		     
			    X = 1		     X =1
	     Client 1	   Server 1		   Server 2

	 snd write x = 2    
						rec write x = 2
	 snd write x = 3   
			   rec write x = 3	rec write x = 3
			   rec write x = 2	

			      X = 2		     X = 3

      - If server 2 fails and client 2 reads X back from server 1,
        then it observes a strange result.
      - More interesting can happen if we have two clients operating
        concurrently on X----in a distributed system events form a
        partial order.
 
   - In this example, we need to guarantee that both servers execute
     all operations in the same order. (State-machine approach of
     replication.)

- What does "happening before" mean in a distributed system?
  
  - In normal life, we look at a physical clock and say something
    happens before or after.

  - Using a physical clocks in a distributed system to determine
    whether an event happens before or after is difficult----it
    requires having physical clocks on each computer and synchronizing
    clocks accurately is difficult.
     
  - The "happen-before" relationship in a distributed system:

    Model: Processes is a sequence of events, where events is an
    abstract notion, which could be a single instruction, a procedure,
    sending a message, receiving an interrupt, etc.

    Happens before within a process: an event A that precedes event B
    in the sequence is said to happen before B. 

    Lets assume two events: send and receiving messages.

    A happens before B
      (1) if A and B are events in the same process and 
      A comes before B, or
      (2) if A is the sending of a message by one process and B is the
      receipt of the same message by another process.
      
    Notation: A happens before B is written as A -> B.

    Two events are concurrent if A didn't happen before B and B didn't
    happen before A.

    Happens-before defines an irreflexive partial ordering.

- Logical clocks

  - Using the happens-before relationship we can define a logical
    clock.

    - Define Ci, a *logical* clock for process Pi as follows:

      Ci(A) = a number for event A.   The number is can be a counter.

    - The global Clock C for the system must adhere two:
	  if A -> B, then C(a) < C(b)

      This condition is satisfied if:
      (1) if A and B are events in Pi, then Ci(A) < Ci(B)
      (2) if A is the sending of a message by process Pi and B is the
      receipt of that message by process Pj, then Ci(A) < Cj(B).

    - Implementation rules:
       (1) Pi increments Ci between any two successive events
       (2) 
          (a) If event A is the sending of a message M by process Pi, the
          the message contains a timestamp Tm = Ci(a).
	  (b) Upon receiving a message M, process Pj sets Cj >= its
	  present value and greater than Tm.

    - Example: use logical clocks to order messages in our NFS example.

- Total ordering of events

  - Order events A -> B by their logical clocks, Break ties by
    ordering them in some arbitrary order
    
  - A ==> B if only if:
    (1) Ci(B) < Cj(B), or
    (2) Ci(A) == Cj((B) and Pi < Pj.

    ==> is a total order.

  - Example: add another client to NFS example doing the same
    operations, totally order all events using the relationships
    defined above.  Note: that a server may need to wait a while
    before it knows where a message fits in the total order.

- Consistency of replicated data

  - If use the total order relationship, we get consistency
    as defined in the previous lecture (i.e., read observes last
    write).