Flooding Link State Information

1. Last time, I showed that in certain situations, the distributed Bellman-Ford algorithm converges quite slowly. As a result, we now need to look for a way to make a distributed version of Dijkstra's algorithm feasible.

   To do this, all we really need is to find a way to send each node's link state information to all other nodes.

2. We cannot depend on routes based on shortest paths to deliver the link-state packets required by Dijkstra's algorithm to determine these shortest paths:

   • If nothing else, think of the initialization problem. How do you distribute the link state packets when the network is first started and nodes have no idea how to route anything.

   • Similar to the start-up problem is the problem of how the network will recover if a damaged link whose failure had partitioned the network into disjoint components becomes operational again.

   • Somewhat less critically, if the load on the current shortest path rises suddenly, then to reroute packets to use less loaded paths a network that used "shortest paths" would have to depend on congested paths to deliver the information needed to route around the congestion.

3. The alternative we will use is packet flooding, but we can't use the simple version of flooding used by transparent bridges to forward broadcast packets and packets.

   • The forwarding of broadcast packets in an extended LAN built using transparent bridges depends on the fact that bridge ports are deactivated as necessary to eliminate loops from the extended LAN. A packet will be forwarded at most as many times as the length of the longest path in the spanning tree.

   • As explained above, for a general-purpose network, we want the ability to use all links -- even those that form parts of cycles.

4. If we ignore some of the more perverse ways network hardware might fail on us, we can devise a fairly simple flooding procedure based on sequence numbers.

   • When a node sends a routing update, we require that it include a sequence number in the update packet.

   • In order to run Dijkstra's algorithm, each node, i, already has to store the contents of the last routing update packet it received from each of the other stations in the network (i.e. all the d_{i ->j}). We will assume that in addition, it saves the sequence number found in each update packet.

   • With this assumption, all we do to flood packets without worrying about cycles is

     • Ignore any link state update packet we receive from a given node j whose sequence number is less than or equal to the last update we received from j.

     • Forward any update we don't ignore on all links other than the link on which it was received.

   • If everything works, we can assume that no node will forward a given update packet through a given link more than once, so the number of copies of a packet will be no greater than the number of edges in the directed graph that represents the network (i.e. twice the number of links).

5. In the real world, there are several difficulties that this simple approach does not quite deal with:

   • If a network node crashes and then restarts, it might not remember its last sequence number.

   • Undetected errors in packets in transit (or those stored at a node to be forwarded), might result in false sequence numbers.

   • Evil spirits might send bogus update packets with false information (including sequence numbers, link states and even source node ids).

6. To appreciate the significance of these concerns, you need to first recognize two issues that make the handling of errors in networks different from reliability issues in other software systems.

   • A network may be expected to operate correctly for a very long time (years). During this very long time it performs thousands of operations in very short time periods. As a result, even events that have very, very low probabilities per operation will almost certainly eventually happen in a network.

     • While techniques like the use of CRC's may make the probability of undetected errors low, they will still eventually happen.

   • It is somewhere between very difficult and impossible to solve a problem in a large network by turning the network off and then restarting it.

7. To appreciate the impact of these assumptions, think about how they effect the issue of the size of sequence number used.

   • Since having sequence numbers wrap around will complicate things, we might think it best to use a large sequence number field (enough bits for 10000 years worth of update packets might be good).

   • Now, suppose that either

     • an undetected error in an update packet produces a sequence number that would not otherwise occur for 5000 years, or

     • a hacker finds a way to deliberately forge a packet that looks like an update with a very large sequence number.

   • In either case, the algorithm proposed above would be in big trouble. Machines in the network would refuse to forward valid update packets for the machine whose update was corrupted or forged for thousands of years unless someone manually reconfigured every switch.

   • As a result, it is better to use a smaller range of sequence numbers. The range should be large enough to let us distinguish young packets from old when things work, but small enough that real sequence numbers will exceed any spurious sequence number within a time period that we consider tolerable to recover from a rare error.

8. Having sequence number wrap around is not difficult.

   • We just need to carefully define what we mean for sequence number n to be older than m.

     • Basically, when given a sequence number m in a system where sequence numbers wrap around every R steps, we really know only that m = M mod R where M is the "real" sequence number of the packet. Put another way, all we know is that M = m + k_m*R for some k_m.

     • As a result, given two sequence numbers m and n such that m < n, it is still possible that M > N where M and N are the "real" sequence numbers corresponding to m and n. Knowing the values of m and n isn't enough to determine the ordering of M and N.

     • What we do to resolve this ambiguity is assume that we are more likely to be comparing sequence numbers of packets that were produced close together in time than far apart in time.

     • Given this assumption, the rule for comparing sequence numbers is that if m < n and n - m < R/2 then we assume that m is actually older than n. However, if m < n and n - m >= R/2 then we assume that n is older than m.

     • The following figure illustrates the logic behind this rule.

       

       Basically, we are deciding whether m is closer to n or to n - R by looking at the difference between n and m. Since the actual sequence number corresponding to n could be any number of the form n + k*R, this enables us to determine which of the "real" sequence numbers that might correspond to n is closest to m.

9. Once we start using wrap around sequence number it becomes necessary to make sure the assumption that we are more likely to be comparing sequence numbers produced close together in time is realistic.

   • As the paper on the ARPANET flooding algorithm failure explains, if you end up with three packets numbered 0, R/3 and 2R/3 floating around your network, then 2R/3 seems younger than R/3 which seems younger than 0, but oddly 0 is also younger than 2R/3. As a result, such packets can circulate around in a loop and one of them will always seem younger than any new packet produced.

   • To avoid this, the network needs another mechanisms to ensure that truly old packets do get discarded.

   • A "simple" approach to this is to include a Time-to-live (or age) field in each packet.

     • While holding a packet, a station periodically subtracts some estimate of the time it has been holding the packet from the time-to-live field.

     • A packet whose time to live becomes 0 is considered older than any packet that still has time to live.