RFC 4379 - 4. Theory of Operation

4. Theory of Operation

An MPLS echo request is used to test a particular LSP. The LSP to be tested is identified by the "FEC Stack"; for example, if the LSP was set up via LDP, and is to an egress IP address of 10.1.1.1, the FEC Stack contains a single element, namely, an LDP IPv4 prefix sub-TLV with value 10.1.1.1/32. If the LSP being tested is an RSVP LSP, the FEC Stack consists of a single element that captures the RSVP Session and Sender Template that uniquely identifies the LSP.

FEC Stacks can be more complex. For example, one may wish to test a VPN IPv4 prefix of 10.1/8 that is tunneled over an LDP LSP with egress 10.10.1.1. The FEC Stack would then contain two sub-TLVs, the bottom being a VPN IPv4 prefix, and the top being an LDP IPv4 prefix. If the underlying (LDP) tunnel were not known, or was considered irrelevant, the FEC Stack could be a single element with just the VPN IPv4 sub-TLV.

When an MPLS echo request is received, the receiver is expected to verify that the control plane and data plane are both healthy (for the FEC Stack being pinged) and that the two planes are in sync. The procedures for this are in section 4.4 below.

4.1. Dealing with Equal-Cost Multi-Path (ECMP)

LSPs need not be simple point-to-point tunnels. Frequently, a single LSP may originate at several ingresses, and terminate at several egresses; this is very common with LDP LSPs. LSPs for a given FEC may also have multiple "next hops" at transit LSRs. At an ingress, there may also be several different LSPs to choose from to get to the desired endpoint. Finally, LSPs may have backup paths, detour paths, and other alternative paths to take should the primary LSP go down.

To deal with the last two first: it is assumed that the LSR sourcing MPLS echo requests can force the echo request into any desired LSP, so choosing among multiple LSPs at the ingress is not an issue. The problem of probing the various flavors of backup paths that will typically not be used for forwarding data unless the primary LSP is down will not be addressed here.

Since the actual LSP and path that a given packet may take may not be known a priori, it is useful if MPLS echo requests can exercise all possible paths. This, although desirable, may not be practical, because the algorithms that a given LSR uses to distribute packets over alternative paths may be proprietary.

To achieve some degree of coverage of alternate paths, there is a certain latitude in choosing the destination IP address and source

UDP port for an MPLS echo request. This is clearly not sufficient; in the case of traceroute, more latitude is offered by means of the Multipath Information of the Downstream Mapping TLV. This is used as follows. An ingress LSR periodically sends an MPLS traceroute message to determine whether there are multipaths for a given LSP. If so, each hop will provide some information how each of its downstream paths can be exercised. The ingress can then send MPLS echo requests that exercise these paths. If several transit LSRs have ECMP, the ingress may attempt to compose these to exercise all possible paths. However, full coverage may not be possible.

4.2. Testing LSPs That Are Used to Carry MPLS Payloads

To detect certain LSP breakages, it may be necessary to encapsulate an MPLS echo request packet with at least one additional label when testing LSPs that are used to carry MPLS payloads (such as LSPs used to carry L2VPN and L3VPN traffic. For example, when testing LDP or RSVP-TE LSPs, just sending an MPLS echo request packet may not detect instances where the router immediately upstream of the destination of the LSP ping may forward the MPLS echo request successfully over an interface not configured to carry MPLS payloads because of the use of penultimate hop popping. Since the receiving router has no means to differentiate whether the IP packet was sent unlabeled or implicitly labeled, the addition of labels shimmed above the MPLS echo request (using the Nil FEC) will prevent a router from forwarding such a packet out unlabeled interfaces.

4.3. Sending an MPLS Echo Request

An MPLS echo request is a UDP packet. The IP header is set as follows: the source IP address is a routable address of the sender; the destination IP address is a (randomly chosen) IPv4 address from the range 127/8 or IPv6 address from the range 0:0:0:0:0:FFFF:127/104. The IP TTL is set to 1. The source UDP port is chosen by the sender; the destination UDP port is set to 3503 (assigned by IANA for MPLS echo requests). The Router Alert option MUST be set in the IP header.

An MPLS echo request is sent with a label stack corresponding to the FEC Stack being tested. Note that further labels could be applied if, for example, the normal route to the topmost FEC in the stack is via a Traffic Engineered Tunnel [RSVP-TE]. If all of the FECs in the stack correspond to Implicit Null labels, the MPLS echo request is considered unlabeled even if further labels will be applied in sending the packet.

If the echo request is labeled, one MAY (depending on what is being pinged) set the TTL of the innermost label to 1, to prevent the ping

request going farther than it should. Examples of where this SHOULD be done include pinging a VPN IPv4 or IPv6 prefix, an L2 VPN endpoint or a pseudowire. Preventing the ping request from going too far can also be accomplished by inserting a Router Alert label above this label; however, this may lead to the undesired side effect that MPLS echo requests take a different data path than actual data. For more information on how these mechanisms can be used for pseudowire connectivity verification, see [VCCV].

In "ping" mode (end-to-end connectivity check), the TTL in the outermost label is set to 255. In "traceroute" mode (fault isolation mode), the TTL is set successively to 1, 2, and so on.

The sender chooses a Sender's Handle and a Sequence Number. When sending subsequent MPLS echo requests, the sender SHOULD increment the Sequence Number by 1. However, a sender MAY choose to send a group of echo requests with the same Sequence Number to improve the chance of arrival of at least one packet with that Sequence Number.

The TimeStamp Sent is set to the time-of-day (in seconds and microseconds) that the echo request is sent. The TimeStamp Received is set to zero.

An MPLS echo request MUST have an FEC Stack TLV. Also, the Reply Mode must be set to the desired reply mode; the Return Code and Subcode are set to zero. In the "traceroute" mode, the echo request SHOULD include a Downstream Mapping TLV.

4.4. Receiving an MPLS Echo Request

Sending an MPLS echo request to the control plane is triggered by one of the following packet processing exceptions: Router Alert option, IP TTL expiration, MPLS TTL expiration, MPLS Router Alert label, or the destination address in the 127/8 address range. The control plane further identifies it by UDP destination port 3503.

For reporting purposes the bottom of stack is considered to be stack-depth of 1. This is to establish an absolute reference for the case where the actual stack may have more labels than there are FECs in the Target FEC Stack.

Furthermore, in all the error codes listed in this document, a stack-depth of 0 means "no value specified". This allows compatibility with existing implementations that do not use the Return Subcode field.

An LSR X that receives an MPLS echo request then processes it as follows.

1. General packet sanity is verified. If the packet is not well- formed, LSR X SHOULD send an MPLS Echo Reply with the Return Code set to "Malformed echo request received" and the Subcode to zero. If there are any TLVs not marked as "Ignore" that LSR X does not understand, LSR X SHOULD send an MPLS "TLV not understood" (as appropriate), and the Subcode set to zero. In the latter case, the misunderstood TLVs (only) are included as sub-TLVs in an Errored TLVs TLV in the reply. The header fields Sender's Handle, Sequence Number, and Timestamp Sent are not examined, but are included in the MPLS echo reply message.

The algorithm uses the following variables and identifiers:

Interface-I: the interface on which the MPLS echo request was received.

Stack-R: the label stack on the packet as it was received.

Stack-D: the label stack carried in the Downstream Mapping TLV (not always present)

Label-L: the label from the actual stack currently being examined. Requires no initialization.

Label-stack-depth: the depth of label being verified. Initialized to the number of labels in the received label stack S.

FEC-stack-depth: depth of the FEC in the Target FEC Stack that should be used to verify the current actual label. Requires no initialization.

Best-return-code: contains the return code for the echo reply packet as currently best known. As algorithm progresses, this code may change depending on the results of further checks that it performs.

Best-rtn-subcode: similar to Best-return-code, but for the Echo Reply Subcode.

FEC-status: result value returned by the FEC Checking algorithm described in section 4.4.1.

/* Save receive context information */

2. If the echo request is good, LSR X stores the interface over which the echo was received in Interface-I, and the label stack with which it came in Stack-R.

/* The rest of the algorithm iterates over the labels in Stack-R, verifies validity of label values, reports associated label switching operations (for traceroute), verifies correspondence between the Stack-R and the Target FEC Stack description in the body of the echo request, and reports any errors. */

/* The algorithm iterates as follows. */

3. Label Validation:

   If Label-stack-depth is 0 {

   /* The LSR needs to report its being a tail-end for the LSP */

   Set FEC-stack-depth to 1, set Label-L to 3 (Implicit Null). Set Best-return-code to 3 ("Replying router is an egress for the FEC at stack depth"), set Best-rtn-subcode to the value of FEC-stack-depth (1) and go to step 5 (Egress Processing). }

   /* This step assumes there is always an entry for well-known label values */

   Set Label-L to the value extracted from Stack-R at depth Label-stack-depth. Look up Label-L in the Incoming Label Map (ILM) to determine if the label has been allocated and an operation is associated with it.

   If there is no entry for L {

   /* Indicates a temporary or permanent label synchronization problem the LSR needs to report an error */

   Set Best-return-code to 11 ("No label entry at stack-depth") and Best-rtn-subcode to Label-stack-depth. Go to step 7 (Send Reply Packet). }

   Else {

   Retrieve the associated label operation from the corresponding NLFE and proceed to step 4 (Label Operation check).

   }

4. Label Operation Check

   If the label operation is "Pop and Continue Processing" {

   /* Includes Explicit Null and Router Alert label cases */

   Iterate to the next label by decrementing Label-stack-depth and loop back to step 3 (Label Validation). }

   If the label operation is "Swap or Pop and Switch based on Popped Label" {

   Set Best-return-code to 8 ("Label switched at stack-depth") and Best-rtn-subcode to Label-stack-depth to report transit switching.

   If a Downstream Mapping TLV is present in the received echo request {

     If the IP address in the TLV is 127.0.0.1 or 0::1 {
        Set Best-return-code to 6 ("Upstream Interface Index
        Unknown").  An Interface and Label Stack TLV SHOULD be
        included in the reply and filled with Interface-I and
        Stack-R.
     }
   
     Else {
   
        Verify that the IP address, interface address, and label
        stack in the Downstream Mapping TLV match Interface-I
        and Stack-R.  If there is a mismatch, set
        Best-return-code to 5, "Downstream Mapping Mismatch".
        An Interface and Label Stack TLV SHOULD be included in
        the reply and filled in based on Interface-I and
        Stack-R.  Go to step 7 (Send Reply Packet).
     }

   }

   For each available downstream ECMP path {

     Retrieve output interface from the NHLFE entry.
   
     /* Note: this return code is set even if Label-stack-depth
        is one */
   
   
   
   
   
   
     If the output interface is not MPLS enabled {
   
        Set Best-return-code to Return Code 9, "Label switched
        but no MPLS forwarding at stack-depth" and set
        Best-rtn-subcode to Label-stack-depth and goto
        Send_Reply_Packet.
     }
   
     If a Downstream Mapping TLV is present {
   
       A Downstream Mapping TLV SHOULD be included in the echo
       reply (see section 3.3) filled in with information about
       the current ECMP path.
     }

   }

   If no Downstream Mapping TLV is present, or the Downstream IP Address is set to the ALLROUTERS multicast address, go to step 7 (Send Reply Packet).

   If the "Validate FEC Stack" flag is not set and the LSR is not configured to perform FEC checking by default, go to step 7 (Send Reply Packet).

   /* Validate the Target FEC Stack in the received echo request.

   First determine FEC-stack-depth from the Downstream Mapping TLV. This is done by walking through Stack-D (the Downstream labels) from the bottom, decrementing the number of labels for each non-Implicit Null label, while incrementing FEC-stack-depth for each label. If the Downstream Mapping TLV contains one or more Implicit Null labels, FEC-stack-depth may be greater than Label-stack-depth. To be consistent with the above stack-depths, the bottom is considered to entry 1. */

   Set FEC-stack-depth to 0. Set i to Label-stack-depth.

   While (i > 0 ) do { ++FEC-stack-depth. if Stack-D[FEC-stack-depth] != 3 (Implicit Null) --i. }

   If the number of labels in the FEC stack is greater than or equal to FEC-stack-depth {

     Perform the FEC Checking procedure (see subsection 4.4.1
     below).
   
     If FEC-status is 2, set Best-return-code to 10 ("Mapping
     for this FEC is not the given label at stack-depth").
   
     If the return code is 1, set Best-return-code to
     FEC-return-code and Best-rtn-subcode to FEC-stack-depth.

   }

   Go to step 7 (Send Reply Packet). }

5. Egress Processing:

   /* These steps are performed by the LSR that identified itself as the tail-end LSR for an LSP. */

   If received echo request contains no Downstream Mapping TLV, or the Downstream IP Address is set to 127.0.0.1 or 0::1 go to step 6 (Egress FEC Validation).

   Verify that the IP address, interface address, and label stack in the Downstream Mapping TLV match Interface-I and Stack-R. If not, set Best-return-code to 5, "Downstream Mapping Mis-match". A Received Interface and Label Stack TLV SHOULD be created for the echo response packet. Go to step 7 (Send Reply Packet).

6. Egress FEC Validation:

   /* This is a loop for all entries in the Target FEC Stack starting with FEC-stack-depth. */

   Perform FEC checking by following the algorithm described in subsection 4.4.1 for Label-L and the FEC at FEC-stack-depth.

   Set Best-return-code to FEC-code and Best-rtn-subcode to the value in FEC-stack-depth.

   If FEC-status (the result of the check) is 1, go to step 7 (Send Reply Packet).

   /* Iterate to the next FEC entry */

   ++FEC-stack-depth.

   If FEC-stack-depth > the number of FECs in the FEC-stack, go to step 7 (Send Reply Packet).

   If FEC-status is 0 { ++Label-stack-depth. If Label-stack-depth > the number of labels in Stack-R, Go to step 7 (Send Reply Packet).

   Label-L = extracted label from Stack-R at depth Label-stack-depth. Loop back to step 6 (Egress FEC Validation). }

7. Send Reply Packet:

   Send an MPLS echo reply with a Return Code of Best-return-code, and a Return Subcode of Best-rtn-subcode. Include any TLVs created during the above process. The procedures for sending the echo reply are found in subsection 4.4.1.

4.4.1. FEC Validation

/* This subsection describes validation of an FEC entry within the Target FEC Stack and accepts an FEC, Label-L, and Interface-I. The algorithm performs the following steps. */

1. Two return values, FEC-status and FEC-return-code, are initialized to 0.

2. If the FEC is the Nil FEC { If Label-L is either Explicit_Null or Router_Alert, return.

   Else { Set FEC-return-code to 10 ("Mapping for this FEC is not the given label at stack-depth"). Set FEC-status to 1 Return. } }

3. Check the FEC label mapping that describes how traffic received on the LSP is further switched or which application it is associated with. If no mapping exists, set FEC-return-code to Return 4, "Replying router has no mapping for the FEC at stack-depth". Set FEC-status to 1. Return.

4. If the label mapping for FEC is Implicit Null, set FEC-status to 2 and proceed to step 5. Otherwise, if the label mapping for FEC is

   Label-L, proceed to step 5. Otherwise, set FEC-return-code to 10 ("Mapping for this FEC is not the given label at stack-depth"), set FEC-status to 1, and return.

5. This is a protocol check. Check what protocol would be used to advertise FEC. If it can be determined that no protocol associated with Interface-I would have advertised an FEC of that FEC-Type, set FEC-return-code to 12 ("Protocol not associated with interface at FEC stack-depth"). Set FEC-status to 1.

6. Return.

4.5. Sending an MPLS Echo Reply

An MPLS echo reply is a UDP packet. It MUST ONLY be sent in response to an MPLS echo request. The source IP address is a routable address of the replier; the source port is the well-known UDP port for LSP ping. The destination IP address and UDP port are copied from the source IP address and UDP port of the echo request. The IP TTL is set to 255. If the Reply Mode in the echo request is "Reply via an IPv4 UDP packet with Router Alert", then the IP header MUST contain the Router Alert IP option. If the reply is sent over an LSP, the topmost label MUST in this case be the Router Alert label (1) (see [LABEL-STACK]).

The format of the echo reply is the same as the echo request. The Sender's Handle, the Sequence Number, and TimeStamp Sent are copied from the echo request; the TimeStamp Received is set to the time-of- day that the echo request is received (note that this information is most useful if the time-of-day clocks on the requester and the replier are synchronized). The FEC Stack TLV from the echo request MAY be copied to the reply.

The replier MUST fill in the Return Code and Subcode, as determined in the previous subsection.

If the echo request contains a Pad TLV, the replier MUST interpret the first octet for instructions regarding how to reply.

If the replying router is the destination of the FEC, then Downstream Mapping TLVs SHOULD NOT be included in the echo reply.

If the echo request contains a Downstream Mapping TLV, and the replying router is not the destination of the FEC, the replier SHOULD compute its downstream routers and corresponding labels for the incoming label, and add Downstream Mapping TLVs for each one to the echo reply it sends back.

If the Downstream Mapping TLV contains Multipath Information requiring more processing than the receiving router is willing to perform, the responding router MAY choose to respond with only a subset of multipaths contained in the echo request Downstream Mapping. (Note: The originator of the echo request MAY send another echo request with the Multipath Information that was not included in the reply.)

Except in the case of Reply Mode 4, "Reply via application level control channel", echo replies are always sent in the context of the IP/MPLS network.

4.6. Receiving an MPLS Echo Reply

An LSR X should only receive an MPLS echo reply in response to an MPLS echo request that it sent. Thus, on receipt of an MPLS echo reply, X should parse the packet to ensure that it is well-formed, then attempt to match up the echo reply with an echo request that it had previously sent, using the destination UDP port and the Sender's Handle. If no match is found, then X jettisons the echo reply; otherwise, it checks the Sequence Number to see if it matches.

If the echo reply contains Downstream Mappings, and X wishes to traceroute further, it SHOULD copy the Downstream Mapping(s) into its next echo request(s) (with TTL incremented by one).

4.7. Issue with VPN IPv4 and IPv6 Prefixes

Typically, an LSP ping for a VPN IPv4 prefix or VPN IPv6 prefix is sent with a label stack of depth greater than 1, with the innermost label having a TTL of 1. This is to terminate the ping at the egress PE, before it gets sent to the customer device. However, under certain circumstances, the label stack can shrink to a single label before the ping hits the egress PE; this will result in the ping terminating prematurely. One such scenario is a multi-AS Carrier's Carrier VPN.

To get around this problem, one approach is for the LSR that receives such a ping to realize that the ping terminated prematurely, and send back error code 13. In that case, the initiating LSR can retry the ping after incrementing the TTL on the VPN label. In this fashion, the ingress LSR will sequentially try TTL values until it finds one that allows the VPN ping to reach the egress PE.

4.8. Non-compliant Routers

If the egress for the FEC Stack being pinged does not support MPLS ping, then no reply will be sent, resulting in possible "false negatives". If in "traceroute" mode, a transit LSR does not support LSP ping, then no reply will be forthcoming from that LSR for some TTL, say, n. The LSR originating the echo request SHOULD try sending the echo request with TTL=n+1, n+2, ..., n+k to probe LSRs further down the path. In such a case, the echo request for TTL > n SHOULD be sent with Downstream Mapping TLV "Downstream IP Address" field set to the ALLROUTERs multicast address until a reply is received with a Downstream Mapping TLV. The label stack MAY be omitted from the Downstream Mapping TLV. Furthermore, the "Validate FEC Stack" flag SHOULD NOT be set until an echo reply packet with a Downstream Mapping TLV is received.