6. The RAW Control Loop
This section preserves the RFC text for the RAW architecture, including RAW, DetNet, TSN, OAM, PREOF, PLR, PSE, PCE, PDR, SLA, SLO, SLI, recovery graphs, protection paths, LL API, diagrams, tables, and security considerations.
Original RFC Text
6. The RAW Control Loop
The RAW architecture is based on an abstract OODA loop that controls
the operation of a recovery graph. The generic concept involves the
following:
1. Operational Plane measurement protocols allow OAM to observe
(like the first "O" in "OODA") some or all hops along a recovery
graph as well as the end-to-end packet delivery.
2. The DetNet Controller Plane establishes primary and protection
paths for use by the RAW Network Plane. The orientation function
reports data and information such as link statistics to be used
by the routing function to compute, install, and maintain the
recovery graphs. The routing function may also generate
intelligence such as a trained model for link quality prediction,
which in turn can be used by the orientation function (like the
second "O" in "OODA") to influence the path selection by the PLR
within the RAW OODA loop.
3. A PLR operates at the DetNet Service sub-layer and hosts the
decision function (like the "D" in "OODA"). The decision
function determines which DetNet paths will be used for future
packets that are routed within the recovery graph.
4. Service protection actions are actuated or triggered over the LL
API by the PLR to increase the reliability of the end-to-end
transmissions. The RAW architecture also covers in-situ
signaling that is embedded within live user traffic [RFC9378]
(e.g., via OAM) when the decision is acted (like the "A" in
"OODA") upon by a node that is downstream in the recovery graph
from the PLR.
The overall OODA loop optimizes the use of redundancy to achieve the
required reliability and availability SLO(s) while minimizing the use
of constrained resources such as spectrum and battery.
6.1. Routing Timescale Versus Forwarding Timescale
With DetNet, the Controller Plane Function (CPF) handles the routing
computation and maintenance. With RAW, the routing operation is
segregated from the RAW control loop, so it may reside in the
Controller Plane, whereas the control loop itself happens in the
Network Plane. To achieve RAW capabilities, the routing operation is
extended to generate the information required by the orientation
function in the loop. For example, the routing function may propose
DetNet paths to be used as a reflex action in response to network
events or provide an aggregated history that the orientation function
can use to make a decision.
In a wireless mesh, the path to a routing function located in the
Controller Plane can be expensive and slow, possibly going across the
whole mesh and back. Reaching the Controller Plane can also be slow
in regard to the speed of events that affect the forwarding operation
in the Network Plane at the radio layer. Note that a distributed
routing protocol may also take time and consume excessive wireless
resources to reconverge to a new optimized state.
As a result, the DetNet routing function is not expected to be aware
of and react to very transient changes. The abstraction of a link at
the routing level is expected to use statistical metrics that
aggregate the behavior of a link over long periods of time and
represent its properties as shades of gray as opposed to numerical
values such as a link quality indicator or a Boolean value for either
up or down.
The interaction between the network nodes and the routing function is
handled by the orientation function, which reports to the routing
function and sends control information in a digested form back to the
RAW node to be used inside a forwarding control loop for traffic
steering.
Figure 8 illustrates a Network Plane recovery graph with links P-Q
and N-E flapping, possibly in a transient fashion due to short-term
interferences and possibly for a longer time (e.g., due to obstacles
between the sender and the receiver or hardware failures). In order
to maintain a received redundancy around a value of 2 (for instance),
RAW may leverage a higher ARQ on these hops if the overall latency
permits the extra delay or enable alternate paths between ingress I
and egress E. For instance, RAW may enable protection path I ==> F
==> N ==> Q ==> M ==> R ==> E that routes around both issues and
provides some degree of spatial diversity with protection path I ==>
A ==> B ==> C ==> D ==> E.
+----------------+
| DetNet |
| Routing |
+----------------+
^
|
Slow
| Controller Plane
_-._-._-._-._-._-. | ._-._-._-._-._-._-._-._-._-._-._-._-
_-._-._-._-._-._-._-. | _-._-._-._-._-._-._-._-._-._-._-._-
| Network Plane
Expensive
|
__...--- | ----.._.
.( | )-._
( v ).
( A--------B---C----D )
_ - / \ / \ --._
( I---F--------N--***-- E -
-_ \ / / )
( P--***---Q----M---R .
_ )- ._
- <------ Fast -------> )
( -._ .-
(_.___.._____________.____.._ __-____)
*** = flapping at this time
Figure 8: Timescales
In the case of wireless, the changes that affect the forwarding
decision can happen frequently and often for short durations. An
example of this is a mobile object that moves between a transmitter
and a receiver and cancels the line-of-sight transmission for a few
seconds. Another example is radar that measures the depth of a pool
using the ISM band and interferes on a particular channel for a split
second.
Thus, there is a desire to separate the long-term computation of the
route and the short-term forwarding decision. In that model, the
routing operation computes a recovery graph that enables multiple
Unequal-Cost Multipath (UCMP) forwarding solutions along so-called
protection paths and leaves it to the Network Plane to make the
short-term decision of which of these possibilities should be used
for which upcoming packets and flows.
In the context of Traffic Engineering (TE), an alternate path can be
used upon the detection of a failure in the main path, e.g., using
OAM in Multiprotocol Label Switching - Transport Profile (MPLS-TP) or
BFD over a collection of Software-Defined Wide Area Network (SD-WAN)
tunnels.
RAW formalizes a forwarding timescale that may be order(s) of
magnitude shorter than the Controller Plane routing timescale and
separates the protocols and metrics that are used at both scales.
Routing can operate on long-term statistics such as delivery ratio
over minutes to hours, but as a first approximation, it can ignore
the cause of transient losses. On the other hand, the RAW forwarding
decision is made at the scale of a burst of packets and uses
information that must be pertinent at the present time for the
current transmission(s).
6.2. OODA Loop
The RAW architecture applies the generic OODA model to continuously
optimize the spectrum and energy used to forward packets within a
recovery graph, instantiating the OODA steps as follows:
Observe: Network Plane measurements, including protocols for OAM,
observe the local state of the links and some or all hops along a
recovery graph as well as the end-to-end packet delivery (see more
in Section 6.3). Information can also be provided by lower-layer
interfaces such as DLEP.
Orient: The orientation function reports data and information such
as the link statistics and leverages offline-computed wisdom and
knowledge to orient the PLR for its forwarding decision (see more
in Section 6.4).
Decide: A local PLR decides which DetNet path to use for future
packet(s) that are routed along the recovery graph (see more in
Section 6.5).
Act: PREOF Data Plane actions are controlled by the PLR over the LL
API to increase the reliability of the end-to-end transmission.
The RAW architecture also covers in-situ signaling when the
decision is acted by a node that is down the recovery graph from
the PLR (see more in Section 6.6).
+-------> Orientation ---------+
| reflex actions |
| pre-trained model |
| |
......................................
| |
| Service sub-layer |
| v
Observe (OAM) Decide (PLR)
^ |
| |
| |
+------- Act (LL API) <--------+
Figure 9: The RAW OODA Loop
The overall OODA loop optimizes the use of redundancy to achieve the
required reliability and availability Service Level Agreement (SLA)
while minimizing the use of constrained resources such as spectrum
and battery.
6.3. Observe: RAW OAM
The RAW in-situ OAM operation in the Network Plane may observe either
a full recovery graph or the DetNet path that is being used at this
time. As packets may be load balanced, replicated, eliminated, and/
or fragmented for Network Coding FEC, the RAW in-situ operation needs
to be able to signal which operation occurred to an individual
packet.
Active RAW OAM may be needed to observe the unused segments and
evaluate the desirability of a rerouting decision.
Finally, the RAW Service sub-layer Service Assurance may observe the
individual PREOF operation of a DetNet relay node to ensure that it
is conforming; this might require injecting an OAM packet at an
upstream point inside the recovery graph and extracting that packet
at another point downstream before it reaches the egress.
This observation feeds the RAW PLR that makes the decision on which
path is used at which RAW node, for one packet or a small continuous
series of packets.
In the case of end-to-end protection in a wireless mesh, the recovery
graph is strict and congruent with the path so all links are
observed.
Conversely, in the case of Radio Access Protection, illustrated in
Figure 10, the recovery graph is loose and only the first hop is
observed; the rest of the path is abstracted and considered
infinitely reliable. The loss of a packet is attributed to the
first-hop Radio Access Network (RAN), even if a particular loss
effectively happens farther down the path. In that case, RAW enables
technology diversity (e.g., Wi-Fi and 5G), which in turn improves the
diversity in spectrum usage.
Opaque to OAM
<---------------------------->
.- .. - ..
RAN 1 --------( ).__
+-------+ / ( ). +------+
|Ingress|- __________Tunnel_______________|Egress|
| End |------ RAN 2 --_______________________________ End |
|System |- ( ) |System|
+-------+ \ ( ). +------+
RAN n ----( )
(_______...___.__...____....__..)
<-------L2------>
Observed by OAM
<----------------------L3----------------------->
Figure 10: Observed Links in Radio Access Protection
The links that are not observed by OAM are opaque to it, meaning that
the OAM information is carried across and possibly echoed as data,
but there is no information captured in intermediate nodes. In the
example above, the tunnel underlay is opaque and not controlled by
RAW; still, RAW OAM measures the end-to-end latency and delivery
ratio for packets sent via RAN 1, RAN 2, and RAN 3, and determines
whether a packet should be sent over either access link or a
collection of those access links.
6.4. Orient: The RAW-Extended DetNet Operational Plane
RAW separates the long timescale at which a recovery graph is
computed and installed from the short timescale at which the
forwarding decision is taken for one or a few packets (see
Section 6.1) that experience the same path until the network
conditions evolve and another path is selected within the same
recovery graph.
The recovery graph computation is out of scope, but RAW expects that
the CPF that installs the recovery graph also provides related
knowledge in the form of metadata about the links, segments, and
possible DetNet paths. That metadata can be a pre-digested
statistical model and may include prediction of future flaps and
packet loss, as well as recommended actions when that happens.
The metadata may include:
* A set of pre-determined DetNet paths that are prepared to match
expected link-degradation profiles, so the DetNet relay nodes can
take reflex rerouting actions when facing a degradation that
matches one such profile; and
* Link-quality statistics history and pre-trained models (e.g., to
predict the short-term variation of quality of the links in a
recovery graph).
The recovery graph is installed with measurable objectives that are
computed by the CPF to achieve the RAW SLA. The objectives can be
expressed as any of the maximum number of packets lost in a row,
bounded latency, maximal jitter, maximum number of interleaved out-
of-order packets, average number of copies received at the
elimination point, and maximal delay between the first and the last
received copy of the same packet.
6.5. Decide: The Point of Local Repair
The RAW OODA loop operates at the path selection timescale to provide
agility versus the brute-force approach of flooding the whole
recovery graph. Within the redundant solutions that are proposed by
the routing function, the OODA loop controls what is used for each
packet and provides a reliable and available service while minimizing
the waste of constrained resources.
To that effect, RAW defines the Point of Local Repair (PLR), which
performs rapid local adjustments of the forwarding tables within the
path diversity that is available in that in the recovery graph. The
PLR enables exploitation of the richer forwarding capabilities at a
faster timescale over a portion of the recovery graph, in either a
loose or a strict fashion.
The PLR operates on metrics that evolve quickly and need to be
advertised at a fast rate (but only locally, within the recovery
graph), and the PLR reacts to the metric updates by changing the
DetNet path in use for the affected flows.
The rapid changes in the forwarding decisions are made and contained
within the scope of a recovery graph, and the actions of the PLR are
not signaled outside the recovery graph. This is as opposed to the
routing function that must observe the whole network and optimize all
the recovery graphs globally, which can only be done at a slow pace
and with long-term statistical metrics, as presented in Table 1.
+===============+=========================+=====================+
| | Controller Plane | PLR |
+===============+=========================+=====================+
| Communication | Slow, distributed | Fast, local |
+===============+-------------------------+---------------------+
| Timescale | Path computation + | Lookup + protection |
| (order) | round trip, | switch, micro to |
| | milliseconds to seconds | milliseconds |
+===============+-------------------------+---------------------+
| Network Size | Large, many recovery | Small, limited set |
| | graphs to optimize | of protection paths |
| | globally | |
+===============+-------------------------+---------------------+
| Considered | Averaged, statistical, | Instantaneous |
| Metrics | shade of grey | values / boolean |
| | | condition |
+===============+-------------------------+---------------------+
Table 1: Centralized Decision Versus PLR
The PLR sits in the DetNet forwarding sub-layer of edge and relay
nodes. The PLR operates on the packet flow, learning the recovery
graph and path-selection information from the packet and possibly
making a local decision and retagging the packet to indicate so. On
the other hand, the PLR interacts with the lower layers (through
triggers and DLEP) and with its peers (through OAM) to obtain up-to-
date information about its links and the quality of the overall
recovery graph, respectively, as illustrated in Figure 11.
|
Packet | going
down the | stack
+==========v==========+=====================+===================+
|(In-situ OAM + iCTRL)| (L2 triggers, DLEP) | (Hybrid OAM) |
+==========v==========+=====================+===================+
| Learn from | | Learn from |
| packet tagging > Maintain < end-to-end |
+----------v----------+ Forwarding | OAM packets |
| Forwarding decision < State +---------^---------|
+----------v----------+ | Enrich or |
+ Retag packet | Learn abstracted > regenerate |
| and forward | metrics about links | OAM packets |
+..........v..........+..........^..........+........^.v........+
| Lower layers |
+..........v.....................^...................^.v........+
Frame | sent Frame | L2 ack Active | | OAM
over | wireless in | in and | | out
v | | v
Figure 11: PLR Conceptual Interfaces
6.6. Act: DetNet Path Selection and Reliability Functions
The main action by the PLR is the swapping of the DetNet path within
the recovery graph for the future packets. The candidate DetNet
paths represent different energy and spectrum profiles and provide
protection against different failures.
The LL API enriches the DetNet protection services (PREOF) with the
possibility to interact with lower-layer, one-hop reliability
functions that are more typical with wireless links than with wired
ones, including ARQ, FEC, and other techniques such as overhearing
and constructive interferences. Because RAW may be leveraged on
wired links (e.g., to save power), it is not expected that all lower
layers support all those capabilities.
RAW provides hints to the lower-layer services on the desired
outcome, and the lower layer acts on those hints to provide the best
approximation of that outcome, e.g., a level of reliability for one-
hop transmission within a bounded budget of time and/or energy.
Thus, the LL API makes possible cross-layer optimization for
reliability depending on the actual abstraction provided. That is,
some reliability functions are controlled from Layer 3 using an
abstract interface, while they are really operated at the lower
layers.
The RAW path selection can be implemented in both centralized and
distributed approaches. In the centralized approach, the PLR may
obtain a set of pre-computed DetNet paths matching a set of expected
failures and apply the appropriate DetNet paths for the current state
of the wireless links. In the distributed approach, the signaling in
the packet may be more abstract than an explicit path, and the PLR
decision might be revised along the selected DetNet path based on a
better knowledge of the rest of the way.
The dynamic DetNet path selection in RAW avoids the waste of critical
resources such as spectrum and energy while providing for the assured
SLA, e.g., by rerouting and/or adding redundancy only when a loss
spike is observed.