2. Synchronisation of RTP Flows
RTP flows are synchronised by receivers based on information that is contained in RTCP SR packets generated by senders (specifically, the NTP-format timestamp and the RTP timestamp). Synchronisation requires that a common reference clock MUST be used to generate the NTP-format timestamps in a set of flows that are to be synchronised (i.e., when synchronising several RTP flows, the RTP timestamps for each flow are derived from separate, and media specific, clocks, but the NTP-format timestamps in the RTCP SR packets of all flows to be synchronised MUST be sampled from the same clock). To achieve faster and more accurate synchronisation, it is further RECOMMENDED that senders and receivers use a synchronised common NTP-format reference clock with common properties, especially timebase, where possible (recognising that this is often not possible when RTP is used outside of controlled environments); the means by which that common reference clock and its properties are signalled and distributed is outside the scope of this memo.
For multimedia sessions, each type of media (e.g., audio or video) is sent in a separate RTP session, and the receiver associates RTP flows to be synchronised by means of the canonical end-point identifier (CNAME) item included in the RTCP Source Description (SDES) packets generated by the sender or signalled out of band [RFC5576]. For layered media, different layers can be sent in different RTP sessions, or using different synchronisation source (SSRC) values within a single RTP session; in both cases, the CNAME is used to identify flows to be synchronised. To ensure synchronisation, an RTP sender MUST therefore send periodic compound RTCP packets following Section 6 of RFC 3550 [RFC3550].
The timing of these periodic compound RTCP packets will depend on the number of members in each RTP session, the fraction of those that are sending data, the session bandwidth, the configured RTCP bandwidth fraction, and whether the session is multicast or unicast (see RFC 3550, Section 6.2 for details). In summary, RTCP control traffic is allocated a small fraction, generally 5%, of the session bandwidth, and of that fraction, one quarter is allocated to active RTP senders, while receivers use the remaining three quarters (these fractions can be configured via the Session Description Protocol (SDP) [RFC3556]). Each member of an RTP session derives an RTCP reporting interval based on these fractions, whether the session is multicast or unicast, the number of members it has observed, and whether it is actively sending data or not. It then sends a compound RTCP packet on average once per reporting interval (the actual packet transmission time is randomised in the range [0.5 ... 1.5] times the reporting interval to avoid synchronisation of reports).
A minimum reporting interval of 5 seconds is RECOMMENDED, except that the delay before sending the initial report "MAY be set to half the minimum interval to allow quicker notification that the new participant is present" [RFC3550]. Also, for unicast sessions, "the delay before sending the initial compound RTCP packet MAY be zero" [RFC3550]. In addition, for unicast sessions, and for active senders in a multicast session, the fixed minimum reporting interval MAY be scaled to "360 divided by the session bandwidth in kilobits/second. This minimum is smaller than 5 seconds for bandwidths greater than 72 kb/s" [RFC3550].
2.1. Initial Synchronisation Delay
A multimedia session comprises a set of concurrent RTP sessions among a common group of participants, using one RTP session for each media type. For example, a videoconference (which is a multimedia session) might contain an audio RTP session and a video RTP session. To allow a receiver to synchronise the components of a multimedia session, a compound RTCP packet containing an RTCP SR packet and an RTCP SDES packet with a CNAME item MUST be sent to each of the RTP sessions in the multimedia session by each sender. A receiver cannot synchronise playout across the multimedia session until such RTCP packets have been received on all of the component RTP sessions. If there is no packet loss, this gives an expected initial synchronisation delay equal to the average time taken to receive the first RTCP packet in the RTP session with the longest RTCP reporting interval. This will vary between unicast and multicast RTP sessions.
The initial synchronisation delay for layered sessions is similar to that for multimedia sessions. The layers cannot be synchronised until the RTCP SR and CNAME information has been received for each layer in the session.
2.1.1. Unicast Sessions
For unicast multimedia or layered sessions, senders SHOULD transmit an initial compound RTCP packet (containing an RTCP SR packet and an RTCP SDES packet with a CNAME item) immediately on joining each RTP session in the multimedia session. The individual RTP sessions are considered to be joined once any in-band signalling for NAT traversal (e.g., [RFC5245]) and/or security keying (e.g., [RFC5764], [ZRTP]) has concluded, and the media path is open. This implies that the initial RTCP packet is sent in parallel with the first data packet following the guidance in RFC 3550 that "the delay before sending the initial compound RTCP packet MAY be zero" and, in the absence of any packet loss, flows can be synchronised immediately.
It is expected that NAT pinholes, firewall holes, quality-of-service, and media security keys will have been negotiated as part of the signalling, whether in-band or out-of-band, before the first RTCP packet is sent. This should ensure that any middleboxes are ready to accept traffic, and reduce the likelihood that the initial RTCP packet will be lost.
2.1.2. Source-Specific Multicast (SSM) Sessions
For multicast sessions, the delay before sending the initial RTCP packet, and hence the synchronisation delay, varies with the session bandwidth and the number of members in the session. For a multicast multimedia or layered session, the average synchronisation delay will depend on the slowest of the component RTP sessions; this will generally be the session with the lowest bandwidth (assuming all the RTP sessions have the same number of members).
When sending to a multicast group, the reduced minimum RTCP reporting interval of 360 seconds divided by the session bandwidth in kilobits per second [RFC3550] should be used when synchronisation latency is likely to be an issue. Also, as usual, the reporting interval is halved for the first RTCP packet. Depending on the session bandwidth and the number of members, this gives the average synchronisation delays shown in Figure 1.
Session| Number of receivers:
Bandwidth| 2 3 4 5 10 100 1000 10000
--+------------------------------------------------
8 kbps| 2.73 4.10 5.47 5.47 5.47 5.47 5.47 5.47
16 kbps| 2.50 2.50 2.73 2.73 2.73 2.73 2.73 2.73
32 kbps| 2.50 2.50 2.50 2.50 2.50 2.50 2.50 2.50
64 kbps| 2.50 2.50 2.50 2.50 2.50 2.50 2.50 2.50
128 kbps| 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41
256 kbps| 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70
512 kbps| 0.35 0.35 0.35 0.35 0.35 0.35 0.35 0.35
1 Mbps| 0.18 0.18 0.18 0.18 0.18 0.18 0.18 0.18
2 Mbps| 0.09 0.09 0.09 0.09 0.09 0.09 0.09 0.09
4 Mbps| 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04
Figure 1: Average Initial Synchronisation Delay in Seconds
for an RTP Session with 1 Sender
These numbers assume a source-specific multicast channel with a single active sender, assuming an average RTCP packet size of 70 octets. These intervals are sufficient for lip-synchronisation without excessive delay, but might be viewed as having too much latency for synchronising parts of a layered video stream.
The RTCP interval is randomised in the usual manner, so the minimum synchronisation delay will be half these intervals, and the maximum delay will be 1.5 times these intervals. Note also that these RTCP intervals are calculated assuming perfect knowledge of the number of members in the session.
2.1.3. Any-Source Multicast (ASM) Sessions
For ASM sessions, the fraction of members that are senders plays an important role, and causes more variation in average RTCP reporting interval. This is illustrated in Figure 2 and Figure 3, which show the RTCP reporting interval for the same session bandwidths and receiver populations as the SSM session described in Figure 1, but for sessions with 2 and 10 senders, respectively. It can be seen that the initial synchronisation delay scales with the number of senders (this is to ensure that the total RTCP traffic from all group members does not grow without bound) and can be significantly larger than for source-specific groups. Despite this, the initial synchronisation time remains acceptable for lip-synchronisation in typical small-to-medium sized group video conferencing scenarios.
Note that multi-sender groups implemented using multi-unicast with a central RTP translator (Topo-Translator in the terminology of [RFC5117]) or mixer (Topo-Mixer), or some forms of video switching MCU (Topo-Video-switch-MCU) distribute RTCP packets to all members of the group, and so scale in the same way as an ASM group with regards to initial synchronisation latency.
Session| Number of receivers:
Bandwidth| 2 3 4 5 10 100 1000 10000
--+------------------------------------------------
8 kbps| 2.73 4.10 5.47 6.84 10.94 10.94 10.94 10.94
16 kbps| 2.50 2.50 2.73 3.42 5.47 5.47 5.47 5.47
32 kbps| 2.50 2.50 2.50 2.50 2.73 2.73 2.73 2.73
64 kbps| 2.50 2.50 2.50 2.50 2.50 2.50 2.50 2.50
128 kbps| 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41
256 kbps| 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70
512 kbps| 0.35 0.35 0.35 0.35 0.35 0.35 0.35 0.35
1 Mbps| 0.18 0.18 0.18 0.18 0.18 0.18 0.18 0.18
2 Mbps| 0.09 0.09 0.09 0.09 0.09 0.09 0.09 0.09
4 Mbps| 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04
Figure 2: Average Initial Synchronisation Delay in Seconds
for an RTP Session with 2 Senders
Session| Number of receivers:
Bandwidth| 2 3 4 5 10 100 1000 10000
--+------------------------------------------------
8 kbps| 2.73 4.10 5.47 6.84 13.67 54.69 54.69 54.69
16 kbps| 2.50 2.50 2.73 3.42 6.84 27.34 27.34 27.34
32 kbps| 2.50 2.50 2.50 2.50 3.42 13.67 13.67 13.67
64 kbps| 2.50 2.50 2.50 2.50 2.50 6.84 6.84 6.84
128 kbps| 1.41 1.41 1.41 1.41 1.41 3.42 3.42 3.42
256 kbps| 0.70 0.70 0.70 0.70 0.70 1.71 1.71 1.71
512 kbps| 0.35 0.35 0.35 0.35 0.35 0.85 0.85 0.85
1 Mbps| 0.18 0.18 0.18 0.18 0.18 0.43 0.43 0.43
2 Mbps| 0.09 0.09 0.09 0.09 0.09 0.21 0.21 0.21
4 Mbps| 0.04 0.04 0.04 0.04 0.04 0.11 0.11 0.11
Figure 3: Average Initial Synchronisation Delay in Seconds
for an RTP Session with 10 Senders
2.1.4. Discussion
For unicast sessions, the existing RTCP SR-based mechanism allows for immediate synchronisation, provided the initial RTCP packet is not lost.
For SSM sessions, the initial synchronisation delay is sufficient for lip-synchronisation, but may be larger than desired for some layered codecs. The rationale for not sending immediate RTCP packets for multicast groups is to avoid implosion of requests when large numbers of members simultaneously join the group ("flash crowd"). This is not an issue for SSM senders, since there can be at most one sender, so it is desirable to allow SSM senders to send an immediate RTCP SR on joining a session (as is currently allowed for unicast sessions, which also don't suffer from the implosion problem). SSM receivers using unicast feedback would not be allowed to send immediate RTCP. For ASM sessions, implosion of responses is a concern, so no change is proposed to the RTCP timing rules.
In all cases, it is possible that the initial RTCP SR packet is lost. In this case, the receiver will not be able to synchronise the media until the reporting interval has passed, and the next RTCP SR packet is sent. This is undesirable. Section 3.2 defines a new RTP/AVPF transport layer feedback message to request that an RTCP SR be generated, allowing rapid resynchronisation in the case of packet loss.
2.2. Synchronisation for Late Joiners
Synchronisation between RTP sessions is potentially slower for late joiners than for participants present at the start of the session. The reasons for this are three-fold:
-
Many of the optimisations that allow rapid transmission of RTCP SR packets apply only at the start of a session. This implies that a new participant may have to wait a complete RTCP reporting interval for each session before receiving the necessary data to synchronise media streams. This might potentially take several seconds, depending on the configured session bandwidth and the number of participants.
-
Additional synchronisation delay comes from the nature of the RTCP timing rules. Packets are generated on average once per reporting interval, but with the exact transmission times being randomised +/- 50% to avoid synchronisation of reports. This is important to avoid network congestion in multicast sessions, but does mean that the timing of RTCP sender reports for different RTP sessions isn't synchronised. Accordingly, a receiver must estimate the skew on the NTP-format clock in order to align RTP timestamps across sessions. This estimation is an essential part of an RTP synchronisation implementation, and can be done with high accuracy given sufficient reports. Collecting sufficient RTCP SR data to perform this estimation, however, may require reception of several RTCP reports, further increasing the synchronisation delay.
-
Many media codecs have the notion of periodic access points, such that a newly joined receiver often cannot start decoding a media stream until the packets corresponding to the access point have been received. These access points may be sent less often than RTCP SR packets, and so may be the limiting factor in starting synchronised media playout for late joiners. The RTP extension for unicast-based rapid acquisition of multicast RTP sessions [AVT-ACQUISITION-RTP] may be used to reduce the time taken to receive the access points in some scenarios.
These delays are likely an issue for tuning in to an ongoing multicast RTP session, or for video switching MCUs.