'Synchronized audio playback across multiple receivers with RTP

I am trying to perform synchronized playback of fixed opus audio files across multiple PCs in my house. I am using gstreamer, rtp, and rtpbin. My mental model of how this is supposed to work is,

  • at the start of playback, the sender randomly generates A) an ssrc id, and B) an initial timestamp

  • rtp sender -> receiver packets contain the ssrc id (A) and a timestamp field proceeding forwards from (B)

  • rtcp sender -> receiver packets contain the ssrc id (A), and C) an offset from the randomly generated timestamp (B) to its actual system clock plus D) some buffer to account for network latency

  • each receiver plays the samples at the adjusted offset, according to its own system clock

  • the sender and receiver machines have separately synchronized their clocks (I am using ntp)

  • thus, the separate receivers play each sample at the same time

Here is my sender pipeline.

#
# send a timestamped audio stream
#
# opus file, demux the opus container, parse the opus packet stream, wrap opus frames into rtp packets
#  the rtpbin element combines
#   1. outputs a stream of rtp packets (send_rtp_src_%d / send_rtp_sink_%d pads)
#   2. outputs a stream of rtcp packets (the send_rtcp_src_%d pad)
#

gst-launch-1.0 rtpbin name=rtpbin \
  filesrc location=rhapsody.opus ! oggdemux ! opusparse ! rtpopuspay ! rtpbin.send_rtp_sink_0  \
  rtpbin.send_rtp_src_0 ! multiudpsink clients=192.168.99.10:12345,192.168.99.11:12345 \
  rtpbin.send_rtcp_src_0 ! multiudpsink clients=192.168.99.10:12346,192.168.99.11:12346 sync=false async=false  # I dont know why this sync and async are doing

And here is the receiver pipeline

gst-launch-1.0 rtpbin name=rtpbin \
  udpsrc port=12345 caps='application/x-rtp,media=(string)audio,clock-rate=(int)48000,encoding-name=(string)OPUS' ! rtpbin.recv_rtp_sink_0 \
  udpsrc port=12346 caps='application/x-rtcp' ! rtpbin.recv_rtcp_sink_0 \
  rtpbin. ! rtpopusdepay ! opusdec ! pulsesink sync=true # I dont know what this sync is doing

Now, when I run these pipelines, the audio plays on all of the receivers. But I do not think it is actually synchronized, I just have a low latency LAN.

In order to demonstrate this, I adjusted the system clock of one of the receivers ahead by 2 seconds, and disabled the ntp daemon. Playback remains synchronized.

What am I missing?

I have tried a variety of the rtpbin parameters (https://gstreamer.freedesktop.org/documentation/rtpmanager/rtpbin.html?gi-language=c) ntp-sync, ntp-time-source, buffer-mode, and ntp-sync-send-time but none of these combinations give the expected results.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source