'Synchronized audio playback across multiple receivers with RTP
I am trying to perform synchronized playback of fixed opus audio files across multiple PCs in my house. I am using gstreamer, rtp, and rtpbin. My mental model of how this is supposed to work is,
at the start of playback, the sender randomly generates A) an ssrc id, and B) an initial timestamp
rtp
sender -> receiverpackets contain the ssrc id (A) and a timestamp field proceeding forwards from (B)rtcp
sender -> receiverpackets contain the ssrc id (A), and C) an offset from the randomly generated timestamp (B) to its actual system clock plus D) some buffer to account for network latencyeach receiver plays the samples at the adjusted offset, according to its own system clock
the sender and receiver machines have separately synchronized their clocks (I am using ntp)
thus, the separate receivers play each sample at the same time
Here is my sender pipeline.
#
# send a timestamped audio stream
#
# opus file, demux the opus container, parse the opus packet stream, wrap opus frames into rtp packets
# the rtpbin element combines
# 1. outputs a stream of rtp packets (send_rtp_src_%d / send_rtp_sink_%d pads)
# 2. outputs a stream of rtcp packets (the send_rtcp_src_%d pad)
#
gst-launch-1.0 rtpbin name=rtpbin \
filesrc location=rhapsody.opus ! oggdemux ! opusparse ! rtpopuspay ! rtpbin.send_rtp_sink_0 \
rtpbin.send_rtp_src_0 ! multiudpsink clients=192.168.99.10:12345,192.168.99.11:12345 \
rtpbin.send_rtcp_src_0 ! multiudpsink clients=192.168.99.10:12346,192.168.99.11:12346 sync=false async=false # I dont know why this sync and async are doing
And here is the receiver pipeline
gst-launch-1.0 rtpbin name=rtpbin \
udpsrc port=12345 caps='application/x-rtp,media=(string)audio,clock-rate=(int)48000,encoding-name=(string)OPUS' ! rtpbin.recv_rtp_sink_0 \
udpsrc port=12346 caps='application/x-rtcp' ! rtpbin.recv_rtcp_sink_0 \
rtpbin. ! rtpopusdepay ! opusdec ! pulsesink sync=true # I dont know what this sync is doing
Now, when I run these pipelines, the audio plays on all of the receivers. But I do not think it is actually synchronized, I just have a low latency LAN.
In order to demonstrate this, I adjusted the system clock of one of the receivers ahead by 2 seconds, and disabled the ntp daemon. Playback remains synchronized.
What am I missing?
I have tried a variety of the rtpbin parameters (https://gstreamer.freedesktop.org/documentation/rtpmanager/rtpbin.html?gi-language=c) ntp-sync, ntp-time-source, buffer-mode, and ntp-sync-send-time but none of these combinations give the expected results.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
