[Galene] Re: Building a streaming gateway for Galène

[Alexandre IOOSS <erdnaxe@crans.org>]

On 4/7/21 1:19 AM, Juliusz Chroboczek wrote:>> When dumping Firefox
WebRTC traffic, I also see "Generic RTP Feeback:
>> NACK" packets, which confirms the fact that GStreamer is not doing NACK
>> in my script. I should see if it's possible to enable it.
> 
> This will improve things a lot.  You'll need to keep a history of recently
> sent packets, I don't know if gstreamer can do that automatically.
> 
> As to keyframes — you should send periodic keyframes, but you shouldn't do
> it too often (Chrome sends one every 120s).  On the other hand, you should
> send a keyframe fairly quickly (within 500ms or so) whenever you receive
> a PLI (Picture Loss Indication) RTCP packet.

I have good and bad news. The good news is that GStreamer webrtcbin
supports NACK and PLI. I added those lines:

# Enable WebRTC negative acknowledgement and FEC
transceiver_count = self.webrtc.emit("get-transceivers").len
for i in range(transceiver_count):
    transceiver = self.webrtc.emit("get-transceiver", i)
    transceiver.set_property("do-nack", True)
    transceiver.set_property("fec-type", GstWebRTC.WebRTCFECType.ULP_RED)

Now I see "Generic RTP Feeback: NACK" packets in Wireshark and the SDP
offer being sent to Galène also looks better:

[...]
a=sendrecv
a=rtpmap:97 VP8/90000
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=framerate:30
a=rtpmap:98 red/90000
a=rtpmap:99 ulpfec/90000
a=rtpmap:100 rtx/90000
a=fmtp:100 apt=98
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=97
a=ssrc-group:FID 1611068776 2492991179
[...]
a=sendrecv
a=rtpmap:96 OPUS/48000/2
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=fmtp:96 sprop-maxcapturerate=48000;sprop-stereo=1
a=rtpmap:102 red/48000
a=rtpmap:103 ulpfec/48000
a=rtpmap:104 rtx/48000
a=fmtp:104 apt=102
a=rtpmap:105 rtx/48000
a=fmtp:105 apt=96
a=ssrc-group:FID 2413482691 909406007
[...]

The bad news is that the stream is still dropping. If I print some
statistics during streaming, GStreamer reports `pli-count=0` and
`nack-count=0`, so something seems broken.

Looking the issue online I found
<https://mediasoup.discourse.group/t/broadcasting-a-vp8-rtp-stream-from-gstreamer/93/18>:
> GStreamer first looks at the sender ssrc. If it gets NULL then it
> tries with media ssrc. [...] somehow, GStreamer is getting “something”
> when looking for sender ssrc = 0, which is crazy.

Looking at the Wireshark dump, Galène seems also to use "0" as sender
SSRC, so that might be the issue. I might try to do a dirty hack
somewhere to confirm this hypothesis.

Does Galène has ulpfec support (https://github.com/pion/webrtc/issues/1418)?
Should I do PLI and ulpfec on audio and video in this specific
application or is it a terrible idea?

Thank you again for all your advice!

-- 
Alexandre Iooss