From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mail.toke.dk; spf=pass (mailfrom) smtp.mailfrom=irif.fr (client-ip=2001:660:3301:8000::1:2; helo=korolev.univ-paris7.fr; envelope-from=jch@irif.fr; receiver=) Received: from korolev.univ-paris7.fr (korolev.univ-paris7.fr [IPv6:2001:660:3301:8000::1:2]) by mail.toke.dk (Postfix) with ESMTPS id D78757C8AEA; Sun, 10 Jan 2021 16:14:45 +0100 (CET) Received: from potemkin.univ-paris7.fr (potemkin.univ-paris7.fr [IPv6:2001:660:3301:8000::1:1]) by korolev.univ-paris7.fr (8.14.4/8.14.4/relay1/82085) with ESMTP id 10AFEiY4001899 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 10 Jan 2021 16:14:44 +0100 Received: from mailhub.math.univ-paris-diderot.fr (mailhub.math.univ-paris-diderot.fr [81.194.30.253]) by potemkin.univ-paris7.fr (8.14.4/8.14.4/relay2/82085) with ESMTP id 10AFEiLA006306; Sun, 10 Jan 2021 16:14:44 +0100 Received: from mailhub.math.univ-paris-diderot.fr (localhost [127.0.0.1]) by mailhub.math.univ-paris-diderot.fr (Postfix) with ESMTP id D48A222705; Sun, 10 Jan 2021 16:14:44 +0100 (CET) X-Virus-Scanned: amavisd-new at math.univ-paris-diderot.fr Received: from mailhub.math.univ-paris-diderot.fr ([127.0.0.1]) by mailhub.math.univ-paris-diderot.fr (mailhub.math.univ-paris-diderot.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id XxLHZvrjTkpv; Sun, 10 Jan 2021 16:14:42 +0100 (CET) Received: from pirx.irif.fr (unknown [78.194.40.74]) (Authenticated sender: jch) by mailhub.math.univ-paris-diderot.fr (Postfix) with ESMTPSA id 172AD22703; Sun, 10 Jan 2021 16:14:42 +0100 (CET) Date: Sun, 10 Jan 2021 16:14:42 +0100 Message-ID: <87lfd04ye5.wl-jch@irif.fr> From: Juliusz Chroboczek To: Toke =?ISO-8859-1?Q?H=F8iland-J=F8rgensen?= In-Reply-To: <877dokhpja.fsf@toke.dk> References: <875z485swt.wl-jch@irif.fr> <87wnwnha9w.fsf@toke.dk> <87v9c77e20.wl-jch@irif.fr> <87turrh6y2.fsf@toke.dk> <87mtxj789e.wl-jch@irif.fr> <87o8hzgr4j.fsf@toke.dk> <871rev6w4a.wl-jch@irif.fr> <87im87gdy3.fsf@toke.dk> <87wnwmnsj9.wl-jch@irif.fr> <877dokhpja.fsf@toke.dk> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/27.1 Mule/6.0 MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=ISO-8859-1 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (korolev.univ-paris7.fr [IPv6:2001:660:3301:8000::1:2]); Sun, 10 Jan 2021 16:14:44 +0100 (CET) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (potemkin.univ-paris7.fr [194.254.61.141]); Sun, 10 Jan 2021 16:14:44 +0100 (CET) X-Miltered: at korolev with ID 5FFB19E4.000 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-Miltered: at potemkin with ID 5FFB19E4.001 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 5FFB19E4.000 from potemkin.univ-paris7.fr/potemkin.univ-paris7.fr/null/potemkin.univ-paris7.fr/ X-j-chkmail-Enveloppe: 5FFB19E4.001 from mailhub.math.univ-paris-diderot.fr/mailhub.math.univ-paris-diderot.fr/null/mailhub.math.univ-paris-diderot.fr/ X-j-chkmail-Score: MSGID : 5FFB19E4.000 on korolev.univ-paris7.fr : j-chkmail score : . : R=. U=. O=. B=0.000 -> S=0.000 X-j-chkmail-Score: MSGID : 5FFB19E4.001 on potemkin.univ-paris7.fr : j-chkmail score : . : R=. U=. O=. B=0.000 -> S=0.000 X-j-chkmail-Status: Ham X-j-chkmail-Status: Ham Content-Transfer-Encoding: quoted-printable Message-ID-Hash: 7APDFIK4KMFM2NEMJ5WG6RUZFCK2FUKA X-Message-ID-Hash: 7APDFIK4KMFM2NEMJ5WG6RUZFCK2FUKA X-MailFrom: jch@irif.fr X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: galene@lists.galene.org X-Mailman-Version: 3.3.2 Precedence: list Subject: [Galene] Congestion control and WebRTC [was: Logging] List-Id: =?utf-8?q?Gal=C3=A8ne_videoconferencing_server_discussion_list?= Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: > No, I don't think multiplexing more streams over the same five-tuple is > a good idea if it can be avoided. If the bottleneck does per-flow > queueing (like FQ-CoDel), you'd want each video flow to be scheduled > separately I think. Tere's a tradeoff here. Using port multiplexing gives more information t= o middleboxes, but using SSID multiplexing reduces the amount of ICE negotation -- adding a new track to an already established flow requires zero packet exchanges after negotiation (you just start sending data with a fresh SSID), while adding a new flow for port multiplexing requires a new set of ICE probes, which might take a few seconds in the TURN case. > Another couple of ideas for packet-level optimisations that may be wort= h > trying (both originally articulated by Dave Taht): Why is Dave not here? > In the presence of an FQ-CoDel'ed bottleneck it may be better to put > audio and video on two separate 5-tuples: That would cause the audio > stream to be treated as a 'sparse flow' with queueing priority and fewe= r > drops when congested. Uh-huh. I'll send you a patch to do that, in case you find the time to test it. > (As an aside, is there a reference for the codec constraints in > browsers? And is it possible to tweak codec parameters, say to burn som= e > bandwidth to enable really high-fidelity audio for special use cases? O= r > is Opus so good that it doesn't matter?) A typical laptop microphone has rather poor frequency response, so Opus a= t 48kbit/s is as good as the original. It's just not worth reducing the audio rate upon congestion, it's the video rate that gets reduced. As to the video rate, you've got plenty of exciting knobs. 1. Congestion control. As implemented in modern browsers, WebRTC uses tw= o congestion controllers: a fairly traditional loss-based controller, and a= n interesting delay-based one. This is described here: https://tools.ietf.org/html/draft-ietf-rmcat-gcc-02 Unlike some other video servers, that mere forward congestion indictions from the receivers to the sender, Gal=E8ne terminates congestion control = on both legs. currently obeys congestion indication from the receivers, and implements the loss-based congestion controller for data received from th= e sender. https://github.com/jech/galene/blob/master/rtpconn/rtpconn.go#L956 We do not currently implement the delay-based controller, which causes collapse if the sender is on a low-rate bufferbloated network. That is why Gal=E8ne's client limits the rate to 700kbit/s by default (in the =AB Send =BB entry in the side menu). Implementing the delay-based controller is number 1 on my wishlist. Your help would be greatly appreciated. 2. Sender-side tweaks. The sender has a number of knobs they can tweak, notably maximum bitrate (separately for each track), and hints about whether to prefer framerate or image quality upon congestion. The sender can also pick the webcam resolution, and they can request downscaling before encoding. 3. SVC. The technology that excites me right now is scalable video codin= g (SVC), which I believe will make simulcast obsolete. With VP8, the sender can request that some frames should not be used as reference for intra prediction; these =AB discardable =BB frames can be dropped by the server without causing corruption. VP9 implements full scalability: temporal scalability, as in VP8, spatial scalability, where the codec generates a low resolution flow and a high resolution flow that uses the low resolution flow for intra prediction, and quality scalability, where the codec generates frames with varying quality. https://en.wikipedia.org/wiki/Scalable_Video_Coding I'm currently planning to skip simulcasting, which I feel is an obsolescent technology, and experiment with SVC instead. Implementing th= e delay-based controller is a higher prioerity, though. > Another packet-based optimisation that could be interesting to try out > is to mark packets containing video key frames as ECN-capable. Keyframes can be huge (120 packets is not unusual), it wouldn't be resonable to mark such a burst as ECN-capable without actually reacting t= o CE. And if we drop part of the keyframe, we'll NACK the missing packets and recover 20ms + 1RTT later. > Ideally you'd also actually respond to CE markings, RFC 6679. I don't know if it's implemented in browsers. -- Juliusz