Galène videoconferencing server discussion list archives
 help / color / mirror / Atom feed
From: Dave Taht <dave.taht@gmail.com>
To: "Toke Høiland-Jørgensen" <toke@toke.dk>
Cc: Juliusz Chroboczek <jch@irif.fr>, galene@lists.galene.org
Subject: [Galene] Re: Congestion control and WebRTC [was: Logging]
Date: Sun, 10 Jan 2021 16:30:03 -0800	[thread overview]
Message-ID: <CAA93jw4rr_6cPNH+eiQibJLuT2vD+fxTNa-MrJMAMcS52ECw3Q@mail.gmail.com> (raw)
In-Reply-To: <87eeismnwy.fsf@toke.dk>

On Sun, Jan 10, 2021 at 2:23 PM Toke Høiland-Jørgensen <toke@toke.dk> wrote:

> >> In the presence of an FQ-CoDel'ed bottleneck it may be better to put
> >> audio and video on two separate 5-tuples: That would cause the audio
> >> stream to be treated as a 'sparse flow' with queueing priority and fewer
> >> drops when congested.
> >
> > Uh-huh.  I'll send you a patch to do that, in case you find the time to
> > test it.
>
> Sounds good, thanks!
>
> >> (As an aside, is there a reference for the codec constraints in
> >> browsers? And is it possible to tweak codec parameters, say to burn some
> >> bandwidth to enable really high-fidelity audio for special use cases? Or
> >> is Opus so good that it doesn't matter?)
> >
> > A typical laptop microphone has rather poor frequency response, so Opus at
> > 48kbit/s is as good as the original.  It's just not worth reducing the
> > audio rate upon congestion, it's the video rate that gets reduced.

There have been some good work (across town) on jacktrip, for
multi-party collaborative audio.

https://www.npr.org/2020/11/21/937043051/musicians-turn-to-new-software-to-play-together-online

but zoom and google etc simply don't cut it.

I am personally extremely interested in high quality, positional
collaborative 3D audio, with the video component highly secondary.

Here's the low end device I've been using of late, mounted to the roof
of my boat.

https://zoomcorp.com/en/us/handheld-recorders/handheld-recorders/h3-vr-360-audio-recorder/

An example of what can be achieved with it, mixed down to binaural,
instead of 5:1
http://www.taht.net/~d/circle/wish_youwerehere_Binaural.mp3

You can clearly hear the airplane go *overhead*, totally
coincidentally. in the outro. I note that I'm not in best voice on
that recording, and one of the big songs I've mostly been recording
this year is over here: https://www.youtube.com/watch?v=tUun-jFFoU4



>
> Right, I see. Looking at the commit that introduced codec support, it
> looks pretty straight-forward to crank up the bitrate; maybe I'll
> experiment with that a bit (but not using my laptop's microphone).

Cool. What comes out of that device is 4 channels of various
ambisonics encodings. Whether opus can feed that into a decoder on the
other side (imagine trying to sit next to the drummer, or piano
player) dunno.

> > As to the video rate, you've got plenty of exciting knobs.
> >
> > 1. Congestion control.  As implemented in modern browsers, WebRTC uses two
> > congestion controllers: a fairly traditional loss-based controller, and an
> > interesting delay-based one.  This is described here:
> >
> >   https://tools.ietf.org/html/draft-ietf-rmcat-gcc-02

I liked this one. Except that I wanted to add either classic or SCE
based ecn to the whole thing.

> > Unlike some other video servers, that mere forward congestion indictions
> > from the receivers to the sender, Galène terminates congestion control on
> > both legs.  currently obeys congestion indication from the receivers, and
> > implements the loss-based congestion controller for data received from the
> > sender.
> >
> >   https://github.com/jech/galene/blob/master/rtpconn/rtpconn.go#L956
> >
> > We do not currently implement the delay-based controller, which causes
> > collapse if the sender is on a low-rate bufferbloated network.  That is
> > why Galène's client limits the rate to 700kbit/s by default (in the
> > « Send » entry in the side menu).
>
> Right, but the browsers do?
>
> > Implementing the delay-based controller is number 1 on my wishlist.  Your
> > help would be greatly appreciated.
>
> Can't promise any hacking time, unfortunately, at least not short-term.
> Happy to test out stuff, though :)
>
> > 2. Sender-side tweaks.  The sender has a number of knobs they can tweak,
> > notably maximum bitrate (separately for each track), and hints about
> > whether to prefer framerate or image quality upon congestion.  The sender
> > can also pick the webcam resolution, and they can request downscaling
> > before encoding.
>
> Ah, hence the "blackboard mode" - gotcha!
>
> > 3. SVC.  The technology that excites me right now is scalable video coding
> > (SVC), which I believe will make simulcast obsolete.  With VP8, the
> > sender can request that some frames should not be used as reference for
> > intra prediction; these « discardable » frames can be dropped by the
> > server without causing corruption.  VP9 implements full scalability:
> > temporal scalability, as in VP8, spatial scalability, where the codec
> > generates a low resolution flow and a high resolution flow that uses the
> > low resolution flow for intra prediction, and quality scalability, where
> > the codec generates frames with varying quality.
> >
> >   https://en.wikipedia.org/wiki/Scalable_Video_Coding
> >
> > I'm currently planning to skip simulcasting, which I feel is an
> > obsolescent technology, and experiment with SVC instead.  Implementing the
> > delay-based controller is a higher prioerity, though.
>
> Uh, hadn't heard about that before; neat!
>
> >> Another packet-based optimisation that could be interesting to try out
> >> is to mark packets containing video key frames as ECN-capable.
> >
> > Keyframes can be huge (120 packets is not unusual), it wouldn't be
> > resonable to mark such a burst as ECN-capable without actually reacting to
> > CE.  And if we drop part of the keyframe, we'll NACK the missing packets
> > and recover 20ms + 1RTT later.
>
> Hmm, right, okay I see what you mean...
>
> >>> Ideally you'd also actually respond to CE markings,
> >> RFC 6679.  I don't know if it's implemented in browsers.
> >
> > It is not.
>
> Ah, too bad :(
>
> -Toke



-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

  parent reply	other threads:[~2021-01-11  0:30 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-07 21:38 [Galene] Logging Juliusz Chroboczek
2021-01-07 22:45 ` [Galene] Logging Michael Ströder
2021-01-08  0:35 ` Antonin Décimo
2021-01-08 12:40 ` Toke Høiland-Jørgensen
2021-01-08 13:28   ` Juliusz Chroboczek
2021-01-08 13:52     ` Toke Høiland-Jørgensen
2021-01-08 14:33       ` Michael Ströder
2021-01-08 15:13         ` Juliusz Chroboczek
2021-01-08 17:34           ` Michael Ströder
2021-01-08 18:00             ` Juliusz Chroboczek
2021-01-08 15:34       ` Juliusz Chroboczek
2021-01-08 19:34         ` Toke Høiland-Jørgensen
2021-01-08 19:56           ` Juliusz Chroboczek
2021-01-09  0:18             ` Toke Høiland-Jørgensen
2021-01-09 13:34               ` Juliusz Chroboczek
2021-01-10 13:47                 ` Toke Høiland-Jørgensen
2021-01-10 15:14                   ` [Galene] Congestion control and WebRTC [was: Logging] Juliusz Chroboczek
2021-01-10 15:23                     ` [Galene] " Juliusz Chroboczek
2021-01-10 22:23                     ` Toke Høiland-Jørgensen
2021-01-10 22:44                       ` Dave Taht
2021-01-11  0:07                       ` Juliusz Chroboczek
2021-01-11  0:20                         ` Toke Høiland-Jørgensen
2021-01-11  0:28                           ` Juliusz Chroboczek
2021-01-11  0:30                       ` Dave Taht [this message]
2021-01-11  6:23                         ` Dave Taht
2021-01-11 12:55                           ` [Galene] Multichannel audio [was: Congestion control...] Juliusz Chroboczek
2021-01-11 17:25                             ` [Galene] " Dave Taht
2021-01-11 13:38                       ` [Galene] Re: Congestion control and WebRTC [was: Logging] Juliusz Chroboczek
2021-01-11 15:17                         ` Toke Høiland-Jørgensen
2021-01-11 17:20                           ` Dave Taht
2021-01-12  1:38                         ` Juliusz Chroboczek
2021-01-10 15:17                   ` [Galene] Re: Logging Juliusz Chroboczek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.galene.org/postorius/lists/galene.lists.galene.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAA93jw4rr_6cPNH+eiQibJLuT2vD+fxTNa-MrJMAMcS52ECw3Q@mail.gmail.com \
    --to=dave.taht@gmail.com \
    --cc=galene@lists.galene.org \
    --cc=jch@irif.fr \
    --cc=toke@toke.dk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox