Galène videoconferencing server discussion list archives
 help / color / mirror / Atom feed
From: Dave Taht <dave.taht@gmail.com>
To: "Toke Høiland-Jørgensen" <toke@toke.dk>
Cc: Juliusz Chroboczek <jch@irif.fr>, galene@lists.galene.org
Subject: [Galene] Re: Congestion control and WebRTC [was: Logging]
Date: Sun, 10 Jan 2021 22:23:56 -0800	[thread overview]
Message-ID: <CAA93jw73KfGkFv3k+XddZaRA5+mH6Fok4R3Yw60Z6tvMgrc4QQ@mail.gmail.com> (raw)
In-Reply-To: <CAA93jw4rr_6cPNH+eiQibJLuT2vD+fxTNa-MrJMAMcS52ECw3Q@mail.gmail.com>

On Sun, Jan 10, 2021 at 4:30 PM Dave Taht <dave.taht@gmail.com> wrote:
>
> On Sun, Jan 10, 2021 at 2:23 PM Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>
> > >> In the presence of an FQ-CoDel'ed bottleneck it may be better to put
> > >> audio and video on two separate 5-tuples: That would cause the audio
> > >> stream to be treated as a 'sparse flow' with queueing priority and fewer
> > >> drops when congested.
> > >
> > > Uh-huh.  I'll send you a patch to do that, in case you find the time to
> > > test it.
> >
> > Sounds good, thanks!
> >
> > >> (As an aside, is there a reference for the codec constraints in
> > >> browsers? And is it possible to tweak codec parameters, say to burn some
> > >> bandwidth to enable really high-fidelity audio for special use cases? Or
> > >> is Opus so good that it doesn't matter?)
> > >
> > > A typical laptop microphone has rather poor frequency response, so Opus at
> > > 48kbit/s is as good as the original.  It's just not worth reducing the
> > > audio rate upon congestion, it's the video rate that gets reduced.
>
> There have been some good work (across town) on jacktrip, for
> multi-party collaborative audio.
>
> https://www.npr.org/2020/11/21/937043051/musicians-turn-to-new-software-to-play-together-online
>
> but zoom and google etc simply don't cut it.
>
> I am personally extremely interested in high quality, positional
> collaborative 3D audio, with the video component highly secondary.
>
> Here's the low end device I've been using of late, mounted to the roof
> of my boat.
>
> https://zoomcorp.com/en/us/handheld-recorders/handheld-recorders/h3-vr-360-audio-recorder/
>
> An example of what can be achieved with it, mixed down to binaural,
> instead of 5:1
> http://www.taht.net/~d/circle/wish_youwerehere_Binaural.mp3
>
> You can clearly hear the airplane go *overhead*, totally
> coincidentally. in the outro. I note that I'm not in best voice on
> that recording, and one of the big songs I've mostly been recording
> this year is over here: https://www.youtube.com/watch?v=tUun-jFFoU4
>
>
>
> >
> > Right, I see. Looking at the commit that introduced codec support, it
> > looks pretty straight-forward to crank up the bitrate; maybe I'll
> > experiment with that a bit (but not using my laptop's microphone).
>
> Cool. What comes out of that device is 4 channels of various
> ambisonics encodings. Whether opus can feed that into a decoder on the
> other side (imagine trying to sit next to the drummer, or piano
> player) dunno.

Can galene, or a web browser, carry multichannel opus audio?
>
> > > As to the video rate, you've got plenty of exciting knobs.
> > >
> > > 1. Congestion control.  As implemented in modern browsers, WebRTC uses two
> > > congestion controllers: a fairly traditional loss-based controller, and an
> > > interesting delay-based one.  This is described here:
> > >
> > >   https://tools.ietf.org/html/draft-ietf-rmcat-gcc-02
>
> I liked this one. Except that I wanted to add either classic or SCE
> based ecn to the whole thing.
>
> > > Unlike some other video servers, that mere forward congestion indictions
> > > from the receivers to the sender, Galène terminates congestion control on
> > > both legs.  currently obeys congestion indication from the receivers, and
> > > implements the loss-based congestion controller for data received from the
> > > sender.
> > >
> > >   https://github.com/jech/galene/blob/master/rtpconn/rtpconn.go#L956
> > >
> > > We do not currently implement the delay-based controller, which causes
> > > collapse if the sender is on a low-rate bufferbloated network.  That is
> > > why Galène's client limits the rate to 700kbit/s by default (in the
> > > « Send » entry in the side menu).
> >
> > Right, but the browsers do?
> >
> > > Implementing the delay-based controller is number 1 on my wishlist.  Your
> > > help would be greatly appreciated.
> >
> > Can't promise any hacking time, unfortunately, at least not short-term.
> > Happy to test out stuff, though :)
> >
> > > 2. Sender-side tweaks.  The sender has a number of knobs they can tweak,
> > > notably maximum bitrate (separately for each track), and hints about
> > > whether to prefer framerate or image quality upon congestion.  The sender
> > > can also pick the webcam resolution, and they can request downscaling
> > > before encoding.
> >
> > Ah, hence the "blackboard mode" - gotcha!
> >
> > > 3. SVC.  The technology that excites me right now is scalable video coding
> > > (SVC), which I believe will make simulcast obsolete.  With VP8, the
> > > sender can request that some frames should not be used as reference for
> > > intra prediction; these « discardable » frames can be dropped by the
> > > server without causing corruption.  VP9 implements full scalability:
> > > temporal scalability, as in VP8, spatial scalability, where the codec
> > > generates a low resolution flow and a high resolution flow that uses the
> > > low resolution flow for intra prediction, and quality scalability, where
> > > the codec generates frames with varying quality.
> > >
> > >   https://en.wikipedia.org/wiki/Scalable_Video_Coding
> > >
> > > I'm currently planning to skip simulcasting, which I feel is an
> > > obsolescent technology, and experiment with SVC instead.  Implementing the
> > > delay-based controller is a higher prioerity, though.
> >
> > Uh, hadn't heard about that before; neat!
> >
> > >> Another packet-based optimisation that could be interesting to try out
> > >> is to mark packets containing video key frames as ECN-capable.
> > >
> > > Keyframes can be huge (120 packets is not unusual), it wouldn't be
> > > resonable to mark such a burst as ECN-capable without actually reacting to
> > > CE.  And if we drop part of the keyframe, we'll NACK the missing packets
> > > and recover 20ms + 1RTT later.
> >
> > Hmm, right, okay I see what you mean...
> >
> > >>> Ideally you'd also actually respond to CE markings,
> > >> RFC 6679.  I don't know if it's implemented in browsers.
> > >
> > > It is not.
> >
> > Ah, too bad :(
> >
> > -Toke
>
>
>
> --
> "For a successful technology, reality must take precedence over public
> relations, for Mother Nature cannot be fooled" - Richard Feynman
>
> dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729



-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

  reply	other threads:[~2021-01-11  6:24 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-07 21:38 [Galene] Logging Juliusz Chroboczek
2021-01-07 22:45 ` [Galene] Logging Michael Ströder
2021-01-08  0:35 ` Antonin Décimo
2021-01-08 12:40 ` Toke Høiland-Jørgensen
2021-01-08 13:28   ` Juliusz Chroboczek
2021-01-08 13:52     ` Toke Høiland-Jørgensen
2021-01-08 14:33       ` Michael Ströder
2021-01-08 15:13         ` Juliusz Chroboczek
2021-01-08 17:34           ` Michael Ströder
2021-01-08 18:00             ` Juliusz Chroboczek
2021-01-08 15:34       ` Juliusz Chroboczek
2021-01-08 19:34         ` Toke Høiland-Jørgensen
2021-01-08 19:56           ` Juliusz Chroboczek
2021-01-09  0:18             ` Toke Høiland-Jørgensen
2021-01-09 13:34               ` Juliusz Chroboczek
2021-01-10 13:47                 ` Toke Høiland-Jørgensen
2021-01-10 15:14                   ` [Galene] Congestion control and WebRTC [was: Logging] Juliusz Chroboczek
2021-01-10 15:23                     ` [Galene] " Juliusz Chroboczek
2021-01-10 22:23                     ` Toke Høiland-Jørgensen
2021-01-10 22:44                       ` Dave Taht
2021-01-11  0:07                       ` Juliusz Chroboczek
2021-01-11  0:20                         ` Toke Høiland-Jørgensen
2021-01-11  0:28                           ` Juliusz Chroboczek
2021-01-11  0:30                       ` Dave Taht
2021-01-11  6:23                         ` Dave Taht [this message]
2021-01-11 12:55                           ` [Galene] Multichannel audio [was: Congestion control...] Juliusz Chroboczek
2021-01-11 17:25                             ` [Galene] " Dave Taht
2021-01-11 13:38                       ` [Galene] Re: Congestion control and WebRTC [was: Logging] Juliusz Chroboczek
2021-01-11 15:17                         ` Toke Høiland-Jørgensen
2021-01-11 17:20                           ` Dave Taht
2021-01-12  1:38                         ` Juliusz Chroboczek
2021-01-10 15:17                   ` [Galene] Re: Logging Juliusz Chroboczek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.galene.org/postorius/lists/galene.lists.galene.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAA93jw73KfGkFv3k+XddZaRA5+mH6Fok4R3Yw60Z6tvMgrc4QQ@mail.gmail.com \
    --to=dave.taht@gmail.com \
    --cc=galene@lists.galene.org \
    --cc=jch@irif.fr \
    --cc=toke@toke.dk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox