From: Dave Taht <dave.taht@gmail.com> To: Juliusz Chroboczek <jch@irif.fr> Cc: galene@lists.galene.org, Dave Taht <dave@taht.net> Subject: [Galene] Re: fq-codel trashing Date: Tue, 12 Jan 2021 11:05:48 -0800 [thread overview] Message-ID: <CAA93jw7Tg+7_210BD4-gAAjuEw5kVt=axybA5M14EJk8qGOe1w@mail.gmail.com> (raw) In-Reply-To: <871reqqb52.wl-jch@irif.fr> On Tue, Jan 12, 2021 at 10:10 AM Juliusz Chroboczek <jch@irif.fr> wrote: > > > IF you have spare cpu, you can also run cake native no shaper and > > strip gro (maxpacket 64k indicates gro is on). > > GRO will only trigger for HTTP, WebSocket and TURN traffic, I believe; > most of Galène's traffic is RTP over UDP, with packets under 1200 bytes. > > We just had a meeting with 70 people and at around 40 cameras switched on, > Galène became unusable — there were too many voice drops, which indicates > two issues: i have generally found vms perform badly for lots of small packets and r/t. please try cake. And collect a capture on the underlying hw if possible. > * I need to think of a better way of prioritising voice over video when > under load; I take it (I really didn't understand) that unbundling these two types is not currently feasible in the javascript or webbrowser. ?Still. I am pretty intersted in this layer of stuff but since adding crypto to rtp it's got really hard to look at it. > * there are fairness issues — some clients were receiving okay-ish > audio, others were not. collisions in fq_codel start to occur as the sqrt(1024) so at 70 users the odds are 2-3 of your sessions were colliding. Cake uses 8 way set associative. 0 collisions at this load. I imagine you were not tracking the actual backlog in fq_codel during this conference? (while :; do tc -s qdisc show dev bla >> whatever; done all that said, I tend to point fingers at loss at the virtio layer and underlying hw first just because I'm defensive about people blaming fq_codel for anything. :) I also hate vms for r/t traffic. you had a lot of GRO traffic it looked like. you can disable gso/gro via ethtool and stick with fq_codel or use cake with the > > Galène recovered after some people switched their cameras off, I didn't > need to restart anything. At the highest point, Galène was at 270% CPU, > and the TURN server was using another 50%. That's on a four-core VM. but it does sound like more virtual cores will help at this load? you in a position to profile? you are context switching like crazy most likely also. > > On the other hand, gro is saves on context switches in vms. > > $ PRODUCT INSTALL GALENE /CONFIGURATION=DEFAULT /LOG > > -- Juliusz -- "For a successful technology, reality must take precedence over public relations, for Mother Nature cannot be fooled" - Richard Feynman dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729
next prev parent reply other threads:[~2021-01-12 19:06 UTC|newest] Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-01-12 13:46 [Galene] " Juliusz Chroboczek 2021-01-12 15:55 ` [Galene] " Toke Høiland-Jørgensen 2021-01-12 16:01 ` Dave Taht 2021-01-12 17:38 ` Juliusz Chroboczek 2021-01-12 17:42 ` Dave Taht 2021-01-12 18:10 ` Juliusz Chroboczek 2021-01-12 19:05 ` Dave Taht [this message] 2021-01-12 19:52 ` Michael Ströder 2021-01-12 21:02 ` Juliusz Chroboczek 2021-01-12 19:29 ` Michael Ströder 2021-01-12 21:22 ` Juliusz Chroboczek 2021-01-13 19:09 ` Michael Ströder 2021-01-14 12:59 ` Juliusz Chroboczek 2021-01-14 13:03 ` Michael Ströder 2021-01-14 13:10 ` Juliusz Chroboczek 2021-01-14 13:23 ` Michael Ströder
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: https://lists.galene.org/postorius/lists/galene.lists.galene.org/ * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CAA93jw7Tg+7_210BD4-gAAjuEw5kVt=axybA5M14EJk8qGOe1w@mail.gmail.com' \ --to=dave.taht@gmail.com \ --cc=dave@taht.net \ --cc=galene@lists.galene.org \ --cc=jch@irif.fr \ --subject='[Galene] Re: fq-codel trashing' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox