Galène videoconferencing server discussion list archives
 help / color / mirror / Atom feed
From: Dave Taht <dave.taht@gmail.com>
To: "Toke Høiland-Jørgensen" <toke@toke.dk>
Cc: Juliusz Chroboczek <jch@irif.fr>,
	galene@lists.galene.org, Dave Taht <dave@taht.net>
Subject: [Galene] Re: fq-codel trashing
Date: Tue, 12 Jan 2021 08:01:18 -0800	[thread overview]
Message-ID: <CAA93jw7Fa8pxfgPL2LtK270Xpvn6nk13w=jW4SSzKJcy5tQgJw@mail.gmail.com> (raw)
In-Reply-To: <87pn2aup3o.fsf@toke.dk>

On Tue, Jan 12, 2021 at 7:55 AM Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>
> Juliusz Chroboczek <jch@irif.fr> writes:
>
> > Dave, Toke,
> >
> > We're having a large meeting on the lab's Galène server, and it looks like
> > fq-codel is trashing: the new_flow_count increases as fast as the packet
> > counter.  How do I fix that?

It is a pretty useless statistic. I never liked it. In this case it is
a positive sign you are
not filling the queue, not a negative one, as toke goes into more detail below.

If you are having fun with fq_codel, try cake instead. Lots of more stats,
some of which are way more relevant to your application.

tc qdisc add dev eth0 root cake bandwidth XMbit # besteffort if you
don't want to bother with diffserv

tc -s qdisc show


                   Bulk  Best Effort        Voice
  thresh      562496bit        9Mbit     2250Kbit
  target         32.3ms        5.0ms        8.1ms
  interval      127.3ms      100.0ms      103.1ms
  pk_delay        9.2ms        224us        494us
  av_delay        2.9ms         29us         35us
  sp_delay        153us          4us          3us
  backlog            0b           0b           0b
  pkts            74853     35044344       162590
  bytes        80542836   9388672264     27301751
  way_inds         2538      2129502        10508
  way_miss         2575       549864         7374
  way_cols            0            0            0
  drops              11         7987            0
  marks              22            1            0
  ack_drop            0      3616230            0
  sp_flows            1            1            1
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len         15605        25262         3169
  quantum           300          300          300





>
> That just sounds like wherever you're running FQ-CoDel is not actually
> the bottleneck? If there's no backpressure the queue will drain for each
> packet, so that by the time the next packet comes along it'll be
> considered "new" again.
>
> If you're running a few video flows on a gigabit link (and thus using a
> fraction of the bandwidth) that would be expected. But it only happens
> if you're also running a shaper (such as HTB or TBF) on the interface,
> since if FQ-CoDel is installed on a physical interface it will set the
> TCQ_F_CAN_BYPASS flag, which means that when the queue is completely
> empty it will be bypassed entirely...
>
> I just replicated the phenomenon like this:
>
> $ sudo tc qdisc replace dev testns root tbf rate 1gbit latency 10ms burst 1024
> $ sudo tc qdisc add dev testns parent 8002: fq_codel
> $ tc -s qdisc
> qdisc tbf 8002: dev testns root refcnt 2 rate 1Gbit burst 1000b lat 10ms
>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
> qdisc fq_codel 8003: dev testns parent 8002: limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
>   maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>   new_flows_len 0 old_flows_len 0
>
> $ ping -c 5 fc00:dead:cafe:1::2
> PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes
> 64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.088 ms
> 64 bytes from fc00:dead:cafe:1::2: icmp_seq=2 ttl=64 time=0.138 ms
> 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3 ttl=64 time=0.102 ms
> 64 bytes from fc00:dead:cafe:1::2: icmp_seq=4 ttl=64 time=0.103 ms
> 64 bytes from fc00:dead:cafe:1::2: icmp_seq=5 ttl=64 time=0.104 ms
>
> --- fc00:dead:cafe:1::2 ping statistics ---
> 5 packets transmitted, 5 received, 0% packet loss, time 4041ms
> rtt min/avg/max/mdev = 0.088/0.107/0.138/0.016 ms
>
> $ tc -s qdisc
> qdisc tbf 8002: dev testns root refcnt 2 rate 1Gbit burst 1000b lat 10ms
>  Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
> qdisc fq_codel 8003: dev testns parent 8002: limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
>  Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
>   maxpacket 118 drop_overlimit 0 new_flow_count 5 ecn_mark 0
>   new_flows_len 0 old_flows_len 0
>
>
>
> Whereas if fq_codel is the root qdisc:
>
> $ sudo tc qdisc replace dev testns root fq_codel
> $ tc -s qdisc
> qdisc fq_codel 8004: dev testns root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
>   maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>   new_flows_len 0 old_flows_len 0
>
> $ ping -c 5 fc00:dead:cafe:1::2
> PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes
> 64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.085 ms
> 64 bytes from fc00:dead:cafe:1::2: icmp_seq=2 ttl=64 time=0.098 ms
> 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3 ttl=64 time=0.098 ms
> 64 bytes from fc00:dead:cafe:1::2: icmp_seq=4 ttl=64 time=0.097 ms
> 64 bytes from fc00:dead:cafe:1::2: icmp_seq=5 ttl=64 time=0.096 ms
>
> --- fc00:dead:cafe:1::2 ping statistics ---
> 5 packets transmitted, 5 received, 0% packet loss, time 4058ms
>
> $ tc -s qdisc
> qdisc fq_codel 8004: dev testns root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
>  Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
>   maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>   new_flows_len 0 old_flows_len 0
>
>
> -Toke
> _______________________________________________
> Galene mailing list -- galene@lists.galene.org
> To unsubscribe send an email to galene-leave@lists.galene.org



-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

  reply	other threads:[~2021-01-12 16:01 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-12 13:46 [Galene] " Juliusz Chroboczek
2021-01-12 15:55 ` [Galene] " Toke Høiland-Jørgensen
2021-01-12 16:01   ` Dave Taht [this message]
2021-01-12 17:38   ` Juliusz Chroboczek
2021-01-12 17:42     ` Dave Taht
2021-01-12 18:10       ` Juliusz Chroboczek
2021-01-12 19:05         ` Dave Taht
2021-01-12 19:52           ` Michael Ströder
2021-01-12 21:02           ` Juliusz Chroboczek
2021-01-12 19:29         ` Michael Ströder
2021-01-12 21:22           ` Juliusz Chroboczek
2021-01-13 19:09             ` Michael Ströder
2021-01-14 12:59               ` Juliusz Chroboczek
2021-01-14 13:03                 ` Michael Ströder
2021-01-14 13:10                   ` Juliusz Chroboczek
2021-01-14 13:23                     ` Michael Ströder

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.galene.org/postorius/lists/galene.lists.galene.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAA93jw7Fa8pxfgPL2LtK270Xpvn6nk13w=jW4SSzKJcy5tQgJw@mail.gmail.com' \
    --to=dave.taht@gmail.com \
    --cc=dave@taht.net \
    --cc=galene@lists.galene.org \
    --cc=jch@irif.fr \
    --cc=toke@toke.dk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox