From: Dave Taht <dave.taht@gmail.com> To: "Toke Høiland-Jørgensen" <toke@toke.dk> Cc: Juliusz Chroboczek <jch@irif.fr>, galene@lists.galene.org, Dave Taht <dave@taht.net> Subject: [Galene] Re: fq-codel trashing Date: Tue, 12 Jan 2021 08:01:18 -0800 [thread overview] Message-ID: <CAA93jw7Fa8pxfgPL2LtK270Xpvn6nk13w=jW4SSzKJcy5tQgJw@mail.gmail.com> (raw) In-Reply-To: <87pn2aup3o.fsf@toke.dk> On Tue, Jan 12, 2021 at 7:55 AM Toke Høiland-Jørgensen <toke@toke.dk> wrote: > > Juliusz Chroboczek <jch@irif.fr> writes: > > > Dave, Toke, > > > > We're having a large meeting on the lab's Galène server, and it looks like > > fq-codel is trashing: the new_flow_count increases as fast as the packet > > counter. How do I fix that? It is a pretty useless statistic. I never liked it. In this case it is a positive sign you are not filling the queue, not a negative one, as toke goes into more detail below. If you are having fun with fq_codel, try cake instead. Lots of more stats, some of which are way more relevant to your application. tc qdisc add dev eth0 root cake bandwidth XMbit # besteffort if you don't want to bother with diffserv tc -s qdisc show Bulk Best Effort Voice thresh 562496bit 9Mbit 2250Kbit target 32.3ms 5.0ms 8.1ms interval 127.3ms 100.0ms 103.1ms pk_delay 9.2ms 224us 494us av_delay 2.9ms 29us 35us sp_delay 153us 4us 3us backlog 0b 0b 0b pkts 74853 35044344 162590 bytes 80542836 9388672264 27301751 way_inds 2538 2129502 10508 way_miss 2575 549864 7374 way_cols 0 0 0 drops 11 7987 0 marks 22 1 0 ack_drop 0 3616230 0 sp_flows 1 1 1 bk_flows 0 1 0 un_flows 0 0 0 max_len 15605 25262 3169 quantum 300 300 300 > > That just sounds like wherever you're running FQ-CoDel is not actually > the bottleneck? If there's no backpressure the queue will drain for each > packet, so that by the time the next packet comes along it'll be > considered "new" again. > > If you're running a few video flows on a gigabit link (and thus using a > fraction of the bandwidth) that would be expected. But it only happens > if you're also running a shaper (such as HTB or TBF) on the interface, > since if FQ-CoDel is installed on a physical interface it will set the > TCQ_F_CAN_BYPASS flag, which means that when the queue is completely > empty it will be bypassed entirely... > > I just replicated the phenomenon like this: > > $ sudo tc qdisc replace dev testns root tbf rate 1gbit latency 10ms burst 1024 > $ sudo tc qdisc add dev testns parent 8002: fq_codel > $ tc -s qdisc > qdisc tbf 8002: dev testns root refcnt 2 rate 1Gbit burst 1000b lat 10ms > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > qdisc fq_codel 8003: dev testns parent 8002: limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > > $ ping -c 5 fc00:dead:cafe:1::2 > PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.088 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=2 ttl=64 time=0.138 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3 ttl=64 time=0.102 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=4 ttl=64 time=0.103 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=5 ttl=64 time=0.104 ms > > --- fc00:dead:cafe:1::2 ping statistics --- > 5 packets transmitted, 5 received, 0% packet loss, time 4041ms > rtt min/avg/max/mdev = 0.088/0.107/0.138/0.016 ms > > $ tc -s qdisc > qdisc tbf 8002: dev testns root refcnt 2 rate 1Gbit burst 1000b lat 10ms > Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > qdisc fq_codel 8003: dev testns parent 8002: limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 > Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 118 drop_overlimit 0 new_flow_count 5 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > > > > Whereas if fq_codel is the root qdisc: > > $ sudo tc qdisc replace dev testns root fq_codel > $ tc -s qdisc > qdisc fq_codel 8004: dev testns root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > > $ ping -c 5 fc00:dead:cafe:1::2 > PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.085 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=2 ttl=64 time=0.098 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3 ttl=64 time=0.098 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=4 ttl=64 time=0.097 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=5 ttl=64 time=0.096 ms > > --- fc00:dead:cafe:1::2 ping statistics --- > 5 packets transmitted, 5 received, 0% packet loss, time 4058ms > > $ tc -s qdisc > qdisc fq_codel 8004: dev testns root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 > Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > > > -Toke > _______________________________________________ > Galene mailing list -- galene@lists.galene.org > To unsubscribe send an email to galene-leave@lists.galene.org -- "For a successful technology, reality must take precedence over public relations, for Mother Nature cannot be fooled" - Richard Feynman dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729
next prev parent reply other threads:[~2021-01-12 16:01 UTC|newest] Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-01-12 13:46 [Galene] " Juliusz Chroboczek 2021-01-12 15:55 ` [Galene] " Toke Høiland-Jørgensen 2021-01-12 16:01 ` Dave Taht [this message] 2021-01-12 17:38 ` Juliusz Chroboczek 2021-01-12 17:42 ` Dave Taht 2021-01-12 18:10 ` Juliusz Chroboczek 2021-01-12 19:05 ` Dave Taht 2021-01-12 19:52 ` Michael Ströder 2021-01-12 21:02 ` Juliusz Chroboczek 2021-01-12 19:29 ` Michael Ströder 2021-01-12 21:22 ` Juliusz Chroboczek 2021-01-13 19:09 ` Michael Ströder 2021-01-14 12:59 ` Juliusz Chroboczek 2021-01-14 13:03 ` Michael Ströder 2021-01-14 13:10 ` Juliusz Chroboczek 2021-01-14 13:23 ` Michael Ströder
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: https://lists.galene.org/postorius/lists/galene.lists.galene.org/ * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CAA93jw7Fa8pxfgPL2LtK270Xpvn6nk13w=jW4SSzKJcy5tQgJw@mail.gmail.com' \ --to=dave.taht@gmail.com \ --cc=dave@taht.net \ --cc=galene@lists.galene.org \ --cc=jch@irif.fr \ --cc=toke@toke.dk \ --subject='[Galene] Re: fq-codel trashing' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox