From: "Toke Høiland-Jørgensen" <toke@toke.dk>
To: Juliusz Chroboczek <jch@irif.fr>, galene@lists.galene.org
Cc: Dave Taht <dave@taht.net>
Subject: [Galene] Re: fq-codel trashing
Date: Tue, 12 Jan 2021 16:55:23 +0100 [thread overview]
Message-ID: <87pn2aup3o.fsf@toke.dk> (raw)
In-Reply-To: <87k0siqndj.wl-jch@irif.fr>
Juliusz Chroboczek <jch@irif.fr> writes:
> Dave, Toke,
>
> We're having a large meeting on the lab's Galène server, and it looks like
> fq-codel is trashing: the new_flow_count increases as fast as the packet
> counter. How do I fix that?
That just sounds like wherever you're running FQ-CoDel is not actually
the bottleneck? If there's no backpressure the queue will drain for each
packet, so that by the time the next packet comes along it'll be
considered "new" again.
If you're running a few video flows on a gigabit link (and thus using a
fraction of the bandwidth) that would be expected. But it only happens
if you're also running a shaper (such as HTB or TBF) on the interface,
since if FQ-CoDel is installed on a physical interface it will set the
TCQ_F_CAN_BYPASS flag, which means that when the queue is completely
empty it will be bypassed entirely...
I just replicated the phenomenon like this:
$ sudo tc qdisc replace dev testns root tbf rate 1gbit latency 10ms burst 1024
$ sudo tc qdisc add dev testns parent 8002: fq_codel
$ tc -s qdisc
qdisc tbf 8002: dev testns root refcnt 2 rate 1Gbit burst 1000b lat 10ms
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 8003: dev testns parent 8002: limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
$ ping -c 5 fc00:dead:cafe:1::2
PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes
64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.088 ms
64 bytes from fc00:dead:cafe:1::2: icmp_seq=2 ttl=64 time=0.138 ms
64 bytes from fc00:dead:cafe:1::2: icmp_seq=3 ttl=64 time=0.102 ms
64 bytes from fc00:dead:cafe:1::2: icmp_seq=4 ttl=64 time=0.103 ms
64 bytes from fc00:dead:cafe:1::2: icmp_seq=5 ttl=64 time=0.104 ms
--- fc00:dead:cafe:1::2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4041ms
rtt min/avg/max/mdev = 0.088/0.107/0.138/0.016 ms
$ tc -s qdisc
qdisc tbf 8002: dev testns root refcnt 2 rate 1Gbit burst 1000b lat 10ms
Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 8003: dev testns parent 8002: limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 118 drop_overlimit 0 new_flow_count 5 ecn_mark 0
new_flows_len 0 old_flows_len 0
Whereas if fq_codel is the root qdisc:
$ sudo tc qdisc replace dev testns root fq_codel
$ tc -s qdisc
qdisc fq_codel 8004: dev testns root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
$ ping -c 5 fc00:dead:cafe:1::2
PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes
64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.085 ms
64 bytes from fc00:dead:cafe:1::2: icmp_seq=2 ttl=64 time=0.098 ms
64 bytes from fc00:dead:cafe:1::2: icmp_seq=3 ttl=64 time=0.098 ms
64 bytes from fc00:dead:cafe:1::2: icmp_seq=4 ttl=64 time=0.097 ms
64 bytes from fc00:dead:cafe:1::2: icmp_seq=5 ttl=64 time=0.096 ms
--- fc00:dead:cafe:1::2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4058ms
$ tc -s qdisc
qdisc fq_codel 8004: dev testns root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
-Toke
next prev parent reply other threads:[~2021-01-12 15:55 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-12 13:46 [Galene] " Juliusz Chroboczek
2021-01-12 15:55 ` Toke Høiland-Jørgensen [this message]
2021-01-12 16:01 ` [Galene] " Dave Taht
2021-01-12 17:38 ` Juliusz Chroboczek
2021-01-12 17:42 ` Dave Taht
2021-01-12 18:10 ` Juliusz Chroboczek
2021-01-12 19:05 ` Dave Taht
2021-01-12 19:52 ` Michael Ströder
2021-01-12 21:02 ` Juliusz Chroboczek
2021-01-12 19:29 ` Michael Ströder
2021-01-12 21:22 ` Juliusz Chroboczek
2021-01-13 19:09 ` Michael Ströder
2021-01-14 12:59 ` Juliusz Chroboczek
2021-01-14 13:03 ` Michael Ströder
2021-01-14 13:10 ` Juliusz Chroboczek
2021-01-14 13:23 ` Michael Ströder
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://lists.galene.org/postorius/lists/galene.lists.galene.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87pn2aup3o.fsf@toke.dk \
--to=toke@toke.dk \
--cc=dave@taht.net \
--cc=galene@lists.galene.org \
--cc=jch@irif.fr \
--subject='[Galene] Re: fq-codel trashing' \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox