From mboxrd@z Thu Jan 1 00:00:00 1970 From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=toke.dk; s=20161023; t=1610466924; bh=2+F6ruAcMym6IMOkgmG3lKn/Z2GrFMNpM49visPJY4o=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=arDkIvgLUilQsW6KZ57Feg/J4WsJR6SIlFgIWmkyYNlut9pfhw5Zn8qoAUQ0Xtubs K9hwTg3EUYCeM/aTTRnPsLl+8z55UAffKi4EzHF07y3geFj8K3b0Tev8pfjC+dGKOR 8w8dGyCEwgZpVLF/zMmIQ/1dkbvoeozg7oV2henYL18lFLN76IZeufdQAKGxlxaaqU r9i7bCKyZL8Nu4LLxifM/mZO6eESnXyVYNOu0WIJn7mSkLb4kdD1HHThRvjQw7bsv1 1X9U8ubCCmRKq6xEyLVgoBB3pCKRDpwWqbjrrotaj+4cobgnEfZgCKB1xxH/aj8Prl e3esDrJFuraBg== To: Juliusz Chroboczek , galene@lists.galene.org In-Reply-To: <87k0siqndj.wl-jch@irif.fr> References: <87k0siqndj.wl-jch@irif.fr> Date: Tue, 12 Jan 2021 16:55:23 +0100 X-Clacks-Overhead: GNU Terry Pratchett Message-ID: <87pn2aup3o.fsf@toke.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: RLKWVBHRWHABIVXKAS4QCYLTKIX37IA4 X-Message-ID-Hash: RLKWVBHRWHABIVXKAS4QCYLTKIX37IA4 X-MailFrom: toke@toke.dk X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: Dave Taht X-Mailman-Version: 3.3.2 Precedence: list Subject: [Galene] Re: fq-codel trashing List-Id: =?utf-8?q?Gal=C3=A8ne_videoconferencing_server_discussion_list?= Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Juliusz Chroboczek writes: > Dave, Toke, > > We're having a large meeting on the lab's Gal=C3=A8ne server, and it look= s like > fq-codel is trashing: the new_flow_count increases as fast as the packet > counter. How do I fix that? That just sounds like wherever you're running FQ-CoDel is not actually the bottleneck? If there's no backpressure the queue will drain for each packet, so that by the time the next packet comes along it'll be considered "new" again. If you're running a few video flows on a gigabit link (and thus using a fraction of the bandwidth) that would be expected. But it only happens if you're also running a shaper (such as HTB or TBF) on the interface, since if FQ-CoDel is installed on a physical interface it will set the TCQ_F_CAN_BYPASS flag, which means that when the queue is completely empty it will be bypassed entirely... I just replicated the phenomenon like this: $ sudo tc qdisc replace dev testns root tbf rate 1gbit latency 10ms burst 1= 024 $ sudo tc qdisc add dev testns parent 8002: fq_codel $ tc -s qdisc qdisc tbf 8002: dev testns root refcnt 2 rate 1Gbit burst 1000b lat 10ms=20 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)=20 backlog 0b 0p requeues 0 qdisc fq_codel 8003: dev testns parent 8002: limit 10240p flows 1024 quantu= m 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64=20 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)=20 backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 $ ping -c 5 fc00:dead:cafe:1::2=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20 PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D1 ttl=3D64 time=3D0.088 ms 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D2 ttl=3D64 time=3D0.138 ms 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D3 ttl=3D64 time=3D0.102 ms 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D4 ttl=3D64 time=3D0.103 ms 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D5 ttl=3D64 time=3D0.104 ms --- fc00:dead:cafe:1::2 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4041ms rtt min/avg/max/mdev =3D 0.088/0.107/0.138/0.016 ms $ tc -s qdisc=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 qdisc tbf 8002: dev testns root refcnt 2 rate 1Gbit burst 1000b lat 10ms=20 Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)=20 backlog 0b 0p requeues 0 qdisc fq_codel 8003: dev testns parent 8002: limit 10240p flows 1024 quantu= m 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64=20 Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)=20 backlog 0b 0p requeues 0 maxpacket 118 drop_overlimit 0 new_flow_count 5 ecn_mark 0 new_flows_len 0 old_flows_len 0 Whereas if fq_codel is the root qdisc: $ sudo tc qdisc replace dev testns root fq_codel $ tc -s qdisc qdisc fq_codel 8004: dev testns root refcnt 2 limit 10240p flows 1024 quant= um 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64=20 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)=20 backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 $ ping -c 5 fc00:dead:cafe:1::2=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20 PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D1 ttl=3D64 time=3D0.085 ms 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D2 ttl=3D64 time=3D0.098 ms 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D3 ttl=3D64 time=3D0.098 ms 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D4 ttl=3D64 time=3D0.097 ms 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D5 ttl=3D64 time=3D0.096 ms --- fc00:dead:cafe:1::2 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4058ms $ tc -s qdisc qdisc fq_codel 8004: dev testns root refcnt 2 limit 10240p flows 1024 quant= um 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64=20 Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)=20 backlog 0b 0p requeues 0 maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 new_flows_len 0 old_flows_len 0 -Toke