From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by mail.toke.dk (Postfix) with ESMTPS id 2156F7CA079 for ; Tue, 12 Jan 2021 17:01:33 +0100 (CET) Authentication-Results: mail.toke.dk; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dFEH4yE0 Received: by mail-io1-xd32.google.com with SMTP id r9so5088275ioo.7 for ; Tue, 12 Jan 2021 08:01:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=0dQIqgjhB5YkRS03MJKmn2Wg/nTxHDztxX6R5RrTHa0=; b=dFEH4yE02XXtw/HrbVrbI3Z2q3FjJliJY+/Syt3ahcVVvNs+Ff3IHDuGI1JqFxomct 8gScG19mo2TPliCbmdDy6udFL8DwdPmWNTqrL+4JngJ8QTmj3Jxhe5QmKxdmomj9d7ez WvQUjR0bUIzIDGPbFZ1eIeOJulP1Qyc1BDPrWfQs3YwJ4xKZb2ZjWAEm2+4VDH4bq63v sB+DQw/fsPA9IlPjGFUPBe06glW/TArOrt1xyVJXoYvBSsclDxpry4XOGcg9ZAcF152/ leJxqMdl5ZAX21D9Z/HrAomlvh2E6aeQHESiQFMd2xL+AavybGNIls1KfVRUWvz9TxJn eDcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=0dQIqgjhB5YkRS03MJKmn2Wg/nTxHDztxX6R5RrTHa0=; b=bIXLSbj2x3TM+eXxbRlTXOaycBWd6x453Vano+rI7tPBeaUdMOTSo4koWC0YUQE0gu wfoNovYUYTX8ogkU5+9IN54nkkgZB8JA/3C/3Ao65BqlRp4mNEwtIWJShc9bzXsITeKl T9/8cUi5L6Z8Nx4cfINcasFkiMdpjpBDF3Xgx868GM0UMyOvHLb4Sb2eqMi6dNe/0ota hLcnKwouUetJ4MaeBg/YUr1pF5atVP6TRi6zQb2G8dfPLAKsplOrq761Y+V0au4i17KG WFAmUAmNRln01QcICZvMDgoHJ/vkjxtDQ3583fnVTOf8qpOPojrAHOYqykop68DB3+xi 84yQ== X-Gm-Message-State: AOAM532uVZcGe2ZLIHandfCUKXNtusd0DxAAUyb8bQC87vWdocApyiyL qUxYYXjUsz5CbI+sH2tfUVdzj0hwpNiWPg/BeYM= X-Google-Smtp-Source: ABdhPJzTjCAsYg+lfByjuAOU4GDzrRAG3+CocoSHVadRKyDwri/JW9m6kzWzx/8aq0pLB1JdYAxmTVtGz+FfBwW8I9k= X-Received: by 2002:a92:da85:: with SMTP id u5mr3443388iln.249.1610467290408; Tue, 12 Jan 2021 08:01:30 -0800 (PST) MIME-Version: 1.0 References: <87k0siqndj.wl-jch@irif.fr> <87pn2aup3o.fsf@toke.dk> In-Reply-To: <87pn2aup3o.fsf@toke.dk> From: Dave Taht Date: Tue, 12 Jan 2021 08:01:18 -0800 Message-ID: To: =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Message-ID-Hash: TGQK3YR4JWJ5N5YCZFMFFLYN4OYPI5IH X-Message-ID-Hash: TGQK3YR4JWJ5N5YCZFMFFLYN4OYPI5IH X-MailFrom: dave.taht@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: Juliusz Chroboczek , galene@lists.galene.org, Dave Taht X-Mailman-Version: 3.3.2 Precedence: list Subject: [Galene] Re: fq-codel trashing List-Id: =?utf-8?q?Gal=C3=A8ne_videoconferencing_server_discussion_list?= Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: On Tue, Jan 12, 2021 at 7:55 AM Toke H=C3=B8iland-J=C3=B8rgensen wrote: > > Juliusz Chroboczek writes: > > > Dave, Toke, > > > > We're having a large meeting on the lab's Gal=C3=A8ne server, and it lo= oks like > > fq-codel is trashing: the new_flow_count increases as fast as the packe= t > > counter. How do I fix that? It is a pretty useless statistic. I never liked it. In this case it is a positive sign you are not filling the queue, not a negative one, as toke goes into more detail be= low. If you are having fun with fq_codel, try cake instead. Lots of more stats, some of which are way more relevant to your application. tc qdisc add dev eth0 root cake bandwidth XMbit # besteffort if you don't want to bother with diffserv tc -s qdisc show Bulk Best Effort Voice thresh 562496bit 9Mbit 2250Kbit target 32.3ms 5.0ms 8.1ms interval 127.3ms 100.0ms 103.1ms pk_delay 9.2ms 224us 494us av_delay 2.9ms 29us 35us sp_delay 153us 4us 3us backlog 0b 0b 0b pkts 74853 35044344 162590 bytes 80542836 9388672264 27301751 way_inds 2538 2129502 10508 way_miss 2575 549864 7374 way_cols 0 0 0 drops 11 7987 0 marks 22 1 0 ack_drop 0 3616230 0 sp_flows 1 1 1 bk_flows 0 1 0 un_flows 0 0 0 max_len 15605 25262 3169 quantum 300 300 300 > > That just sounds like wherever you're running FQ-CoDel is not actually > the bottleneck? If there's no backpressure the queue will drain for each > packet, so that by the time the next packet comes along it'll be > considered "new" again. > > If you're running a few video flows on a gigabit link (and thus using a > fraction of the bandwidth) that would be expected. But it only happens > if you're also running a shaper (such as HTB or TBF) on the interface, > since if FQ-CoDel is installed on a physical interface it will set the > TCQ_F_CAN_BYPASS flag, which means that when the queue is completely > empty it will be bypassed entirely... > > I just replicated the phenomenon like this: > > $ sudo tc qdisc replace dev testns root tbf rate 1gbit latency 10ms burst= 1024 > $ sudo tc qdisc add dev testns parent 8002: fq_codel > $ tc -s qdisc > qdisc tbf 8002: dev testns root refcnt 2 rate 1Gbit burst 1000b lat 10ms > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > qdisc fq_codel 8003: dev testns parent 8002: limit 10240p flows 1024 quan= tum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > > $ ping -c 5 fc00:dead:cafe:1::2 > PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D1 ttl=3D64 time=3D0.088 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D2 ttl=3D64 time=3D0.138 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D3 ttl=3D64 time=3D0.102 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D4 ttl=3D64 time=3D0.103 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D5 ttl=3D64 time=3D0.104 ms > > --- fc00:dead:cafe:1::2 ping statistics --- > 5 packets transmitted, 5 received, 0% packet loss, time 4041ms > rtt min/avg/max/mdev =3D 0.088/0.107/0.138/0.016 ms > > $ tc -s qdisc > qdisc tbf 8002: dev testns root refcnt 2 rate 1Gbit burst 1000b lat 10ms > Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > qdisc fq_codel 8003: dev testns parent 8002: limit 10240p flows 1024 quan= tum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 > Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 118 drop_overlimit 0 new_flow_count 5 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > > > > Whereas if fq_codel is the root qdisc: > > $ sudo tc qdisc replace dev testns root fq_codel > $ tc -s qdisc > qdisc fq_codel 8004: dev testns root refcnt 2 limit 10240p flows 1024 qua= ntum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > > $ ping -c 5 fc00:dead:cafe:1::2 > PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D1 ttl=3D64 time=3D0.085 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D2 ttl=3D64 time=3D0.098 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D3 ttl=3D64 time=3D0.098 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D4 ttl=3D64 time=3D0.097 ms > 64 bytes from fc00:dead:cafe:1::2: icmp_seq=3D5 ttl=3D64 time=3D0.096 ms > > --- fc00:dead:cafe:1::2 ping statistics --- > 5 packets transmitted, 5 received, 0% packet loss, time 4058ms > > $ tc -s qdisc > qdisc fq_codel 8004: dev testns root refcnt 2 limit 10240p flows 1024 qua= ntum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 > Sent 590 bytes 5 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > > > -Toke > _______________________________________________ > Galene mailing list -- galene@lists.galene.org > To unsubscribe send an email to galene-leave@lists.galene.org --=20 "For a successful technology, reality must take precedence over public relations, for Mother Nature cannot be fooled" - Richard Feynman dave@taht.net CTO, TekLibre, LLC Tel: 1-831-435-0729