[Galene] OBS Studio and Galene WHIP

[Galene] OBS Studio and Galene WHIP

[Alexandre IOOSS]

[-- Attachment #1.1: Type: text/plain, Size: 1020 bytes --]

Hello all,

WHIP output got merged into OBS Studio master early this morning.

I tested Galène with OBS master, and it's breathtaking.
Here are the steps I followed for those who want to reproduce:

  1. Clone Galène source code, checkout "whip" branch and rebase on 
master, then run Galène as usual,

  2. Create a Galène group with:

     "presenter": [{}],
     "allow-anonymous": true,
     "codecs": ["vp8", "opus", "h264"]

This is required as OBS streams h264+opus, and by default Galène does 
not include h264,

  3. Build OBS from master,

  4. Open OBS and go to "Settings > Stream" and change:
     - Service: WHIP
     - Server: https://YOUR_LOGIN@galene.example.com/group/YOUR_GROUP/

Unfortunately this setup requires clients to be able to decode h264 and 
sometimes Firefox can have broken h264 depending on distributions.

The latency and quality are really impressive, much much better than 
what Galène-stream could achieve with Gstreamer.

Best,
-- 
Alexandre

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[Galene] Re: OBS Studio and Galene WHIP

[Juliusz Chroboczek]

> WHIP output got merged into OBS Studio master early this morning.

Excellent!

>  1. Clone Galène source code, checkout "whip" branch and rebase on master,

Note that the whip branch is an old version of Galene, and it's an old
version of the WHIP protocol.  It will take quite some work to get WHIP
support in master.

-- Juliusz

[Galene] Re: OBS Studio and Galene WHIP

[Juliusz Chroboczek]

Hi Alexandre,

> the whip branch is an old version of Galene, and it's an old version of
> the WHIP protocol.

I've just rewritten the WHIP code to obey the latest spec (draft -08).  It
implements all of the optional features of -08, except for ICE restarts.

It lives in the branch "whip2".

Unlike the previous version, whose goal was to experiment with WHIP, this
version aims to comply with the draft even where I disagree.  So there are
some changes in behaviour:

  - the WHIP endpoint is now

      https://galene.example.org/group/groupname/.whip

    since the spec does not allow using a single URL for multiple protocols;

  - only token authentication is supported (i requested support for HTTP
    Basic, but it was rejected by the working group).

Unfortunately, OBS master crashes for me at startup, so i couldn't test
the code.  Please test the code, and let me know whether you feel it is
fit for merging.

-- Juliusz

[Galene] Re: OBS Studio and Galene WHIP

[Sean DuBois]

[-- Attachment #1.1: Type: text/html, Size: 4931 bytes --]

[-- Attachment #1.2: obs-studio.png --]
[-- Type: image/png, Size: 148208 bytes --]

[Galene] Re: OBS Studio and Galene WHIP

[Juliusz Chroboczek]

> Can you try the builds from my latest PR?

Doesn't work :-(

The SDP exchange works fine, the SDP looks correct (with just host
candidates from the client, but a full set of candidates from the server).
The client is sending a UDP/IPv6 flow to the server, the packets reach the
server, but the server-side OnTrack callback never triggers.

The source/destination addresses do seem to correspond to a pair of host
candidates.

FWIW, my code works fine against https://web.whip.eyevinn.technology/

-- Juliusz

[Galene] Re: OBS Studio and Galene WHIP

[Sean DuBois]

[-- Attachment #1: Type: text/plain, Size: 987 bytes --]



> On Jul 10, 2023, at 7:12 PM, Juliusz Chroboczek <jch@irif.fr> wrote:
> 
> 
>> 
>> Can you try the builds from my latest PR?
> 
> Doesn't work :-(
> 
> The SDP exchange works fine, the SDP looks correct (with just host
> candidates from the client, but a full set of candidates from the server).
> The client is sending a UDP/IPv6 flow to the server, the packets reach the
> server, but the server-side OnTrack callback never triggers.
> 
> The source/destination addresses do seem to correspond to a pair of host
> candidates.
> 
> FWIW, my code works fine against https://web.whip.eyevinn.technology/
> 
> -- Juliusz
> _______________________________________________


I am sure a bug in OBS! Do you see RTP packets flowing in tcpdump?

H264 is enabled in Galene?

I will check it out tonight and get a fix in OBS! Sorry about the breakage.

Does https://github.com/Glimesh/broadcast-box work for you? Pion based WHIP/WHEP server to compare against 

[-- Attachment #2: Type: text/html, Size: 1669 bytes --]

[Galene] Re: OBS Studio and Galene WHIP

[Juliusz Chroboczek]

> Do you see RTP packets flowing in tcpdump?

Yes.  I've also checked that they reach the server.  ICE chose an IPv6
pair, so it could be an IPv6-related bug.

> H264 is enabled in Galene?

Yes.

> I will check it out tonight and get a fix in OBS!

Beware that the endpoint has changed, you need to add "/.whip' after the
group URL.

-- Juliusz

[Galene] Re: OBS Studio and Galene WHIP

[Alexandre IOOSS]

[-- Attachment #1.1: Type: text/plain, Size: 990 bytes --]

On 7/11/23 00:04, Sean DuBois wrote:
> Can you try the builds from my latest PR?

When going to OBS Studio settings, I see "Enable Simulcast" only when 
"Output Mode" is set to "Simple". When "Output Mode" is set to 
"Advanced", it seems that the checkbox disappears. This might be a 
little UI bug.


When Simulcast is disabled in OBS, I get the same behavior as with OBS 
"master" branch. I am able to stream to Galene WHIP endpoint successfully.
When Simulcast is enabled in OBS, but disabled in Galene JavaScript 
client, I get the same result.
When Simulcast is both enabled in OBS and Galene JavaScript client, I 
get a very unusable video stream: I see some frames, then it hangs for 
~12 seconds, then it starts again for less than a second, then hangs 
again...


Another observation, OBS Studio crashes with "Segmentation fault (core 
dumped)" when I stop the stream (with and without Simulcast enabled).


Thank you for your awesome work,
-- 
Alexandre

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[Galene] Re: OBS Studio and Galene WHIP

[Sean DuBois]

On Tue, Jul 11, 2023 at 07:39:03PM +0200, Alexandre IOOSS wrote:
> On 7/11/23 00:04, Sean DuBois wrote:
> > Can you try the builds from my latest PR?
> 
> When going to OBS Studio settings, I see "Enable Simulcast" only when
> "Output Mode" is set to "Simple". When "Output Mode" is set to "Advanced",
> it seems that the checkbox disappears. This might be a little UI bug.
Missed that! I will fix today, thanks for testing :)
> 
> 
> When Simulcast is disabled in OBS, I get the same behavior as with OBS
> "master" branch. I am able to stream to Galene WHIP endpoint successfully.
> When Simulcast is enabled in OBS, but disabled in Galene JavaScript client,
> I get the same result.
> When Simulcast is both enabled in OBS and Galene JavaScript client, I get a
> very unusable video stream: I see some frames, then it hangs for ~12
> seconds, then it starts again for less than a second, then hangs again...

How does Simulcast perform for you against https://b.siobud.com ?

> 
> 
> Another observation, OBS Studio crashes with "Segmentation fault (core
> dumped)" when I stop the stream (with and without Simulcast enabled).
> 
Fixed! Sorry about that

> 
> Thank you for your awesome work,
> -- 
> Alexandre

[Galene] Re: OBS Studio and Galene WHIP

[Alexandre IOOSS]

[-- Attachment #1.1: Type: text/plain, Size: 572 bytes --]

Hi Juliusz,

> Unfortunately, OBS master crashes for me at startup, so i couldn't test
> the code.  Please test the code, and let me know whether you feel it is
> fit for merging.

Streaming with OBS Studio (master branch) works fine with Galene "whip2" 
branch. Here is a recording of the test:
https://owncloud.crans.org/s/rkqbv9P9T6Tkhu4

The test was done using Chromium as my Firefox currently has a broken 
H264 decoder. A friend on ArchLinux also tried to join with Firefox and 
it is working fine for her (video and audio).

Thanks,
-- 
Alexandre

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

[Galene] Re: OBS Studio and Galene WHIP

[Juliusz Chroboczek]

> Streaming with OBS Studio (master branch) works fine with Galene "whip2"
> branch. Here is a recording of the test:
> https://owncloud.crans.org/s/rkqbv9P9T6Tkhu4

Excellent news, thanks for testing, Alexandre!  I have no idea why my test
failed last night.

Sean, now you need to implement VP9 SVC in OBS :-)

-- Juliusz

[Galene] congestion control, loss (recovery), whip and obs...

[Dave Taht]

I would dearly like to be capturing more data and pictures of what
videoconferencing traffic looks like, through the CAKE algorithms in
libreqos. I know that sean put google congestion control (GCC) into
pion and I think now, twitch? GCC was recently "dissed" somewhat in
this paper, but I think it was some old versions in there:

https://tma.ifip.org/2023/wp-content/uploads/sites/12/2023/06/tma2023-final37.pdf

Naturally fq_codel shines. The paper also introduces a new means of
measuring videoconference QoE, called BRISQUE.

I know there are some really complicated high stress OBS setups out there.

I am very interested in the difference between vp8 and vp9 in
particular, in bandwidth demands and loss response.



On Tue, Jul 11, 2023 at 7:36 AM Juliusz Chroboczek <jch@irif.fr> wrote:
>
> > Streaming with OBS Studio (master branch) works fine with Galene "whip2"
> > branch. Here is a recording of the test:
> > https://owncloud.crans.org/s/rkqbv9P9T6Tkhu4
>
> Excellent news, thanks for testing, Alexandre!  I have no idea why my test
> failed last night.
>
> Sean, now you need to implement VP9 SVC in OBS :-)
>
> -- Juliusz
> _______________________________________________
> Galene mailing list -- galene@lists.galene.org
> To unsubscribe send an email to galene-leave@lists.galene.org



--
Podcast: https://www.linkedin.com/feed/update/urn:li:activity:7058793910227111937/
Dave Täht CSO, LibreQos

[Galene] Re: OBS Studio and Galene WHIP

[Sean DuBois]

On Tue, Jul 11, 2023 at 03:36:32PM +0200, Juliusz Chroboczek wrote:
> > Streaming with OBS Studio (master branch) works fine with Galene "whip2"
> > branch. Here is a recording of the test:
> > https://owncloud.crans.org/s/rkqbv9P9T6Tkhu4
> 
> Excellent news, thanks for testing, Alexandre!  I have no idea why my test
> failed last night.
> 
> Sean, now you need to implement VP9 SVC in OBS :-)
I am implementing AV1 right now! I don't think VP9 will ever be
available in OBS.

How much better is SVC then Simulcast? I can bring it up with OBS devs,
but Simulcast is already pretty tough to convince people it is worth it.

> 
> -- Juliusz

[Galene] Re: OBS Studio and Galene WHIP

[Juliusz Chroboczek]

(Removed nounous from CC.)

> How much better is SVC then Simulcast? I can bring it up with OBS devs,
> but Simulcast is already pretty tough to convince people it is worth it.

It's simpler to implement, and it allows much more fine-grained control:
you don't need a keyframe in order to switch layers, and you don't need to
consult NTP timestamps (which might not be available) to ensure that time
doesn't flow backwards when you switch.

In the specific case of Galene, SVC is rock solid, while there are
occasional visual glitches when switching simulcast streams.  And the
simulcast switching code can be used to frighten little children, while
the SVC code way simpler and more elegant.

-- Juliusz