I gave galene a BIG plug in the last 5 minutes of my most recent podcast, here: https://twit.tv/shows/floss-weekly/episodes/638?autostart=false Some features 1) Better video logging and combination of speakers video and audio. 2) speech to text and text to speech. Notably feeding the chat - somehow - into a speech synth - for those of us that are increasingly deaf. 3) Roundtable mode - where each speaker is given a limited time to speak, with a visible clock, other speakers can grant the existing speaker some time from their pool, and a moderator (automated) that silences that speaker and then queues the next speaker 4) whatever someone with something cools wants to propose. I can write a sure-to-hit- grant proposal for any of these if only there was someone willing to tackle it. Deadline is July 30th. -- Latest Podcast: https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/ Dave Täht CTO, TekLibre, LLC
> 1) Better video logging and combination of speakers video and audio. Yes, it's an important missing feature. I would reformulate it as better export of recordings -- storing recordings as multiple independent files is good design, but there should be a way to export a recording as a single file. This could probably be done in Javascript on the user device. > 2) speech to text and text to speech. Notably feeding the chat - > somehow - into a speech synth - for those of us that are increasingly > deaf. Do any free software browsers support the Speech Synthesis API? Dave, what does your browser say if you type window.speechSynthesis.getVoices(); in the Javascript console? For me, it returns an empty list. > 3) Roundtable mode - where each speaker is given a limited time to > speak, with a visible clock, other speakers can grant the existing > speaker some time from their pool, and a moderator (automated) that > silences that speaker and then queues the next speaker The server only allows you to mute a user remotely, not to unmute them. That's an important property, and one that I'd be unwilling to break. -- Juliusz
[-- Attachment #1: Type: text/plain, Size: 364 bytes --] > > 2) speech to text and text to speech. Notably feeding the chat - > somehow - into a speech synth - for those of us that are increasingly > deaf. > Wouldn't speech-to-text be more useful in that case? There's also the Speech Recognition API for browsers. Notably, Google, Microsoft, and Amazon offer realtime Speech to Text services at approx 2$ per hour. > [-- Attachment #2: Type: text/html, Size: 808 bytes --]
[-- Attachment #1: Type: text/plain, Size: 594 bytes --] Maybe something self-hosted like this would work? https://voice2json.org/ ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ Op vrijdag 16 juli 2021 om 2:16 PM schreef Antonin Décimo <antonin.decimo@gmail.com>: >> 2) speech to text and text to speech. Notably feeding the chat - >> somehow - into a speech synth - for those of us that are increasingly >> deaf. > > Wouldn't speech-to-text be more useful in that case? There's also the Speech Recognition API for browsers. > > Notably, Google, Microsoft, and Amazon offer realtime Speech to Text services at approx 2$ per hour. > >> [-- Attachment #2: Type: text/html, Size: 1537 bytes --]
On 16.07.21 13:53, Juliusz Chroboczek wrote:
>> 3) Roundtable mode - where each speaker is given a limited time to
>> speak, with a visible clock, other speakers can grant the existing
>> speaker some time from their pool, and a moderator (automated) that
>> silences that speaker and then queues the next speaker
> The server only allows you to mute a user remotely, not to unmute them.
> That's an important property, and one that I'd be unwilling to break.
I'd also like such kind of automatic moderation which Dave is proposing.
And I agree that automatic unmuting is a no-go. The next speaker could
however receive an invitation (text message) to unmute her:himself.
Ideally the forwarding would be controlled by a pluggable algorithm: The
algorithm would signal the SFU which stream to forward. And the op would
be able to set the initial algo and choose a different one later. The
algo also would be able to send text messages to individual or all
users. Maybe the algo could be reduced to some elementary syntax that
could be described by a json-config file. Then for each algo that is
wanted the server admin could provide such a forwarding control file
that the ops can choose for their sessions.
Just some not well-thought-out ideas ...
ibu
[-- Attachment #1: Type: text/plain, Size: 507 bytes --] > On 16 Jul 2021, at 03:41, Dave Taht <dave.taht@gmail.com> wrote: > > 2) speech to text and text to speech. Notably feeding the chat - > somehow - into a speech synth - for those of us that are increasingly > deaf. I’m finding Google chrome’s auto subtitles works pretty well for public-ish conversations. It makes some hideous mistakes of course, but over all it is worth it, especially for when one has an attention break, reading the text from a few seconds back is super useful. T. [-- Attachment #2: Type: text/html, Size: 2844 bytes --]
>> The server only allows you to mute a user remotely, not to unmute them. >> That's an important property, and one that I'd be unwilling to break. > I'd also like such kind of automatic moderation which Dave is proposing. > And I agree that automatic unmuting is a no-go. The next speaker could > however receive an invitation (text message) to unmute her:himself. In that case, there's no need for any server changes, just add something like this to the client: let speakers = the list of speaker ids; let i = 0; while(true) { let speaker = speakers[i % speakers.length]; serverConnection.userMessage('warning', speaker, 'Please unmute yourself'); await sleep(30 * 1000); serverConnection.userMessage('mute', speaker); } The tricky bit is integrating it into the UI, and I suck at UI. -- Juliusz
> 2) speech to text and text to speech. Notably feeding the chat - > somehow - into a speech synth - for those of us that are increasingly > deaf. Does the following work for you? https://galene.org/speech.html
Open the pod bay doors, hal. On Fri, Jul 16, 2021 at 11:32 AM Juliusz Chroboczek <jch@irif.fr> wrote: > > > 2) speech to text and text to speech. Notably feeding the chat - > > somehow - into a speech synth - for those of us that are increasingly > > deaf. > > Does the following work for you? > > https://galene.org/speech.html -- Latest Podcast: https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/ Dave Täht CTO, TekLibre, LLC
>> Does the following work for you? >> >> https://galene.org/speech.html > Open the pod bay doors, hal. I assume this means it works for you. So how do you envision the feature? - Messages are read as they arrive, or when you click on a button? - What happens if a new message arrives before the previous one has finished reading? The new message is queued, or the old message is interrupted? - How is the language of the message determined? Manually by the user? In the group configuration? I'm quite willing to implement it, but I need to be assured that there's someone actually using it to help me tune the UI. -- Juliusz
Let me rope in a friend that is totally blind to help on the spec. Frank, the galene crew. Galene crew, frank. Frank was adding a few html assistance features to galene when last we spoke. On Sun, Jul 18, 2021 at 3:17 PM Juliusz Chroboczek <jch@irif.fr> wrote: > > >> Does the following work for you? > >> > >> https://galene.org/speech.html > > > Open the pod bay doors, hal. > > I assume this means it works for you. So how do you envision the feature? Yes, you made me laugh with that url. Thx. > - Messages are read as they arrive, or when you click on a button? As they arrive but if at all possible panned to one side or waiting for a gap in conversation. > - What happens if a new message arrives before the previous one has > finished reading? The new message is queued, or the old message is > interrupted? queued. > > - How is the language of the message determined? Manually by the user? > In the group configuration? Heh. auto-translation to another language on the fly would be amazing... Not in the group. The senders language should probably be in the "locale" info per browser Same for the reciever's language (though less so) > > I'm quite willing to implement it, but I need to be assured that there's > someone actually using it to help me tune the UI. Frank? > > -- Juliusz -- Latest Podcast: https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/ Dave Täht CTO, TekLibre, LLC
[-- Attachment #1: Type: text/plain, Size: 1848 bytes --] Hi all, I read through the plan to record all streams separately as independent files, and then recombine them. However, since h264 is not supported as a recording format, any stream that comes in with this format would be lost. Are there any plans to be able to convert h264 streams into a different format for recording? -Eliot On Fri, Jul 16, 2021 at 4:53 AM Juliusz Chroboczek <jch@irif.fr> wrote: > > 1) Better video logging and combination of speakers video and audio. > > Yes, it's an important missing feature. > > I would reformulate it as better export of recordings -- storing > recordings as multiple independent files is good design, but there should > be a way to export a recording as a single file. > > This could probably be done in Javascript on the user device. > > > 2) speech to text and text to speech. Notably feeding the chat - > > somehow - into a speech synth - for those of us that are increasingly > > deaf. > > Do any free software browsers support the Speech Synthesis API? Dave, > what does your browser say if you type > > window.speechSynthesis.getVoices(); > > in the Javascript console? For me, it returns an empty list. > > > 3) Roundtable mode - where each speaker is given a limited time to > > speak, with a visible clock, other speakers can grant the existing > > speaker some time from their pool, and a moderator (automated) that > > silences that speaker and then queues the next speaker > > The server only allows you to mute a user remotely, not to unmute them. > That's an important property, and one that I'd be unwilling to break. > > -- Juliusz > _______________________________________________ > Galene mailing list -- galene@lists.galene.org > To unsubscribe send an email to galene-leave@lists.galene.org > -- Eliot Mack CEO Lightcraft Technology (310) 821-3888 <(805)%20338-7199> [-- Attachment #2: Type: text/html, Size: 3102 bytes --]
> However, since h264 is not supported as a recording format, any stream that
> comes in with this format would be lost.
>
> Are there any plans to be able to convert h264 streams into a different format
> for recording?
Galene does not do any server-side transcoding, and I hope to keep it that
way. The proper fix would be to allow recording H.264 + Opus, either in
MP4 or in MKV. I'll see what I can do.
Just out of curiosity -- why are you using H.264?
-- Juliusz
[-- Attachment #1: Type: text/plain, Size: 945 bytes --] Hi Juliusz, We have a stream coming in from a custom iOS app that uses the on-board H.264/H.265 compression hardware to generate the stream. Being able to record that along with the other streams would be very useful. If enabling H.264 + Opus recording into mp4 solves this, great! -Eliot On Tue, Jul 27, 2021 at 3:12 PM Juliusz Chroboczek <jch@irif.fr> wrote: > > However, since h264 is not supported as a recording format, any stream > that > > comes in with this format would be lost. > > > > Are there any plans to be able to convert h264 streams into a different > format > > for recording? > > Galene does not do any server-side transcoding, and I hope to keep it that > way. The proper fix would be to allow recording H.264 + Opus, either in > MP4 or in MKV. I'll see what I can do. > > Just out of curiosity -- why are you using H.264? > > -- Juliusz > -- Eliot Mack CEO Lightcraft Technology (310) 821-3888 <(805)%20338-7199> [-- Attachment #2: Type: text/html, Size: 1948 bytes --]
> We have a stream coming in from a custom iOS app that uses the on-board > H.264/H.265 compression hardware to generate the stream. Cool. How do you communicate with Galène ? Would you be willing to contribute an iOS app that does screen sharing ? Some of my fellow lecturers would like to be able to use iPads for teaching, and screenshare is not possible on iOS without a native app. (Same is true of Android, by the way, so if anyone here knows how to do WebRTC in an Android app, screensharing with Galène would be a useful little project.) > Being able to record that along with the other streams would be very useful. If > enabling H.264 + Opus recording into mp4 solves this, great! Ok, I'll see what I can do. -- Juliusz
I haven't had time to look at this, but I really would like to help this effort along. Has there been any more discussion on what work needs to happen?
Would it be okay if I just provided feedback?
--FC
> On Jul 18, 2021, at 6:44 PM, Dave Taht <dave.taht@gmail.com> wrote:
>
> Let me rope in a friend that is totally blind to help on the spec.
> Frank, the galene crew. Galene crew, frank. Frank was adding a few
> html assistance features to galene when last we spoke.
>
> On Sun, Jul 18, 2021 at 3:17 PM Juliusz Chroboczek <jch@irif.fr> wrote:
>>
>>>> Does the following work for you?
>>>>
>>>> https://galene.org/speech.html
>>
>>> Open the pod bay doors, hal.
>>
>> I assume this means it works for you. So how do you envision the feature?
>
> Yes, you made me laugh with that url. Thx.
>
>> - Messages are read as they arrive, or when you click on a button?
>
> As they arrive but if at all possible panned to one side or waiting
> for a gap in conversation.
>
>> - What happens if a new message arrives before the previous one has
>> finished reading? The new message is queued, or the old message is
>> interrupted?
>
> queued.
>
>>
>> - How is the language of the message determined? Manually by the user?
>> In the group configuration?
>
> Heh. auto-translation to another language on the fly would be amazing...
>
> Not in the group.
>
> The senders language should probably be in the "locale" info per browser
> Same for the reciever's language (though less so)
>
>
>
>>
>> I'm quite willing to implement it, but I need to be assured that there's
>> someone actually using it to help me tune the UI.
>
> Frank?
>
>>
>> -- Juliusz
>
>
>
> --
> Latest Podcast:
> https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/
>
> Dave Täht CTO, TekLibre, LLC
> I haven't had time to look at this, but I really would like to help this
> effort along.
That's useful information in itself -- there's at least two people
interested in text-to-speech, and who might be willing to help with
testing. I'll try to do a prototype if I find some time next week.
-- Juliusz
> I haven't had time to look at this, but I really would like to help this > effort along. Has there been any more discussion on what work needs to > happen? > Would it be okay if I just provided feedback? Frank, I've put a proof-of-concept on https://galene.org:8444 After joining a group, type /speak in the chat. After that, all the chat messages you receive will be read out loud. Please let me know if you think it's useful, and I'll add a proper UI. -- Juliusz