[Galene] Galene-stt with the Moonshine model

[Galene] Galene-stt with the Moonshine model

[Juliusz Chroboczek]

Hi,

Some of you might remember galene-stt, the Galene client for automatic
subtitling.

  https://github.com/jech/galene-stt

Galene-stt uses the whisper.cpp library, which is not really adapted to
streaming: it has been trained on 30s chunks of audio.  Galene-stt works
around the issue by chopping the audio into chunks of at most 3s, and
passing the chunks, padded with silence, to the model.  This hack works
surprisingly well, at least when using the larger Whisper models.

I've recently been made aware of the Moonshine model, which is optimised
for streaming.  Since I'm currently procrastinating Real Work (sorry,
Toke), I decided to see how well Moonshine works for galene-stt.  The code
lives in the "moonshine" branch of the galene-stt repository, feel free to
experiment with it.

Advantages:

  - the API is very nice, my code got much, much simpler;
  - it's a real streaming API, so previously decoded audio serves as
    a context to the model;
  - it doesn't require a GPU;
  - latency is in the low hundreds of milliseconds.

Disadvantages:

  - it doesn't use the GPU, so you need a powerful CPU;
  - in my experiments, accuracy is much worse than the Whisper
    large-turbo model, with frequent hallucinations (even though the
    published benchmarks imply otherwise);
  - French is not supported (Matoken-san: Japanese is supported).

Without French support, this is essentially useless for me.  If people
tell me that this is useful, I might find a way to merge the two versions
and provide a single version that groks both models.

By the way, there are two other open-weights voice recognition models,
NVidia's Parakeet and Mistral's Voxtral.  Unfortunately, I couldn't find
any usable C or C++ libraries that use these models (the ones that I found
are vibe-coded nonsense, not something I'm willing to depend on).

Regards,

-- Juliusz