AI Speech Technologies

This page is a collection of notes and links related to speech technologies, including Text-to-Speech (TTS), Speech-to-Text (STT), voice synthesis, voice cloning, and other related frippery in the modern space.

Resources

FieldCategoryDateLinkNotes
Speech Agents2026TalkWithMe

Local single-user AI chat application with configurable personas and TTS/STT integrations, using an OpenAI-compatible LLM server

Generative AudioModels2023bark

a text-prompted genereative audio model

Speech AgentsTools2026Xiaozhi Hub Server

Self-hosted gateway for Xiaozhi-compatible voice devices; handles speech recognition and synthesis, routing requests to Claude Code or supported model APIs. Runs on macOS, Linux and Windows.

speech-to-speech

an open-source local voice agent pipeline combining VAD, STT, LLM, and TTS

dograh

Open source voice agent platform.

RCLI

an on-device Apple Silicon voice agent for macOS that combines STT, local LLM inference, TTS, 38 local actions, and document RAG in a low-latency TUI/CLI.

2025Asterisk-AI-Voice-Agent

An Asterisk-based AI voice agent project integrating telephony with voice AI workflows.

Speech RecognitionApps2026BetterWispr

Open-source macOS dictation app that transcribes locally with Apple speech, NVIDIA Parakeet or Whisper; optional API connections are available. Requires macOS 14+; Apple Silicon recommended

Libraries2026Desert Ant Core

Cross-platform SDK for on-device speech recognition (Voz) and speaker labeling (Who), plus audio enhancement and other speech models; Swift, Kotlin and JavaScript

2025WhisperKit

a Swift package that integrates Whisper with Apple’s CoreML

Models2026transcribe.cpp

ggml speech-to-text inference for 16+ model families

parakeet.cpp

a ggml-based C++ inference port of NVIDIA NeMo Parakeet ASR models with streaming support

2024moonshine

a family of models optimized for fast and accurate automatic speech recognition on resource-constrained devices. Designed to run efficiently on smaller hardware. The micro subdirectory contains the very-low-latency speech stack.

WhisperLive

a real-time text-to-speech system based on Whisper

2023distil-whisper

a distilled version of whisper that is 6 times faster

2022whisper

a general purpose speech recognition model

whisper.cpp

a C++ implementation of whisper that can run in consumer hardware

Tools2026Phone Whisper

Android push-to-talk dictation app with on-device sherpa-onnx models or OpenAI cloud transcription; uses Accessibility to insert text into focused fields and falls back to the clipboard.

voice-notes-watcher

macOS daemon and menu bar app that watches Apple Voice Memos, transcribes them locally with Parakeet MLX, and writes structured Markdown notes

Bootlegger

FastAPI server for Moonshine STT

TypeWhisper

A macOS dictation and transcription app that can use Apple’s speech stack as well as other local on-device engines, with optional prompt-driven post-processing.

Ghost Pepper

a 100% local macOS hold-to-talk speech-to-text menu bar app with WhisperKit transcription and local LLM cleanup.

dictate

a Go-based local voice-to-text tool for Linux terminals that streams whisper.cpp transcription to stdout, files, or keystroke injection for dictating into focused terminal apps.

Handy

An offline, cross-platform speech-to-text app built with Tauri that transcribes locally. Uses Whisper and Parakeet models without sending audio to the cloud.

2024audapolis

an editor for spoken-word audio with automatic transcription

2023insanely-fast-whisper

An opinionated CLI for audio transcription

Transcription Tools2026Who Said What

macOS app that transcribes podcasts and videos on-device, assigning each line to a speaker; built on Who and Apple’s speech transcriber

buzz

A cross-platform Whisper desktop app that works quite well on the Mac (including speaker diarization)

2025OpenTranscribe

an open-source all-in-one recording transcription and diarization stack

Speech SynthesisImplementations2026pocket-tts.c

A minimal, dependency-free C scaffold for Pocket-TTS, aimed at CPU-only TTS. Includes a tiny CLI in the flux2.c style.

Models2026Audio8 TTS

Compact zero-shot voice cloning TTS models: the 0.6B Realistic Air release supports 23 languages; the 0.1B preview offers CPU inference via ONNX INT8 and has a separate model license.

Paradee

A 7.88M-parameter English text-to-speech model distilled from Kokoro, with four voices and an 8.5 MB int8 ONNX model that runs on CPU.

Drama 3

Fish Audio text-to-speech MoE model with 35B parameters (3B active), voice cloning, natural-language control and multi-speaker generation

OmniVoice

Multilingual zero-shot text-to-speech model and toolkit with voice cloning, voice design, pronunciation controls, and support for more than 600 languages

Inflect-Micro-v2

a lightweight text-to-speech model

silero-models

Multi-language neural text-to-speech models

sopro

a lightweight text-to-speech model

2025chatterbox

a text-to-speech model that can generate expressive speech with a variety of styles and emotions.

Orpheus-TTS

an open-source text-to-speech system built on Llama-3b

csm

a speech generation model from Sesame that generates RVQ audio codes from text and audio inputs.

2024ChatTTS

a text-to-speech model designed specifically for dialogue scenarios, with decent prosody

WhisperSpeech

a text-to-speech system built by inverting Whisper

Real-Time-Voice-Cloning

a PyTorch implementation of a voice cloning model

2023StyleTTS2

A text to speech model that supports style diffusion

Resources2025Training a voice for piper TTS

a detailed walkthrough of how to customize a voice model

Tools2026agent-voice

Python CLI, stdin and HTTP server that gives LLM agents a cloned voice through an OmniVoice-compatible text-to-speech service

breeze-tts

official PyTorch inference implementation for the Breeze TTS 2 English/Chinese text-to-speech model, with voice cloning and streaming

inflect-speechd

local text-to-speech server around Inflect Nano v2 with Speech Dispatcher integration

Voicebox

an open-source voice cloning studio with DAW-like features, local-first voice synthesis powered by Qwen3-TTS, multi-track timeline editor, and REST API

pocket-tts

A lightweight text-to-speech (TTS) application designed to run efficiently on CPUs that supports voice cloning

2025voice-pro

a toolkit for doing speech processing and voice cloning

abogen

a tool for generating audiobooks from text using the Kokoro open weights model

edge-tts

a text-to-speech module that leverages the Microsoft Edge TTS API

podcastfy

a tool for generating podcasts from text

2024OpenVoice

a tool that enables accurate voice cloning with multi-lingual support and flexible style control.

Speech TranslationModels2026Hy-SimulMT

Real-time simultaneous Chinese–English speech translation model with inference and training tools; speech output and voice cloning require a configured TTS service.

This page is referenced in: