sound-and-vision
Self-hosted music creation platform with local YuE2 generation, writing assistance, kinetic lyric videos, visualizers, and cover art. Supports OpenAI accounts or local models (Qwen3.5 + Ollama, Krea 2 Turbo, SDXL). Single-owner Linux installation on compatible NVIDIA GPU (~24GB VRAM).
Ficha actualizada 23 sept 2026, 5:00 · upstream 23 sept 2026, 4:54 · release sin release detectada
Qué es sound-and-vision
Descripción editorial construida a partir del repositorio, su documentación y sus señales de mantenimiento.
Sound and Vision is a self-hosted music creation workspace that combines local AI generation with visual storytelling capabilities. Users can generate songs using locally hosted models such as Qwen3.5 + Ollama, Krea 2 Turbo, or SDXL without needing an OpenAI account. The platform offers kinetic lyric videos where animated text appears over the song in real-time, music video creation with scene-based composition, and visualizer videos that react to audio frequencies. It also supports album cover generation from generated tracks. The application runs on a single Linux computer requiring approximately 24 GB of free VRAM.
- Generate kinetic lyric videos where animated text appears over the song in real-time
- Create music video scenes with thematic storytelling and visual composition
- Build audio-reactive visualizers that respond to sound frequencies
- Generate album artwork from generated songs using reference images
- Transcribe lyrics directly from recorded audio tracks
- Supports local YuE2 models without requiring an OpenAI account for basic functionality
- Generates high-quality 1080p MP4 videos at 30 fps with original audio preserved
- Single-owner private deployment on Linux with no public multi-user support currently available
- Experimental scene-based music video workflow with customizable themes and storyboards
- Lip-sync performance for sung vocals remains in development (Coming soon)
- Reference-image contract is not supported when using local image presets
- Requires NVIDIA GPU with approximately 24 GB of free VRAM for optimal performance
- H3LIX service and FFmpeg are required for rendering music videos, which may require additional hardware
Perfil de infraestructura
Estimación inicial derivada del repositorio y de los servicios detectados.
Despliegue detectado
Qué señales reales de instalación hemos encontrado.
Install the backend on a Linux machine with a compatible NVIDIA GPU (minimum ~24GB VRAM). Follow the SELF_HOSTED.md guide to set up the Python environment, connect to local AI providers via Ollama or ComfyUI, and configure your first song. The frontend can run locally without a backend for basic editing, but generation requires the full stack. For production use, consider containerizing with Docker Compose or running on a cloud instance with GPU access.
Stack y dependencias
Componentes inferidos a partir de manifests, configuración y documentación.
Puede sustituir a
Relaciones detectadas por el análisis del repositorio y su categoría.
Experiencia real de la comunidad
Ayuda a contrastar la estimación automática con instalaciones reales.
Todavía no hay datos comunitarios para este proyecto.