GoKawiillibargus
An unmanaged, zero-allocation native AI execution runtime consolidating Vision, Speech, and LLM compute pipelines behind a single Project Panama FFM boundary.
Note v1.0.0 Stable — Unified Hardware Orchestration & Zero-Allocation ABI
libargus is an ultra-lean, high-performance, model-agnostic inference wrapper engineered to consolidate LLM text generation, Whisper-based speech-to-text (ASR), Speech-LLM text-to-speech (TTS), and bleeding-edge Multimodal (Vision, Audio, and Video) encoding and evaluation pipelines into a single process-global native execution runtime.
Built directly on top of the modular GGML and llama.cpp (libmtmd) compute engines, libargus provides a unified, thread-safe C API designed explicitly for frictionless, zero-copy compilation alongside modern unmanaged orchestration frameworks, featuring out-of-the-box structural alignment for the JDK 22+ Project Panama Foreign Function & Memory (FFM) API.
Core Architectural Pillars
Process-Global Backend Singularity: Eliminates VRAM fragmentation and multi-context driver race conditions by orchestrating a singular, shared initialization pathway ( ggml_backend_load_all() ) across text, audio, speech, and multimodal subsystems.
Eliminates VRAM fragmentation and multi-context driver race conditions by orchestrating a singular, shared initialization pathway ( ) across text, audio, speech, and multimodal subsystems. Decoupled Weights & Execution: Separates model weight loading ( argus_model_t ) from evaluation context memory states ( argus_context_t ), allowing model reuse across multiple concurrent sessions.
Separates model weight loading ( ) from evaluation context memory states ( ), allowing model reuse across multiple concurrent sessions. Bleeding-Edge Multimodal Projectors: Integrates the new libmtmd C++ engine to ingest raw bitmaps, audio PCM arrays, and video files/streams. Tokenizes prompts and media into a unified chunk sequence, executes projection on the GPU, and automatically configures M-RoPE position grids and non-causal attention matrices.
Integrates the new C++ engine to ingest raw bitmaps, audio PCM arrays, and video files/streams. Tokenizes prompts and media into a unified chunk sequence, executes projection on the GPU, and automatically configures M-RoPE position grids and non-causal attention matrices. Unmanaged Video Iteration Pipe: Decodes and streams video files frame-by-frame using internal FFmpeg subprocess pipes, yielding raw RGB frames or localized timestamp text chunks (e.g., [12m34s] ) at a specified target frame rate.
... continue reading