Components
2 minute read
Components
LeapfrogAI API
LeapfrogAI offers an API closely aligned with OpenAI’s, facilitating seamless compatibility for tools developed with OpenAI/ChatGPT to operate seamlessly with a LeapfrogAI backend. The LeapfrogAI API is a Python API that exposes LLM backends, via FastAPI and gRPC, in the OpenAI API specification.
Backend
LeapfrogAI offers several backends for a variety of use cases:
| Backend | AMD64 Support | ARM64 Support | Cuda Support | Docker Ready | K8s Ready | Zarf Ready |
|---|---|---|---|---|---|---|
| llama-cpp-python | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| whisper | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| text-embeddings | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| vllm | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |
Flavors
Each component has different images and values that refer to a specific image registry and/or hardening source. These images are packaged using Zarf Flavors:
upstream: uses upstream vendor images from open source container registries and repositories- 🚧
registry1: uses IronBank hardened images from the Repo1 harbor registry
Below is the current component flavors list:
| Component | upstream | registry1 |
|---|---|---|
| api | ✅ | ✅ |
| ui | ✅ | 🚧 |
| supabase | ✅ | 🚧 |
| migrations | ✅ | 🚧 |
| llama-cpp-python | ✅ | 🚧 |
| whisper | ✅ | 🚧 |
| text-embeddings | ✅ | 🚧 |
| vllm | ✅ | 🚧 |
| vllm | ✅ | 🚧 |
Flavors with any components labelled as 🚧 are not available as a quick start bundle deployment yet. Please refer to the DEVELOPMENT.md for instructions on how to build a component’s Zarf package for local testing.
Artifact Support
LeapfrogAI contains built-in embeddings for RAG and transcription / translation solutions that can handle many different file types. Many of these capabilities are accessible via the LeapfrogAI API. The supported artifact types are as follows:
Transcription / Translation
- All formats supported by
ffmpegas listed usingffmpeg -formats, e.g.,.mp3,.wav,.mp4, etc.
Embeddings for RAG
.pdf.txt.html.htm.csv.md.doc.docx.xlsx.xls.pptx.ppt
Software Development Kit
The LeapfrogAI SDK offers a standardized collection of Protobuf and Python utilities designed to facilitate the implementation of backends and gRPC. Please see the LeapfrogAI SDK sub-directory for the source code and details.
User Interface
LeapfrogAI offers user-friendly interfaces tailored for common use-cases, including chat, summarization, and transcription, providing accessible options for users to initiate these tasks. Please see the LeapfrogAI UI GitHub repository for additional information.