BerriAI-litellm

LiteLLM is an open-source project hosted on GitHub that allows users to call all Language Model (LLM) APIs using the OpenAI format. It supports a wide range of models from different providers such as Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, and Replicate.

• Support for Multiple LLMs: LiteLLM supports over 100+ LLMs, making it a versatile tool for various language model APIs.
• Consistent Output: Regardless of the LLM used, text responses will always be available at ['choices'][0]['message']['content'].
• Load Balancing: LiteLLM can load-balance multiple deployments, for example, Azure/OpenAI, and can handle up to 1k+ requests per second.

LiteLLM offers several features that enhance its usability:

• Translation of Inputs: It translates inputs to the provider's completion and embedding endpoints.
• Exception Mapping: LiteLLM maps common exceptions across providers to the OpenAI exception types.
• Support for Streaming: LiteLLM supports streaming the model response back.

LiteLLM can be used in various scenarios where interaction with multiple LLM APIs is required. It simplifies the process of making API calls and ensures consistent output across different LLMs. It can be particularly useful in applications that require high request rates or need to switch between different LLMs.

As an open-source project, LiteLLM is available for free on GitHub. However, users should be aware that while LiteLLM itself does not have any costs associated with it, the LLM APIs that LiteLLM interacts with may have their own pricing models.

1. LangSmith: A platform for building and deploying LLM applications.
2. Metaphor Search API: An API to connect your LLM to the internet.
3. reliableGPT: An open-source project by BerriAI that provides 100% uptime and reliability from OpenAI.
4. openai-proxy: Another open-source project by BerriAI that provides a proxy server for OpenAI.
5. liteLLM-proxy: A lightweight proxy server for LiteLLM.