LiteLLM

LiteLLM supports calling 100+ LLMs using the OpenAI Input/Output Format. LiteLLM documentation
LiteLLM + Nebius Token Factory documentation

Prerequisites

Create an API key to authorize requests to Nebius Token Factory.
Save the API key into a NEBIUS_API_KEY environment variable:
export NEBIUS_API_KEY="<API_key>"
Install the litellm package:
pip install litellm

Create a chat completion

Paste the following code into your script:

from litellm import completion

response = completion(
    model="nebius/meta-llama/Llama-3.3-70B-Instruct",
    messages=[
        {
            "role": "user",
            "content": "What is the capital of Netherlands?",
        }
    ],
    temperature=0.1,
)

print (response.choices[0].message.content)

Checkout an example notebook in our cookbook repository.

⌘I

Get Started

AI Models Inference

Observability

Post-training

Data Lab

Sandboxes

Teams & Access Management

Other Capabilities

Integrations

Prerequisites

Create a chat completion

​Prerequisites

​Create a chat completion

Prerequisites

Create a chat completion