The CBORG API provides access to a set of AI models hosted on-premises at Lawrence Berkeley National Laboratory. These models run entirely within LBL infrastructure — no data leaves the lab network. These models are also free to use with no API cost.

Recommended Model Aliases

We recommend using the lbl/cborg-* model aliases rather than referencing underlying model names directly. The aliases are mapped to specific model configurations that may be updated over time (e.g. when a newer or better model becomes available). Using the aliases ensures your application remains robust against future changes and version updates without requiring code changes on your end.

Chat & Reasoning Models

Specialized Models

Embedding Models

The following embedding models are available on-premises:

Because nomic-embed-text and nomic-embed-vision share the same embedding space, you can embed both text and images and compare them directly — useful for multimodal search and retrieval applications.

General Usage Tips

Parallelism

Limit your application to 5 parallel requests to on-premises models. Exceeding this will result in a 429 Rate Limit Exceeded error.

Long-Running & Agentic Workloads

It is perfectly fine to run agents and automated pipelines around the clock against the on-premises models. There is no time-of-day restriction.

Handling Rate Limit and Network Errors

If you receive a 429 Too Many Requests error, use exponential backoff to retry your requests. For network-level errors (connection refused, timeouts, service unavailable), your application should retry indefinitely with a capped backoff — the service will come back online after maintenance completes.

Example using the Python tenacity library:

import openai
import os
import httpx
from tenacity import (
    retry,
    wait_exponential,
    stop_after_attempt,
    retry_if_exception_type,
    retry_any,
)

client = openai.OpenAI(
    base_url="https://api.cborg.lbl.gov",
    api_key=os.environ["CBORG_API_KEY"],
)

# Retry on rate limits (up to 6 attempts) and on network/connection errors
# (unlimited retries — wait for the service to come back after maintenance).
@retry(
    retry=retry_if_exception_type(openai.RateLimitError),
    wait=wait_exponential(multiplier=1, min=2, max=60),
    stop=stop_after_attempt(6),
)
@retry(
    retry=retry_any(
        retry_if_exception_type(openai.APIConnectionError),
        retry_if_exception_type(openai.APIStatusError),
        retry_if_exception_type(httpx.ConnectError),
        retry_if_exception_type(httpx.RemoteProtocolError),
    ),
    wait=wait_exponential(multiplier=2, min=5, max=120),
    # No stop= here: keep retrying until the service is restored
)
def chat(prompt: str) -> str:
    response = client.chat.completions.create(
        model="lbl/cborg-chat",
        messages=[{"role": "user", "content": prompt}],
    )
    return response.choices[0].message.content

print(chat("Summarize the key findings of my experiment."))

The inner decorator retries indefinitely on connection and protocol errors (backing off up to 2 minutes between attempts), so a long-running agent will automatically resume once the model service is restored after maintenance.

Quick Start Example

import openai
import os

client = openai.OpenAI(
    base_url="https://api.cborg.lbl.gov",
    api_key=os.environ["CBORG_API_KEY"],
)

# Use a cborg alias — robust against future model updates
response = client.chat.completions.create(
    model="lbl/cborg-chat",
    messages=[{"role": "user", "content": "Hello! What can you help me with?"}],
    stream=True,
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Support

For questions or assistance, contact the Science IT team:

• Email: scienceit@lbl.gov
• Google Chat: CBorg Users Chat Group