Chat Completions API

POST /v1/chat/completions

Create a chat completion with guardrails applied. This endpoint is compatible with the OpenAI Chat Completions API with additional guardrails-specific extensions.

Request

string

required

The LLM model to use for chat completion (e.g., “gpt-4o”, “llama-3.1-8b”).

array

The list of messages in the current conversation.

[
  {"role": "user", "content": "Hello!"},
  {"role": "assistant", "content": "Hi there!"}
]

boolean

default:"false"

If set, partial message deltas will be sent as server-sent events.

integer

The maximum number of tokens to generate.

number

Sampling temperature to use (0.0 to 2.0).

number

Top-p sampling parameter (0.0 to 1.0).

string | array

Stop sequences where the API will stop generating further tokens.

number

Presence penalty parameter (-2.0 to 2.0).

number

Frequency penalty parameter (-2.0 to 2.0).

Guardrails Extensions

object

Guardrails-specific options:

string

The guardrails configuration ID to use.

array

List of configuration IDs to combine. Cannot be used with config_id.

string

The ID of an existing thread to continue (minimum 16 characters).

object

Additional context data for the conversation.

{
  "user_name": "Alice",
  "user_id": "12345"
}

GenerationOptions

Additional generation options:

rails: Which rails to enable ({"input": true, "output": true})
log: Logging options ({"activated_rails": true, "llm_calls": true})
output_vars: Variables to extract from context

object

State object to continue the interaction. Must contain events or state key.

Response

string

Unique identifier for the chat completion.

string

Always “chat.completion”.

integer

Unix timestamp of when the completion was created.

string

The model used for the completion.

array

Array of completion choices.

integer

The index of this choice.

object

The generated message.

string

Always “assistant”.

string

The content of the message.

array

Tool calls generated by the model (if any).

string

The reason the generation stopped: “stop”, “length”, or “content_filter”.

object

Guardrails-specific output data:

string

The configuration ID that was used.

object

Updated state object for continuing the conversation.

object

Generation log data (if requested):

activated_rails: List of rails that were activated
llm_calls: Details of LLM calls made
stats: Performance statistics

curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "Hello!"}
    ],
    "guardrails": {
      "config_id": "my-config"
    }
  }'

curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "Tell me a story"}
    ],
    "stream": true,
    "guardrails": {
      "config_id": "my-config"
    }
  }'

curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "What is my name?"}
    ],
    "guardrails": {
      "config_id": "my-config",
      "context": {
        "user_name": "Alice",
        "user_id": "12345"
      }
    }
  }'

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed"  # API key not required for local server
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "Hello!"}
    ],
    extra_body={
        "guardrails": {
            "config_id": "my-config"
        }
    }
)

print(response.choices[0].message.content)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed"
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "Hello!"}
    ],
    extra_body={
        "guardrails": {
            "config_id": "my-config",
            "options": {
                "log": {
                    "activated_rails": True,
                    "llm_calls": True
                }
            }
        }
    }
)

# Access guardrails log data
if hasattr(response, 'guardrails') and response.guardrails.log:
    print("Activated rails:", response.guardrails.log.activated_rails)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed"
)

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "Tell me a story"}
    ],
    stream=True,
    extra_body={
        "guardrails": {
            "config_id": "my-config"
        }
    }
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Response Examples

{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      },
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "guardrails": {
    "config_id": "my-config",
    "state": {
      "events": [...]
    },
    "log": {
      "stats": {
        "total_llm_calls": 2,
        "total_time": 1.5
      }
    }
  }
}

{
  "id": "chatcmpl-124",
  "object": "chat.completion",
  "created": 1677652289,
  "model": "gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "I'm sorry, I can't help with that."
      },
      "finish_reason": "content_filter",
      "logprobs": null
    }
  ],
  "guardrails": {
    "config_id": "my-config",
    "log": {
      "activated_rails": [
        {
          "type": "input",
          "name": "jailbreak_detection",
          "decision": "blocked"
        }
      ]
    }
  }
}

Error Responses

object

string

Human-readable error message.

string

Error type: “invalid_request_error”, “authentication_error”, “server_error”, etc.

string

Error code.

{
  "error": {
    "message": "No guardrails config_id provided and server has no default configuration",
    "type": "invalid_request_error",
    "code": "missing_config"
  }
}

{
  "error": {
    "message": "Invalid state format: state must contain 'events' or 'state' key",
    "type": "invalid_request_error",
    "code": "invalid_state"
  }
}

Python API

Server API

CLI Reference

POST /v1/chat/completions

Request

Guardrails Extensions

Response

Response Examples

Error Responses

​POST /v1/chat/completions

​Request

​Guardrails Extensions

​Response

​Response Examples

​Error Responses

POST /v1/chat/completions

Request

Guardrails Extensions

Response

Response Examples

Error Responses