> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/NVIDIA-NeMo/Guardrails/llms.txt
> Use this file to discover all available pages before exploring further.

# Chat Completions API

> OpenAI-compatible chat completions endpoint with guardrails

## POST /v1/chat/completions

Create a chat completion with guardrails applied. This endpoint is compatible with the OpenAI Chat Completions API with additional guardrails-specific extensions.

### Request

<ParamField body="model" type="string" required>
  The LLM model to use for chat completion (e.g., "gpt-4o", "llama-3.1-8b").
</ParamField>

<ParamField body="messages" type="array">
  The list of messages in the current conversation.

  ```json theme={null}
  [
    {"role": "user", "content": "Hello!"},
    {"role": "assistant", "content": "Hi there!"}
  ]
  ```
</ParamField>

<ParamField body="stream" type="boolean" default="false">
  If set, partial message deltas will be sent as server-sent events.
</ParamField>

<ParamField body="max_tokens" type="integer">
  The maximum number of tokens to generate.
</ParamField>

<ParamField body="temperature" type="number">
  Sampling temperature to use (0.0 to 2.0).
</ParamField>

<ParamField body="top_p" type="number">
  Top-p sampling parameter (0.0 to 1.0).
</ParamField>

<ParamField body="stop" type="string | array">
  Stop sequences where the API will stop generating further tokens.
</ParamField>

<ParamField body="presence_penalty" type="number">
  Presence penalty parameter (-2.0 to 2.0).
</ParamField>

<ParamField body="frequency_penalty" type="number">
  Frequency penalty parameter (-2.0 to 2.0).
</ParamField>

### Guardrails Extensions

<ParamField body="guardrails" type="object">
  Guardrails-specific options:

  <ParamField body="guardrails.config_id" type="string">
    The guardrails configuration ID to use.
  </ParamField>

  <ParamField body="guardrails.config_ids" type="array">
    List of configuration IDs to combine. Cannot be used with `config_id`.
  </ParamField>

  <ParamField body="guardrails.thread_id" type="string">
    The ID of an existing thread to continue (minimum 16 characters).
  </ParamField>

  <ParamField body="guardrails.context" type="object">
    Additional context data for the conversation.

    ```json theme={null}
    {
      "user_name": "Alice",
      "user_id": "12345"
    }
    ```
  </ParamField>

  <ParamField body="guardrails.options" type="GenerationOptions">
    Additional generation options:

    * `rails`: Which rails to enable (`{"input": true, "output": true}`)
    * `log`: Logging options (`{"activated_rails": true, "llm_calls": true}`)
    * `output_vars`: Variables to extract from context
  </ParamField>

  <ParamField body="guardrails.state" type="object">
    State object to continue the interaction. Must contain `events` or `state` key.
  </ParamField>
</ParamField>

### Response

<ResponseField name="id" type="string">
  Unique identifier for the chat completion.
</ResponseField>

<ResponseField name="object" type="string">
  Always "chat.completion".
</ResponseField>

<ResponseField name="created" type="integer">
  Unix timestamp of when the completion was created.
</ResponseField>

<ResponseField name="model" type="string">
  The model used for the completion.
</ResponseField>

<ResponseField name="choices" type="array">
  Array of completion choices.

  <ResponseField name="choices[].index" type="integer">
    The index of this choice.
  </ResponseField>

  <ResponseField name="choices[].message" type="object">
    The generated message.

    <ResponseField name="choices[].message.role" type="string">
      Always "assistant".
    </ResponseField>

    <ResponseField name="choices[].message.content" type="string">
      The content of the message.
    </ResponseField>

    <ResponseField name="choices[].message.tool_calls" type="array">
      Tool calls generated by the model (if any).
    </ResponseField>
  </ResponseField>

  <ResponseField name="choices[].finish_reason" type="string">
    The reason the generation stopped: "stop", "length", or "content\_filter".
  </ResponseField>
</ResponseField>

<ResponseField name="guardrails" type="object">
  Guardrails-specific output data:

  <ResponseField name="guardrails.config_id" type="string">
    The configuration ID that was used.
  </ResponseField>

  <ResponseField name="guardrails.state" type="object">
    Updated state object for continuing the conversation.
  </ResponseField>

  <ResponseField name="guardrails.log" type="object">
    Generation log data (if requested):

    * `activated_rails`: List of rails that were activated
    * `llm_calls`: Details of LLM calls made
    * `stats`: Performance statistics
  </ResponseField>
</ResponseField>

<CodeGroup>
  ```bash cURL - Basic Request theme={null}
  curl -X POST http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
      "model": "gpt-4o",
      "messages": [
        {"role": "user", "content": "Hello!"}
      ],
      "guardrails": {
        "config_id": "my-config"
      }
    }'
  ```

  ```bash cURL - With Streaming theme={null}
  curl -X POST http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
      "model": "gpt-4o",
      "messages": [
        {"role": "user", "content": "Tell me a story"}
      ],
      "stream": true,
      "guardrails": {
        "config_id": "my-config"
      }
    }'
  ```

  ```bash cURL - With Context theme={null}
  curl -X POST http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
      "model": "gpt-4o",
      "messages": [
        {"role": "user", "content": "What is my name?"}
      ],
      "guardrails": {
        "config_id": "my-config",
        "context": {
          "user_name": "Alice",
          "user_id": "12345"
        }
      }
    }'
  ```

  ```python Python SDK theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="http://localhost:8000/v1",
      api_key="not-needed"  # API key not required for local server
  )

  response = client.chat.completions.create(
      model="gpt-4o",
      messages=[
          {"role": "user", "content": "Hello!"}
      ],
      extra_body={
          "guardrails": {
              "config_id": "my-config"
          }
      }
  )

  print(response.choices[0].message.content)
  ```

  ```python Python - With Logging theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="http://localhost:8000/v1",
      api_key="not-needed"
  )

  response = client.chat.completions.create(
      model="gpt-4o",
      messages=[
          {"role": "user", "content": "Hello!"}
      ],
      extra_body={
          "guardrails": {
              "config_id": "my-config",
              "options": {
                  "log": {
                      "activated_rails": True,
                      "llm_calls": True
                  }
              }
          }
      }
  )

  # Access guardrails log data
  if hasattr(response, 'guardrails') and response.guardrails.log:
      print("Activated rails:", response.guardrails.log.activated_rails)
  ```

  ```python Python - Streaming theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="http://localhost:8000/v1",
      api_key="not-needed"
  )

  stream = client.chat.completions.create(
      model="gpt-4o",
      messages=[
          {"role": "user", "content": "Tell me a story"}
      ],
      stream=True,
      extra_body={
          "guardrails": {
              "config_id": "my-config"
          }
      }
  )

  for chunk in stream:
      if chunk.choices[0].delta.content:
          print(chunk.choices[0].delta.content, end="")
  ```
</CodeGroup>

### Response Examples

<CodeGroup>
  ```json Non-Streaming Response theme={null}
  {
    "id": "chatcmpl-123",
    "object": "chat.completion",
    "created": 1677652288,
    "model": "gpt-4o",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": "Hello! How can I help you today?"
        },
        "finish_reason": "stop",
        "logprobs": null
      }
    ],
    "guardrails": {
      "config_id": "my-config",
      "state": {
        "events": [...]
      },
      "log": {
        "stats": {
          "total_llm_calls": 2,
          "total_time": 1.5
        }
      }
    }
  }
  ```

  ```json Blocked Response theme={null}
  {
    "id": "chatcmpl-124",
    "object": "chat.completion",
    "created": 1677652289,
    "model": "gpt-4o",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": "I'm sorry, I can't help with that."
        },
        "finish_reason": "content_filter",
        "logprobs": null
      }
    ],
    "guardrails": {
      "config_id": "my-config",
      "log": {
        "activated_rails": [
          {
            "type": "input",
            "name": "jailbreak_detection",
            "decision": "blocked"
          }
        ]
      }
    }
  }
  ```
</CodeGroup>

## Error Responses

<ResponseField name="error" type="object">
  <ResponseField name="error.message" type="string">
    Human-readable error message.
  </ResponseField>

  <ResponseField name="error.type" type="string">
    Error type: "invalid\_request\_error", "authentication\_error", "server\_error", etc.
  </ResponseField>

  <ResponseField name="error.code" type="string">
    Error code.
  </ResponseField>
</ResponseField>

<CodeGroup>
  ```json 422 - Invalid Request theme={null}
  {
    "error": {
      "message": "No guardrails config_id provided and server has no default configuration",
      "type": "invalid_request_error",
      "code": "missing_config"
    }
  }
  ```

  ```json 422 - Invalid State theme={null}
  {
    "error": {
      "message": "Invalid state format: state must contain 'events' or 'state' key",
      "type": "invalid_request_error",
      "code": "invalid_state"
    }
  }
  ```
</CodeGroup>