Configuration Overview

Learn how to configure NeMo Guardrails for your LLM applications

NeMo Guardrails uses a structured configuration approach to define guardrails for your LLM applications. A guardrails configuration defines the LLM(s) to be used, the rails that should be active, and custom settings.

Configuration Structure

The standard structure for a guardrails configuration folder looks like this:

.
├── config
│   ├── actions.py
│   ├── config.py
│   ├── config.yml
│   ├── rails.co
│   ├── ...

Main Configuration Files

A complete guardrails configuration typically consists of three main components:

config.yml

General configuration options including LLM models, active rails, and custom settings

.co files

Colang definitions that define various types of rails and conversation flows

actions.py

Custom Python actions that can be called from your guardrails

Configuration Loading

To use a guardrails configuration in your application:

from nemoguardrails import LLMRails, RailsConfig

# Load a guardrails configuration from the specified path
config = RailsConfig.from_path("PATH/TO/CONFIG")
rails = LLMRails(config)

completion = rails.generate(
    messages=[{"role": "user", "content": "Hello world!"}]
)

Sample output:

{"role": "assistant", "content": "Hi! How can I help you?"}

Types of Guardrails

NeMo Guardrails supports five main types of guardrails that can be configured:

1. Input Rails

   Applied to user input; can reject or alter input (e.g., mask sensitive data, rephrase)

   rails:
     input:
       flows:
         - check jailbreak
         - mask sensitive data on input
   
2. Dialog Rails

   Influence how the LLM is prompted; operate on canonical form messages and determine conversation flow

   rails:
     dialog:
       single_call:
         enabled: False
   
3. Retrieval Rails

   Applied to retrieved chunks in RAG scenarios; can reject or alter chunks

   rails:
     retrieval:
       flows:
         - check relevance
   
4. Execution Rails

   Applied to input/output of custom actions (tools) called by the LLM

5. Output Rails

   Applied to LLM output; can reject or alter output before returning to user

   rails:
     output:
       flows:
         - self check facts
         - self check hallucination
   

Basic Configuration Example

Here’s a minimal example from examples/configs/sample/config.yml:

instructions:
  - type: general
    content: |
      Below is a conversation between a bot and a user. The bot is talkative and
      quirky. If the bot does not know the answer to a question, it truthfully says it does not know.

sample_conversation: |
  user "Hello there!"
    express greeting
  bot express greeting
    "Hello! How can I assist you today?"
  user "What can you do for me?"
    ask about capabilities
  bot respond about capabilities
    "I am an AI assistant built to help you."

models:
  - type: main
    engine: openai
    model: gpt-3.5-turbo-instruct

rails:
  input:
    flows:
      - dummy input rail

  output:
    flows:
      - dummy output rail

Next Steps

config.yml Schema

Learn about all available configuration options

Rails Definition

Define custom rails using Colang

Custom Actions

Create Python actions for your guardrails

LLM Configuration

Configure different LLM providers