LLMRails

Main class for interacting with NeMo Guardrails

LLMRails

The LLMRails class is the primary interface for creating and managing guardrails in NeMo Guardrails. It provides methods for generating responses with guardrails applied.

Constructor

LLMRails(
    config: RailsConfig,
    llm: Optional[Union[BaseLLM, BaseChatModel]] = None,
    verbose: bool = False
)

config · RailsConfig · required

A RailsConfig object containing the guardrails configuration.

llm · Optional[Union[BaseLLM, BaseChatModel]]

An optional LLM engine to use. If provided, this will be used as the main LLM and will take precedence over any main LLM specified in the config.

verbose · bool · default: "False"

Whether the logging should be verbose or not.

Methods

generate_async

Generate a completion or next message asynchronously.

async def generate_async(
    prompt: Optional[str] = None,
    messages: Optional[List[dict]] = None,
    options: Optional[Union[dict, GenerationOptions]] = None,
    state: Optional[Union[dict, State]] = None,
    streaming_handler: Optional[StreamingHandler] = None
) -> Union[str, dict, GenerationResponse, Tuple[dict, dict]]

prompt · Optional[str]

The prompt to be used for completion.

messages · Optional[List[dict]]

The history of messages to be used to generate the next message. Messages have the format:

[
    {"role": "context", "content": {"user_name": "John"}},
    {"role": "user", "content": "Hello! How are you?"},
    {"role": "assistant", "content": "I am fine, thank you!"},
    {"role": "event", "event": {"type": "UserSilent"}}
]

options · Optional[Union[dict, GenerationOptions]]

Options specific for the generation.

state · Optional[Union[dict, State]]

The state object that should be used as the starting point.

streaming_handler · Optional[StreamingHandler]

If specified, and the config supports streaming, the provided handler will be used for streaming.

response · Union[str, dict, GenerationResponse, Tuple[dict, dict]]

The completion (when a prompt is provided) or the next message.

Example - Basic Usage

from nemoguardrails import RailsConfig, LLMRails

config = RailsConfig.from_path("path/to/config")
rails = LLMRails(config)

response = await rails.generate_async(
    messages=[{"role": "user", "content": "Hello!"}]
)

generate

Synchronous version of generate_async.

def generate(
    prompt: Optional[str] = None,
    messages: Optional[List[dict]] = None,
    options: Optional[Union[dict, GenerationOptions]] = None,
    state: Optional[dict] = None
) -> Union[str, dict, GenerationResponse, Tuple[dict, dict]]

stream_async

Stream the response tokens asynchronously.

def stream_async(
    prompt: Optional[str] = None,
    messages: Optional[List[dict]] = None,
    options: Optional[Union[dict, GenerationOptions]] = None,
    state: Optional[Union[dict, State]] = None,
    include_metadata: bool = False,
    generator: Optional[AsyncIterator[str]] = None
) -> AsyncIterator[Union[str, dict]]

include_metadata · bool · default: "False"

Whether to include metadata in the streamed chunks.

generator · Optional[AsyncIterator[str]]

If provided, uses this external generator for streaming.

chunks · AsyncIterator[Union[str, dict]]

An async iterator that yields token chunks as strings, or dicts if include_metadata=True.

Example - Streaming

from nemoguardrails import RailsConfig, LLMRails

config = RailsConfig.from_path("path/to/config")
rails = LLMRails(config)

async for chunk in rails.stream_async(
    messages=[{"role": "user", "content": "Tell me a story"}]
):
    print(chunk, end="")

check_async

Run rails on messages to check for policy violations.

async def check_async(
    messages: List[dict],
    rail_types: Optional[List[RailType]] = None
) -> RailsResult

messages · List[dict] · required

List of message dicts with ‘role’ and ‘content’ fields.

rail_types · Optional[List[RailType]]

Optional list of rail types to run (e.g., [RailType.INPUT] or [RailType.OUTPUT]). When not provided, automatically determines which rails to run based on message roles.

result · RailsResult

Contains:

• status: PASSED, MODIFIED, or BLOCKED
• content: The final content after rails processing
• rail: Name of the rail that blocked (if blocked)

Example - Check Input

from nemoguardrails import RailsConfig, LLMRails

config = RailsConfig.from_path("path/to/config")
rails = LLMRails(config)

result = await rails.check_async([
    {"role": "user", "content": "Harmful message"}
])

if result.status == RailStatus.BLOCKED:
    print(f"Blocked by: {result.rail}")

check

Synchronous version of check_async.

def check(
    messages: List[dict],
    rail_types: Optional[List[RailType]] = None
) -> RailsResult

register_action

Register a custom action for the rails configuration.

def register_action(
    action: Callable,
    name: Optional[str] = None
) -> Self

action · Callable · required

The action function to register.

name · Optional[str]

The name to use for the action. If not provided, uses the function name.

Example - Register Action

from nemoguardrails import RailsConfig, LLMRails

config = RailsConfig.from_path("path/to/config")
rails = LLMRails(config)

def custom_action(context: dict):
    return "Custom response"

rails.register_action(custom_action, name="my_action")

register_filter

Register a custom filter for the rails configuration.

def register_filter(
    filter_fn: Callable,
    name: Optional[str] = None
) -> Self

register_embedding_provider

Register a custom embedding provider.

def register_embedding_provider(
    cls: Type[EmbeddingModel],
    name: Optional[str] = None
) -> Self

cls · Type[EmbeddingModel] · required

The embedding model class.

name · Optional[str]

The name of the embedding engine.

explain

Returns detailed information about the latest generation.

def explain() -> ExplainInfo

info · ExplainInfo

An object containing detailed explanation information including LLM calls, activated rails, and Colang history.

Attributes

config · RailsConfig

The rails configuration object.

llm · Optional[Union[BaseLLM, BaseChatModel]]

The main LLM engine being used.

runtime · Runtime

The Colang runtime instance.

verbose · bool

Whether verbose logging is enabled.