Docker Deployment

Deploy NeMo Guardrails using Docker containers for consistent and isolated environments.

Docker Deployment

Docker provides a reliable way to deploy NeMo Guardrails with all dependencies packaged in a container. This guide shows you how to build and run NeMo Guardrails using Docker.

Prerequisites

• Docker installed on your system (Get Docker)
• NeMo Guardrails source code or configuration files
• Basic familiarity with Docker commands

Official Dockerfile

NeMo Guardrails includes an official Dockerfile that sets up the complete environment:

Building the Docker Image

1. Clone the Repository

   If you haven’t already, clone the NeMo Guardrails repository:

   git clone https://github.com/NVIDIA/NeMo-Guardrails.git
   cd NeMo-Guardrails
   
2. Build the Image

   Build the Docker image using the provided Dockerfile:

   docker build -t nemoguardrails:latest .
   

   This process may take several minutes as it installs all dependencies.

3. Verify the Build

   Verify the image was created successfully:

   docker images | grep nemoguardrails
   

The Docker image includes the all-MiniLM-L6-v2 embedding model pre-downloaded for faster startup times.

Running the Container

Using Example Configurations

The Docker image comes with example bot configurations. Run with default settings:

docker run -p 8000:8000 nemoguardrails:latest

Using Custom Configurations

Mount your own configuration directory:

1. Prepare Your Configuration

   Ensure your configuration directory contains the necessary files:

   /path/to/your/config/
   ├── config.yml
   ├── config.co
   └── kb/
   
2. Run with Volume Mount

   Mount your configuration directory to the container:

   docker run -p 8000:8000 \
     -v /path/to/your/config:/config \
     nemoguardrails:latest server --config=/config
   
3. Test the Deployment

   Test your guardrails server:

   curl -X POST http://localhost:8000/v1/chat/completions \
     -H "Content-Type: application/json" \
     -d '{
       "messages": [{"role": "user", "content": "Hello!"}]
     }'
   

Docker Compose

For more complex deployments, use Docker Compose:

Run with:

docker-compose up -d

Environment Variables

Pass environment variables for API keys and configuration:

docker run -p 8000:8000 \
  -e OPENAI_API_KEY="your-api-key" \
  -e COHERE_API_KEY="your-cohere-key" \
  -v /path/to/config:/config \
  nemoguardrails:latest server --config=/config

Never hardcode API keys in your Dockerfile or commit them to version control. Use environment variables or secrets management.

Resource Limits

Set resource constraints for the container:

docker run -p 8000:8000 \
  --memory="2g" \
  --cpus="1.5" \
  -v /path/to/config:/config \
  nemoguardrails:latest server --config=/config

Container Management

View Running Containers

docker ps

View Container Logs

docker logs <container-id>

Stop the Container

docker stop <container-id>

Access Container Shell

docker exec -it <container-id> /bin/bash

Production Considerations

When deploying to production:

1. Use Multi-Stage Builds: Optimize image size by using multi-stage builds
2. Pin Dependencies: Use specific version tags instead of latest
3. Health Checks: Add Docker health check configurations
4. Logging: Configure proper logging drivers
5. Security: Run as non-root user and scan images for vulnerabilities

See the Production Deployment guide for more details.

Troubleshooting

Container Exits Immediately

Check the logs for errors:

docker logs <container-id>

Port Already in Use

Use a different host port:

docker run -p 8080:8000 nemoguardrails:latest

Permission Denied for Mounted Volumes

Ensure proper permissions on your host directory:

chmod -R 755 /path/to/config

Next Steps

Production Deployment

Learn production best practices

Configuration

Configure your guardrails