> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/NVIDIA-NeMo/Guardrails/llms.txt
> Use this file to discover all available pages before exploring further.
# Docker Deployment
> Deploy NeMo Guardrails using Docker containers for consistent and isolated environments.
# Docker Deployment
Docker provides a reliable way to deploy NeMo Guardrails with all dependencies packaged in a container. This guide shows you how to build and run NeMo Guardrails using Docker.
## Prerequisites
* Docker installed on your system ([Get Docker](https://docs.docker.com/get-docker/))
* NeMo Guardrails source code or configuration files
* Basic familiarity with Docker commands
## Official Dockerfile
NeMo Guardrails includes an official Dockerfile that sets up the complete environment:
```dockerfile theme={null}
# syntax=docker/dockerfile:experimental
FROM python:3.12-slim
RUN apt-get update && apt-get install -y --no-install-recommends git gcc g++ \
&& rm -rf /var/lib/apt/lists/*
# Set POETRY_VERSION environment variable
ENV POETRY_VERSION=1.8.2
RUN if [ "$(uname -m)" = "x86_64" ]; then \
export ANNOY_COMPILER_ARGS="-D_CRT_SECURE_NO_WARNINGS,-DANNOYLIB_MULTITHREADED_BUILD,-march=x86-64"; \
fi
# Install Poetry
RUN pip install --no-cache-dir poetry==$POETRY_VERSION
# Copy project files
WORKDIR /nemoguardrails
COPY pyproject.toml poetry.lock /nemoguardrails/
# Copy the rest of the project files
COPY . /nemoguardrails
RUN poetry config virtualenvs.create false && poetry install --all-extras --no-interaction --no-ansi && poetry install --with dev --no-interaction --no-ansi
# Make port 8000 available to the world outside this container
EXPOSE 8000
# We copy the example bot configurations
WORKDIR /config
COPY ./examples/bots /config
# Run app.py when the container launches
WORKDIR /nemoguardrails
# Download the 'all-MiniLM-L6-v2' model
RUN python -c "from fastembed.embedding import FlagEmbedding; FlagEmbedding('sentence-transformers/all-MiniLM-L6-v2');"
RUN nemoguardrails --help
# Ensure the entry point is installed as a script
RUN poetry install --all-extras --no-interaction --no-ansi
ENTRYPOINT ["poetry", "run", "nemoguardrails"]
CMD ["server", "--verbose", "--config=/config"]
```
## Building the Docker Image
If you haven't already, clone the NeMo Guardrails repository:
```bash theme={null}
git clone https://github.com/NVIDIA/NeMo-Guardrails.git
cd NeMo-Guardrails
```
Build the Docker image using the provided Dockerfile:
```bash theme={null}
docker build -t nemoguardrails:latest .
```
This process may take several minutes as it installs all dependencies.
Verify the image was created successfully:
```bash theme={null}
docker images | grep nemoguardrails
```
The Docker image includes the `all-MiniLM-L6-v2` embedding model pre-downloaded for faster startup times.
## Running the Container
### Using Example Configurations
The Docker image comes with example bot configurations. Run with default settings:
```bash theme={null}
docker run -p 8000:8000 nemoguardrails:latest
```
### Using Custom Configurations
Mount your own configuration directory:
Ensure your configuration directory contains the necessary files:
```
/path/to/your/config/
├── config.yml
├── config.co
└── kb/
```
Mount your configuration directory to the container:
```bash theme={null}
docker run -p 8000:8000 \
-v /path/to/your/config:/config \
nemoguardrails:latest server --config=/config
```
Test your guardrails server:
```bash theme={null}
curl -X POST http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [{"role": "user", "content": "Hello!"}]
}'
```
## Docker Compose
For more complex deployments, use Docker Compose:
```yaml theme={null}
version: '3.8'
services:
nemoguardrails:
image: nemoguardrails:latest
ports:
- "8000:8000"
volumes:
- ./config:/config
environment:
- OPENAI_API_KEY=${OPENAI_API_KEY}
command: server --verbose --config=/config
restart: unless-stopped
```
Run with:
```bash theme={null}
docker-compose up -d
```
## Environment Variables
Pass environment variables for API keys and configuration:
```bash theme={null}
docker run -p 8000:8000 \
-e OPENAI_API_KEY="your-api-key" \
-e COHERE_API_KEY="your-cohere-key" \
-v /path/to/config:/config \
nemoguardrails:latest server --config=/config
```
Never hardcode API keys in your Dockerfile or commit them to version control. Use environment variables or secrets management.
## Resource Limits
Set resource constraints for the container:
```bash theme={null}
docker run -p 8000:8000 \
--memory="2g" \
--cpus="1.5" \
-v /path/to/config:/config \
nemoguardrails:latest server --config=/config
```
## Container Management
### View Running Containers
```bash theme={null}
docker ps
```
### View Container Logs
```bash theme={null}
docker logs
```
### Stop the Container
```bash theme={null}
docker stop
```
### Access Container Shell
```bash theme={null}
docker exec -it /bin/bash
```
## Production Considerations
When deploying to production:
1. **Use Multi-Stage Builds**: Optimize image size by using multi-stage builds
2. **Pin Dependencies**: Use specific version tags instead of `latest`
3. **Health Checks**: Add Docker health check configurations
4. **Logging**: Configure proper logging drivers
5. **Security**: Run as non-root user and scan images for vulnerabilities
See the [Production Deployment](./production) guide for more details.
## Troubleshooting
### Container Exits Immediately
Check the logs for errors:
```bash theme={null}
docker logs
```
### Port Already in Use
Use a different host port:
```bash theme={null}
docker run -p 8080:8000 nemoguardrails:latest
```
### Permission Denied for Mounted Volumes
Ensure proper permissions on your host directory:
```bash theme={null}
chmod -R 755 /path/to/config
```
## Next Steps
Learn production best practices
Configure your guardrails