tritonserver-pytorch-backend

Updated 4 hours ago

Triton Inference Server PyTorch Backend Overview

Secure your stack with a hardened Triton Inference Server PyTorch Backend image freshly-built by Minimus. Minimus images always include the most up-to-date package version for all packages and dependencies contained in the image.

Triton Inference Server PyTorch Backend runs NVIDIA Triton Inference Server pre-configured with only the PyTorch (LibTorch/TorchScript) backend, so teams serving PyTorch models can skip the size and attack surface of a full multi-framework image built for TensorFlow, ONNX, and TensorRT as well. It still handles dynamic batching, concurrent model execution, and model versioning behind Triton's standard HTTP/gRPC API, giving teams a smaller, PyTorch-focused image to run in production.

Try It Out

Prerequisites: A Triton model repository containing your PyTorch (TorchScript) model files laid out in the format Triton expects.

Start Triton with your model repository mounted:

docker run --gpus=all --rm \
  -p 8000:8000 -p 8001:8001 -p 8002:8002 \
  -v /path/to/model_repository:/models \
  reg.mini.dev/tritonserver-pytorch-backend \
  tritonserver --model-repository=/models

Omit --gpus=all if you're only serving CPU-compatible models.

Check that the server is up and all models loaded successfully:

curl -v localhost:8000/v2/health/ready

Technical Considerations

The Triton Inference Server PyTorch Backend image provided by Minimus is a slim, security-hardened alternative to the public image from NVIDIA NGC. The images are largely interchangeable, with a few differences as noted below.

Triton Inference Server PyTorch Backend built by Minimus:

Listens on ports 8000/TCP (HTTP), 8001/TCP (gRPC), and 8002/TCP (Prometheus metrics) by default. The image does not expose these ports.
Drill down on the version specification tab to see the default user, listening ports, entrypoint, volumes, environment variables, etc.

The Payoff

A hardened, minimal image that will remain more secure for the long run and accrue vulnerabilities at a slower rate.

See the risk reduction dashboard for a detailed CVE comparison over the past 30 days.
Review the compliance report to see the default hardening and security configurations for the image.

Terms & Info

Trademark

This catalog is published by Minimus. All product names, logos, and marks, other than those belonging to Minimus, shown are owned by their respective rights holders and appear here only to identify the open source software each image contains. Minimus claims no ownership of those marks and implies no affiliation with, endorsement by, certification by, or sponsorship by any rights holder.

Disclaimer

Images are provided "as-is" without warranty of any kind. "Hardened" refers to the security configuration applied at the time of build and does not constitute a guarantee of ongoing security or absence of vulnerabilities. The free tier is provided without support, SLA, or guaranteed patching timelines. Security updates may be applied to paid subscriptions before or instead of free tier images. By pulling or using any image you agree to our Terms of Use.