Red Hat shows a proxy pattern for adding NeMo Guardrails to LangGraph agents

Red Hat has published a hands-on walkthrough for adding NeMo Guardrails to a LangGraph agent on Red Hat OpenShift AI without rewriting the graph. The video demonstration, dated September 21, places a guardrails proxy between the agent and an in-cluster vLLM endpoint and shows the guarded and unguarded paths side by side.

What the example changes

The accompanying Guardrailed Agent example uses NeMo Guardrails as a passthrough proxy between a banking customer-service agent and its language model. Red Hat’s repository says that this pattern leaves the agent source unchanged while checking requests and responses against configured rails.

The demonstration layers four checks in sequence: a regex input check, content-safety classification, a topic-boundary check and an output-safety check. In the video’s test, an unguarded agent answers a check-fraud prompt, while the guarded route refuses it; optional tracing then identifies the rail that fired. The repository documents two profiles: a local profile that uses the same model for self-checks, and a nemoguard profile that assigns dedicated classifier roles to the safety layers.

The order matters operationally. According to the example’s architecture notes, a blocking rail stops the chain immediately, allowing a low-cost regex match to reject known patterns before model-backed content and topic checks run. The same repository also documents separate responses for blocked traffic and for an unavailable guardrails service, making failure behavior visible to application teams rather than silently bypassing the filter.

The OpenShift AI integration point

Red Hat’s OpenShift AI 3.5 documentation says NeMo Guardrails is included with OpenShift AI and can be deployed through a NemoGuardrails custom resource managed by the TrustyAI Operator. The service exposes endpoints for guarded chat completions, pass-or-block checks over message histories, and standalone checks that can transform content such as personally identifiable information.

There is an important design boundary in the same documentation: the guarded chat-completions endpoint is not universally transparent and may alter or drop request parameters. Red Hat recommends separating model inference from calls to the guardrail-check endpoint when an application needs transparent payload handling or advanced features such as tool calling.

For platform teams, the useful takeaway is not that one rail configuration fits every agent. It is that the walkthrough provides a reproducible integration seam: keep application orchestration in LangGraph, deploy policy enforcement as an OpenShift-managed service, and test both blocked-content behavior and guardrail outages before promotion. Teams adopting the example should tune topic policy and classifier choices for their own domain rather than treating the banking defaults as production policy.