ngrok: AI & API Gateway - Secure Tunnels & Traffic

Summary

ngrok has evolved from a simple localhost tunneling tool into a full cloud networking gateway that handles API routing, AI model traffic, DDoS protection, and secure connectivity - all configured through a declarative Traffic Policy system. The platform now competes with nginx + Cloudflare + VPN stacks by consolidating all traffic management into a single gateway with CEL-based routing rules.

Key Insight

• ngrok now positions itself as a unified replacement for nginx, network load balancers, VPNs, model routers, and Cloudflare - a single gateway for all traffic types (HTTP, TLS, TCP, gRPC, WebSocket, MQTT, SSH)
• Traffic Policy is the core differentiator: a CEL-expression-based rules engine that processes requests at different lifecycle phases (on_http_request, on_http_response), enabling rate limiting, WAF (OWASP CRS), geo-blocking, URL rewriting, and custom responses without Lua plugins or cryptic nginx configs
• AI Gateway (ngrok.ai) is the newest addition - routes, secures, and transforms traffic to any AI model, suggesting they’re targeting the model-router/gateway market
• Endpoint pooling for load balancing is notably simple: start endpoints with the same URL and they self-register; stop them and they’re removed
• The “dogfooding” example is revealing - ngrok.com itself runs on ngrok, with their full Traffic Policy config shown (rate limiting, WAF, geo-blocking, path-based routing to different backends like S3 for docs, CMS for legacy pages)
• Device Gateway use case is interesting for IoT/field deployments - SSH/RDP into remote devices without opening inbound ports
• Site-to-Site Connectivity targets SaaS companies needing to access customer network APIs/databases without full VPN access - scoped to just the endpoints needed
• Native SDKs available for embedding tunnels directly in application code (import "ngrok") instead of running a sidecar agent