Hi, I am Hao Wu.

Building AI networking and MaaS infrastructure for production LLM systems.

I work on Tencent Cloud TokenHub, building MaaS infrastructure for unified model access, request governance, tenant isolation, quotas, observability, and reliability.

I am also a core committer of vllm-project/semantic-router, focusing on intelligent semantic routing across request semantics, model selection, safety governance, semantic cache, feedback learning, and production data planes.

MaaS Infrastructure Platforms

Building unified model service infrastructure across API compatibility, provider abstraction, tenant isolation, quota governance, fallback, and production observability.

Intelligent Semantic Routing

Designing routing decisions from request semantics, model capability matching, safety signals, semantic cache, feedback learning, and cost / latency / quality tradeoffs.

Cloud-Native Networking

Connecting LLM gateways to proven data-plane ideas from Kubernetes, container networking, service discovery, load balancing, and end-to-end troubleshooting.

Routing Stack

How I think about intelligent LLM traffic

The router is not just a forwarding layer. It is a decision boundary where semantics, policy, backends, cost, latency, quality, safety, and feedback meet.

01

Request Context

  • Prompt and history
  • Parameters
  • Tenant policy
  • Feedback signals
02

Policy Layer

  • Intent recognition
  • Safety checks
  • Semantic cache
  • Cost constraints
03

Model Routing

  • Capability match
  • Latency / quality
  • Context length
  • Fallback
04

Inference Backends

  • Self-hosted models
  • Vendor providers
  • Streaming
  • KV-cache affinity
05

Observability

  • Metrics and traces
  • Usage correction
  • Errors and cost
  • Feedback loop
Open Source

Working in the open on semantic routing.

vllm-project/semantic-router

Intelligent semantic routing project in the official vLLM ecosystem for Mixture-of-Models, AI Gateway, and production LLM traffic governance.

Core Committer
Agentic Intelligence Lab

Research and engineering community around agentic systems, infrastructure primitives, and collective intelligence.

Member
Semantic Router

Community space for semantic routing, evaluation, model selection, and deployment-oriented routing systems.

Member
Experience

From cloud-native data planes to MaaS platforms.

TokenHub / MaaS Platform Development Engineer

Mar 2026 - Present

Tencent Cloud

Building unified LLM API Gateway and MaaS platform capabilities across model access, request governance, capacity protection, provider abstraction, fallback, quota, and observability.

Container Networking R&D Engineer

Jul 2024 - Mar 2026

Tencent Cloud

Built and maintained VPC CNI, Pod networking, Kubernetes Service, Ingress, and Gateway capabilities across production cloud-native traffic paths.

Current Threads
Resume Work experience, open-source identity, and core skills. Projects Semantic routing, LLM Gateway research, and MaaS infrastructure notes. Writing A longer background note from cloud-native networking to intelligent routing.