fix(proxy): reuse upstream connections for OpenCode API requests (#2916)
* fix(proxy): reuse upstream connections for OpenCode API requests `createProxyMiddleware` was constructed without an `agent`, so `http-proxy` fell back to `agent: false`. That disables connection pooling and forces `Connection: close` on every proxied request, consuming one ephemeral port per request. Measured against a real `opencode serve` instance, 200 sequential requests through the proxy created 201 TIME_WAIT entries (1.005 ports/request). With a keep-alive agent the same load creates 0. On macOS the ephemeral range is 16,384 ports and TIME_WAIT lasts 30s, so sustained traffic around 546 req/sec exhausts the pool — after which every process on the host fails to open outbound connections with EADDRNOTAVAIL. `maxSockets: Infinity` preserves the unbounded concurrency of `agent: false`, so this changes connection reuse only, not request throughput. Partially addresses #2915. * fix(proxy): derive proxy agent class from the target scheme Addresses review feedback on #2916. The first commit created an unconditional `http.Agent`, which regresses external OpenCode servers configured over https via `OPENCODE_HOST` (accepted by env-config.js). http-proxy dispatches through `https.request` when the target protocol is `https:` (http-proxy/lib/http-proxy/passes/web-incoming.js:126), and `http.Agent#createConnection` is plain `net.createConnection` — so an http.Agent would open a plaintext socket to a TLS port and fail every proxied request. `agent: false` previously worked for both schemes. `createOpenCodeProxyAgent(target)` now returns an `https.Agent` for https targets and an `http.Agent` otherwise, derived once from `resolveProxyTarget()` at registration so the single shared instance is preserved across `apiProxy` and `interactiveOAuthProxy`. Guarded in both test layers, verified to fail when the selection is reverted to an unconditional http.Agent. `https.Agent` extends `http.Agent`, so the http cases assert `not.toBeInstanceOf(https.Agent)`. * Round 2: fix: resolve the proxy agent lazily so cold starts honor https Addresses the round-2 blocker on #2916. Deriving the agent class at registration is too early: startup-pipeline-runtime.js calls setupProxy() (line 104) before bootstrapOpenCodeAtStartup() (line 141), so on a fresh process state.openCodePort is null, buildOpenCodeUrl() throws (network-runtime.js:86-88), and resolveProxyTarget() returns the http loopback fallback. An external server configured via OPENCODE_HOST=https:// only appears on state.openCodeBaseUrl after bootstrap, so it was still getting a plain http.Agent — the regression the previous commit intended to fix. `agent` is now a getter backed by a per-scheme memoizing resolver. http-proxy-middleware rebuilds per-request options with `Object.assign({}, this.proxyOptions)` in prepareProxyRequest, which invokes getters, so resolution happens at request time while still yielding one shared pool per scheme. Tests now model the production ordering — registration while the port is null and buildOpenCodeUrl throws, then an https base URL appearing after bootstrap — and fail against the eager implementation. A behavioral test pins the http-proxy-middleware option re-read the fix depends on, so a library change that froze options would fail loudly instead of silently regressing https targets. The resolver is module-private; `bun run dead-code` flagged it as an unused export when it was exported. * Round 3: docs(changelog): note upstream connection reuse under [Unreleased] Repo precedent adds [Unreleased] bullets for comparable proxy/stability fixes (1.18.4 Stability, 1.9.3 Reliability/Proxy). Non-blocker raised in review on #2916. * Round 3: docs(changelog): use repo-standard 'behavior' spelling * Round 4: docs(changelog): don't imply a restart is the only recovery The ephemeral port pool drains on its own once the exhausting traffic stops (TIME_WAIT expiry), so a restart is sufficient but not necessary. Optional nit raised in review on #2916. * Round 5: fix: construct the proxy agent through one factory; widen the pool Review found the https branch was mutation-uncovered: the resolver re-implemented agent construction inline instead of calling the exported `createOpenCodeProxyAgent(target)`, so replacing its https branch with `new https.Agent()` — dropping OPENCODE_AGENT_OPTIONS, and with it keep-alive — left the entire suite green. Since `createOpenCodeProxyAgent` also had no production callers, its four tests were pinning dead code. Delegating collapses both: the factory is now the single construction path, and the mutation fails 2 tests including the live resolver path. Also from review: - maxFreeSockets 32 -> 256 (Node's own default). The lower cap evicted pooled sockets under concurrency, reintroducing the churn this agent exists to prevent: at 64 concurrent requests it left 303 sockets in TIME_WAIT versus 0 at 256. - Added `timeout` to OPENCODE_AGENT_OPTIONS. Free-socket eviction is governed by agent.options.timeout, which was unset, so idle sockets persisted until the peer closed them. `keepAliveMsecs` is the TCP probe delay, not the idle lifetime. - resolveProxyTarget() now checks openCodePort before calling buildOpenCodeUrl instead of relying on it throwing. The port is nulled on several runtime paths (health-check failure, failed restart), so a degraded OpenCode made every proxied request pay for a thrown-and-caught exception — and the getter added a second call per request. - Test fixtures use :4096 rather than :443; WHATWG URL elides the default port, so parseInt('') is NaN and env-config rejects that host. The fixtures modeled a state that cannot reach production. - The getter-read assertion is now exact (0 at construction, 1, then 2) rather than >= 2, which would have passed if the getter were read twice at construction and never per-request. - listen() rejects on 'error' and servers start inside try/finally, so a bind failure fails the test instead of hanging to timeout.
This commit is contained in:
Aaron Hogue
@@ -1,3 +1,6 @@
|
||||
import http from 'node:http';
|
||||
import https from 'node:https';
|
||||
|
||||
import { createProxyMiddleware } from 'http-proxy-middleware';
|
||||
|
||||
import {
|
||||
@@ -11,6 +14,96 @@ import { recordStartupPerformance } from './startup-performance.js';
|
||||
|
||||
const DEFAULT_SSE_HEARTBEAT_INTERVAL_MS = 20_000;
|
||||
|
||||
const OPENCODE_AGENT_KEEP_ALIVE_MS = 30_000;
|
||||
// Node's own default. A lower cap evicts pooled sockets under concurrency,
|
||||
// which reintroduces exactly the per-request connection churn this agent
|
||||
// exists to prevent (measured: at 64 concurrent requests, a cap of 32 left
|
||||
// 303 sockets in TIME_WAIT versus 0 at 256).
|
||||
const OPENCODE_AGENT_MAX_FREE_SOCKETS = 256;
|
||||
// Evicts idle free sockets from our side. Without it the only thing that
|
||||
// retires an idle pooled socket is the upstream closing it. Note this is
|
||||
// distinct from `keepAliveMsecs`, which is the TCP keep-alive probe delay.
|
||||
const OPENCODE_AGENT_IDLE_TIMEOUT_MS = 60_000;
|
||||
|
||||
const OPENCODE_AGENT_OPTIONS = {
|
||||
keepAlive: true,
|
||||
keepAliveMsecs: OPENCODE_AGENT_KEEP_ALIVE_MS,
|
||||
maxSockets: Infinity,
|
||||
maxFreeSockets: OPENCODE_AGENT_MAX_FREE_SOCKETS,
|
||||
timeout: OPENCODE_AGENT_IDLE_TIMEOUT_MS,
|
||||
};
|
||||
|
||||
const isHttpsProxyTarget = (target) => {
|
||||
if (typeof target !== 'string') {
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
return new URL(target).protocol === 'https:';
|
||||
} catch {
|
||||
return /^https:/i.test(target.trim());
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Agent for proxied OpenCode API requests.
|
||||
*
|
||||
* When no agent is supplied, `http-proxy` falls back to `agent: false`, which
|
||||
* both disables connection pooling and forces `Connection: close` on every
|
||||
* proxied request (http-proxy/lib/http-proxy/common.js). That consumes one
|
||||
* ephemeral port per request, and sustained traffic can exhaust the host's
|
||||
* ephemeral port range — after which every process on the machine fails to
|
||||
* open outbound connections with EADDRNOTAVAIL.
|
||||
*
|
||||
* The agent must match the target scheme: http-proxy dispatches through
|
||||
* `https.request` when `target.protocol === 'https:'`
|
||||
* (http-proxy/lib/http-proxy/passes/web-incoming.js), and an `http.Agent`
|
||||
* would open a plaintext socket to a TLS port. External servers may be
|
||||
* configured over https via `OPENCODE_HOST` (see env-config.js), so derive the
|
||||
* agent class from the resolved target.
|
||||
*
|
||||
* `maxSockets: Infinity` preserves the unbounded concurrency of `agent: false`,
|
||||
* so this changes connection reuse only, not request throughput.
|
||||
*/
|
||||
export const createOpenCodeProxyAgent = (target) => (
|
||||
isHttpsProxyTarget(target)
|
||||
? new https.Agent(OPENCODE_AGENT_OPTIONS)
|
||||
: new http.Agent(OPENCODE_AGENT_OPTIONS)
|
||||
);
|
||||
|
||||
/**
|
||||
* Lazily resolves the proxy agent, memoized per scheme.
|
||||
*
|
||||
* The scheme cannot be decided at registration time: `setupProxy()` runs before
|
||||
* `bootstrapOpenCodeAtStartup()` (startup-pipeline-runtime.js), so on a cold
|
||||
* start `state.openCodePort` is still null, `buildOpenCodeUrl()` throws
|
||||
* (network-runtime.js) and `resolveProxyTarget()` falls back to the http
|
||||
* loopback default. An external server configured over https via
|
||||
* `OPENCODE_HOST` only becomes visible on `state.openCodeBaseUrl` after
|
||||
* bootstrap completes.
|
||||
*
|
||||
* http-proxy-middleware rebuilds its per-request options with
|
||||
* `Object.assign({}, this.proxyOptions)` inside `prepareProxyRequest`, which
|
||||
* invokes getters, so exposing `agent` as a getter defers resolution to request
|
||||
* time. Memoizing per scheme keeps a single shared pool per scheme rather than
|
||||
* allocating an agent per request.
|
||||
*/
|
||||
const createOpenCodeProxyAgentResolver = (resolveTarget) => {
|
||||
const agents = new Map();
|
||||
|
||||
return () => {
|
||||
const target = resolveTarget();
|
||||
const scheme = isHttpsProxyTarget(target) ? 'https:' : 'http:';
|
||||
let agent = agents.get(scheme);
|
||||
if (!agent) {
|
||||
// Construct through the shared factory rather than inline, so both
|
||||
// schemes are built from OPENCODE_AGENT_OPTIONS by the same code path.
|
||||
agent = createOpenCodeProxyAgent(target);
|
||||
agents.set(scheme, agent);
|
||||
}
|
||||
return agent;
|
||||
};
|
||||
};
|
||||
|
||||
export const createDirectoryQueryCanonicalizer = ({ realpath, ...cacheOptions } = {}) => {
|
||||
const realpathCache = createRealpathCache({ fallbackOnError: true, realpath, ...cacheOptions });
|
||||
|
||||
@@ -285,15 +378,22 @@ export const registerOpenCodeProxy = (app, deps) => {
|
||||
// and direct fetch helpers use. This avoids split-brain state where /health
|
||||
// succeeds against an external host but /api/* still proxies to 127.0.0.1.
|
||||
const resolveProxyTarget = () => {
|
||||
try {
|
||||
const resolved = normalizeProxyTarget(buildOpenCodeUrl('/', ''));
|
||||
if (resolved) {
|
||||
return resolved;
|
||||
const runtimeState = getRuntime();
|
||||
|
||||
// `buildOpenCodeUrl` throws while the port is unknown, and the port is
|
||||
// nulled on several runtime paths (health-check failure, failed restart),
|
||||
// not just cold start. Checking first keeps a degraded OpenCode from
|
||||
// making every proxied request pay for a thrown-and-caught exception.
|
||||
if (runtimeState.openCodePort) {
|
||||
try {
|
||||
const resolved = normalizeProxyTarget(buildOpenCodeUrl('/', ''));
|
||||
if (resolved) {
|
||||
return resolved;
|
||||
}
|
||||
} catch {
|
||||
}
|
||||
} catch {
|
||||
}
|
||||
|
||||
const runtimeState = getRuntime();
|
||||
const externalBase = normalizeProxyTarget(runtimeState.openCodeBaseUrl);
|
||||
if (externalBase) {
|
||||
return externalBase;
|
||||
@@ -767,8 +867,18 @@ export const registerOpenCodeProxy = (app, deps) => {
|
||||
});
|
||||
|
||||
// Generic proxy for non-SSE OpenCode API routes.
|
||||
// The agent is exposed as a getter so its class is resolved per request, not
|
||||
// at registration: the proxy is registered before OpenCode bootstraps, so an
|
||||
// https target configured via OPENCODE_HOST is not yet visible here. Agents
|
||||
// are memoized per scheme, so this is still one shared pool per scheme across
|
||||
// `apiProxy` and `interactiveOAuthProxy`.
|
||||
const resolveOpenCodeProxyAgent = createOpenCodeProxyAgentResolver(resolveProxyTarget);
|
||||
|
||||
const createApiProxy = (timeoutMs) => createProxyMiddleware({
|
||||
target: resolveProxyTarget(),
|
||||
get agent() {
|
||||
return resolveOpenCodeProxyAgent();
|
||||
},
|
||||
changeOrigin: true,
|
||||
pathRewrite: { '^/api': '' },
|
||||
timeout: timeoutMs,
|
||||
|
||||
Reference in New Issue
Block a user