fix(proxy): reuse upstream connections for OpenCode API requests (#2916)

* fix(proxy): reuse upstream connections for OpenCode API requests

`createProxyMiddleware` was constructed without an `agent`, so `http-proxy`
fell back to `agent: false`. That disables connection pooling and forces
`Connection: close` on every proxied request, consuming one ephemeral port
per request.

Measured against a real `opencode serve` instance, 200 sequential requests
through the proxy created 201 TIME_WAIT entries (1.005 ports/request). With
a keep-alive agent the same load creates 0.

On macOS the ephemeral range is 16,384 ports and TIME_WAIT lasts 30s, so
sustained traffic around 546 req/sec exhausts the pool — after which every
process on the host fails to open outbound connections with EADDRNOTAVAIL.

`maxSockets: Infinity` preserves the unbounded concurrency of `agent: false`,
so this changes connection reuse only, not request throughput.

Partially addresses #2915.

* fix(proxy): derive proxy agent class from the target scheme

Addresses review feedback on #2916. The first commit created an
unconditional `http.Agent`, which regresses external OpenCode servers
configured over https via `OPENCODE_HOST` (accepted by env-config.js).

http-proxy dispatches through `https.request` when the target protocol is
`https:` (http-proxy/lib/http-proxy/passes/web-incoming.js:126), and
`http.Agent#createConnection` is plain `net.createConnection` — so an
http.Agent would open a plaintext socket to a TLS port and fail every
proxied request. `agent: false` previously worked for both schemes.

`createOpenCodeProxyAgent(target)` now returns an `https.Agent` for https
targets and an `http.Agent` otherwise, derived once from
`resolveProxyTarget()` at registration so the single shared instance is
preserved across `apiProxy` and `interactiveOAuthProxy`.

Guarded in both test layers, verified to fail when the selection is
reverted to an unconditional http.Agent. `https.Agent` extends
`http.Agent`, so the http cases assert `not.toBeInstanceOf(https.Agent)`.

* Round 2: fix: resolve the proxy agent lazily so cold starts honor https

Addresses the round-2 blocker on #2916. Deriving the agent class at
registration is too early: startup-pipeline-runtime.js calls setupProxy()
(line 104) before bootstrapOpenCodeAtStartup() (line 141), so on a fresh
process state.openCodePort is null, buildOpenCodeUrl() throws
(network-runtime.js:86-88), and resolveProxyTarget() returns the http
loopback fallback. An external server configured via OPENCODE_HOST=https://
only appears on state.openCodeBaseUrl after bootstrap, so it was still
getting a plain http.Agent — the regression the previous commit intended
to fix.

`agent` is now a getter backed by a per-scheme memoizing resolver.
http-proxy-middleware rebuilds per-request options with
`Object.assign({}, this.proxyOptions)` in prepareProxyRequest, which invokes
getters, so resolution happens at request time while still yielding one
shared pool per scheme.

Tests now model the production ordering — registration while the port is
null and buildOpenCodeUrl throws, then an https base URL appearing after
bootstrap — and fail against the eager implementation. A behavioral test
pins the http-proxy-middleware option re-read the fix depends on, so a
library change that froze options would fail loudly instead of silently
regressing https targets.

The resolver is module-private; `bun run dead-code` flagged it as an
unused export when it was exported.

* Round 3: docs(changelog): note upstream connection reuse under [Unreleased]

Repo precedent adds [Unreleased] bullets for comparable proxy/stability
fixes (1.18.4 Stability, 1.9.3 Reliability/Proxy). Non-blocker raised in
review on #2916.

* Round 3: docs(changelog): use repo-standard 'behavior' spelling

* Round 4: docs(changelog): don't imply a restart is the only recovery

The ephemeral port pool drains on its own once the exhausting traffic
stops (TIME_WAIT expiry), so a restart is sufficient but not necessary.
Optional nit raised in review on #2916.

* Round 5: fix: construct the proxy agent through one factory; widen the pool

Review found the https branch was mutation-uncovered: the resolver
re-implemented agent construction inline instead of calling the exported
`createOpenCodeProxyAgent(target)`, so replacing its https branch with
`new https.Agent()` — dropping OPENCODE_AGENT_OPTIONS, and with it
keep-alive — left the entire suite green. Since `createOpenCodeProxyAgent`
also had no production callers, its four tests were pinning dead code.
Delegating collapses both: the factory is now the single construction
path, and the mutation fails 2 tests including the live resolver path.

Also from review:

- maxFreeSockets 32 -> 256 (Node's own default). The lower cap evicted
  pooled sockets under concurrency, reintroducing the churn this agent
  exists to prevent: at 64 concurrent requests it left 303 sockets in
  TIME_WAIT versus 0 at 256.
- Added `timeout` to OPENCODE_AGENT_OPTIONS. Free-socket eviction is
  governed by agent.options.timeout, which was unset, so idle sockets
  persisted until the peer closed them. `keepAliveMsecs` is the TCP probe
  delay, not the idle lifetime.
- resolveProxyTarget() now checks openCodePort before calling
  buildOpenCodeUrl instead of relying on it throwing. The port is nulled
  on several runtime paths (health-check failure, failed restart), so a
  degraded OpenCode made every proxied request pay for a thrown-and-caught
  exception — and the getter added a second call per request.
- Test fixtures use :4096 rather than :443; WHATWG URL elides the default
  port, so parseInt('') is NaN and env-config rejects that host. The
  fixtures modeled a state that cannot reach production.
- The getter-read assertion is now exact (0 at construction, 1, then 2)
  rather than >= 2, which would have passed if the getter were read twice
  at construction and never per-request.
- listen() rejects on 'error' and servers start inside try/finally, so a
  bind failure fails the test instead of hanging to timeout.
This commit is contained in:
Aaron Hogue
2026-08-17 23:44:38 +03:00
committed by GitHub
parent 344c1b3ce3
commit 7611076436
4 changed files with 427 additions and 7 deletions
@@ -0,0 +1,152 @@
import http from 'node:http';
import https from 'node:https';
import { beforeEach, describe, expect, it, vi } from 'vitest';
const { createProxyMiddlewareMock } = vi.hoisted(() => ({
createProxyMiddlewareMock: vi.fn(),
}));
vi.mock('http-proxy-middleware', () => ({
createProxyMiddleware: createProxyMiddlewareMock,
}));
const { registerOpenCodeProxy } = await import('./proxy.js');
const createStubApp = () => {
const settings = new Map();
const noop = () => {};
return {
get: (...args) => (args.length === 1 ? settings.get(args[0]) : undefined),
set: (key, value) => {
settings.set(key, value);
},
use: noop,
post: noop,
put: noop,
patch: noop,
delete: noop,
all: noop,
};
};
/**
* `state` is intentionally mutable so a test can model the production ordering:
* the proxy is registered before OpenCode bootstraps, so the port/base URL only
* become resolvable afterwards.
*/
const createStubDeps = (state) => ({
fs: { promises: { realpath: async (value) => value } },
os: {},
path: {},
OPEN_CODE_READY_GRACE_MS: 0,
LONG_REQUEST_TIMEOUT_MS: 1_000,
getRuntime: () => ({ openCodePort: state.port, openCodeBaseUrl: state.baseUrl }),
getOpenCodeAuthHeaders: () => ({}),
// Mirrors network-runtime.js: throws until the port is known.
buildOpenCodeUrl: (pathname) => {
if (!state.port) {
throw new Error('OpenCode port is not available');
}
return `${state.baseUrl}${pathname}`;
},
ensureOpenCodeApiPrefix: (pathname) => pathname,
});
const managedState = () => ({ port: 49303, baseUrl: 'http://127.0.0.1:49303' });
const coldState = () => ({ port: null, baseUrl: null });
const agentsFromCalls = () => createProxyMiddlewareMock.mock.calls.map(([options]) => options.agent);
describe('OpenCode API proxy agent wiring', () => {
beforeEach(() => {
createProxyMiddlewareMock.mockReset();
createProxyMiddlewareMock.mockImplementation(() => (_req, _res, next) => next?.());
});
it('constructs every proxy with a keep-alive agent', () => {
registerOpenCodeProxy(createStubApp(), createStubDeps(managedState()));
expect(createProxyMiddlewareMock).toHaveBeenCalled();
for (const agent of agentsFromCalls()) {
// Without an explicit agent, http-proxy falls back to `agent: false`,
// which forces `Connection: close` and burns one ephemeral port per
// request. See createOpenCodeProxyAgent in ./proxy.js.
expect(agent).toBeTruthy();
expect(agent.options?.keepAlive).toBe(true);
}
});
it('shares one agent instance across the API and OAuth proxies', () => {
registerOpenCodeProxy(createStubApp(), createStubDeps(managedState()));
const agents = agentsFromCalls();
expect(agents.length).toBeGreaterThan(1);
expect(agents.every(Boolean)).toBe(true);
expect(new Set(agents).size).toBe(1);
});
it('memoizes the agent per scheme rather than allocating one per resolution', () => {
registerOpenCodeProxy(createStubApp(), createStubDeps(managedState()));
const [options] = createProxyMiddlewareMock.mock.calls[0];
expect(options.agent).toBe(options.agent);
});
// Production ordering: startup-pipeline-runtime.js calls setupProxy() before
// bootstrapOpenCodeAtStartup(), so at registration the port is null,
// buildOpenCodeUrl throws, and resolveProxyTarget() falls back to the http
// loopback default. An external https server configured via OPENCODE_HOST is
// only visible after bootstrap, so the agent must be resolved lazily.
it('resolves an https agent after bootstrap even though registration ran cold', () => {
const state = coldState();
registerOpenCodeProxy(createStubApp(), createStubDeps(state));
// Cold: nothing resolvable yet, so the http fallback target applies.
for (const agent of agentsFromCalls()) {
expect(agent).not.toBeInstanceOf(https.Agent);
}
// Bootstrap completes against an external https server.
state.baseUrl = 'https://opencode.example.com:4096';
for (const agent of agentsFromCalls()) {
expect(agent).toBeInstanceOf(https.Agent);
// Asserted on the live resolver path, not just the exported factory:
// the https branch is the one a mutation could silently strip.
expect(agent.options?.keepAlive).toBe(true);
expect(agent.options?.maxFreeSockets).toBe(256);
}
});
it('keeps a plain http agent when bootstrap resolves an http target', () => {
const state = coldState();
registerOpenCodeProxy(createStubApp(), createStubDeps(state));
Object.assign(state, managedState());
for (const agent of agentsFromCalls()) {
// https.Agent extends http.Agent, so the negative assertion is load-bearing.
expect(agent).toBeInstanceOf(http.Agent);
expect(agent).not.toBeInstanceOf(https.Agent);
}
});
it('derives an https agent when the target is already https at registration', () => {
registerOpenCodeProxy(
createStubApp(),
createStubDeps({ port: 4096, baseUrl: 'https://opencode.example.com:4096' }),
);
const agents = agentsFromCalls();
expect(agents.length).toBeGreaterThan(0);
for (const agent of agents) {
expect(agent).toBeInstanceOf(https.Agent);
}
});
});
+116 -6
View File
@@ -1,3 +1,6 @@
import http from 'node:http';
import https from 'node:https';
import { createProxyMiddleware } from 'http-proxy-middleware';
import {
@@ -11,6 +14,96 @@ import { recordStartupPerformance } from './startup-performance.js';
const DEFAULT_SSE_HEARTBEAT_INTERVAL_MS = 20_000;
const OPENCODE_AGENT_KEEP_ALIVE_MS = 30_000;
// Node's own default. A lower cap evicts pooled sockets under concurrency,
// which reintroduces exactly the per-request connection churn this agent
// exists to prevent (measured: at 64 concurrent requests, a cap of 32 left
// 303 sockets in TIME_WAIT versus 0 at 256).
const OPENCODE_AGENT_MAX_FREE_SOCKETS = 256;
// Evicts idle free sockets from our side. Without it the only thing that
// retires an idle pooled socket is the upstream closing it. Note this is
// distinct from `keepAliveMsecs`, which is the TCP keep-alive probe delay.
const OPENCODE_AGENT_IDLE_TIMEOUT_MS = 60_000;
const OPENCODE_AGENT_OPTIONS = {
keepAlive: true,
keepAliveMsecs: OPENCODE_AGENT_KEEP_ALIVE_MS,
maxSockets: Infinity,
maxFreeSockets: OPENCODE_AGENT_MAX_FREE_SOCKETS,
timeout: OPENCODE_AGENT_IDLE_TIMEOUT_MS,
};
const isHttpsProxyTarget = (target) => {
if (typeof target !== 'string') {
return false;
}
try {
return new URL(target).protocol === 'https:';
} catch {
return /^https:/i.test(target.trim());
}
};
/**
* Agent for proxied OpenCode API requests.
*
* When no agent is supplied, `http-proxy` falls back to `agent: false`, which
* both disables connection pooling and forces `Connection: close` on every
* proxied request (http-proxy/lib/http-proxy/common.js). That consumes one
* ephemeral port per request, and sustained traffic can exhaust the host's
* ephemeral port range — after which every process on the machine fails to
* open outbound connections with EADDRNOTAVAIL.
*
* The agent must match the target scheme: http-proxy dispatches through
* `https.request` when `target.protocol === 'https:'`
* (http-proxy/lib/http-proxy/passes/web-incoming.js), and an `http.Agent`
* would open a plaintext socket to a TLS port. External servers may be
* configured over https via `OPENCODE_HOST` (see env-config.js), so derive the
* agent class from the resolved target.
*
* `maxSockets: Infinity` preserves the unbounded concurrency of `agent: false`,
* so this changes connection reuse only, not request throughput.
*/
export const createOpenCodeProxyAgent = (target) => (
isHttpsProxyTarget(target)
? new https.Agent(OPENCODE_AGENT_OPTIONS)
: new http.Agent(OPENCODE_AGENT_OPTIONS)
);
/**
* Lazily resolves the proxy agent, memoized per scheme.
*
* The scheme cannot be decided at registration time: `setupProxy()` runs before
* `bootstrapOpenCodeAtStartup()` (startup-pipeline-runtime.js), so on a cold
* start `state.openCodePort` is still null, `buildOpenCodeUrl()` throws
* (network-runtime.js) and `resolveProxyTarget()` falls back to the http
* loopback default. An external server configured over https via
* `OPENCODE_HOST` only becomes visible on `state.openCodeBaseUrl` after
* bootstrap completes.
*
* http-proxy-middleware rebuilds its per-request options with
* `Object.assign({}, this.proxyOptions)` inside `prepareProxyRequest`, which
* invokes getters, so exposing `agent` as a getter defers resolution to request
* time. Memoizing per scheme keeps a single shared pool per scheme rather than
* allocating an agent per request.
*/
const createOpenCodeProxyAgentResolver = (resolveTarget) => {
const agents = new Map();
return () => {
const target = resolveTarget();
const scheme = isHttpsProxyTarget(target) ? 'https:' : 'http:';
let agent = agents.get(scheme);
if (!agent) {
// Construct through the shared factory rather than inline, so both
// schemes are built from OPENCODE_AGENT_OPTIONS by the same code path.
agent = createOpenCodeProxyAgent(target);
agents.set(scheme, agent);
}
return agent;
};
};
export const createDirectoryQueryCanonicalizer = ({ realpath, ...cacheOptions } = {}) => {
const realpathCache = createRealpathCache({ fallbackOnError: true, realpath, ...cacheOptions });
@@ -285,15 +378,22 @@ export const registerOpenCodeProxy = (app, deps) => {
// and direct fetch helpers use. This avoids split-brain state where /health
// succeeds against an external host but /api/* still proxies to 127.0.0.1.
const resolveProxyTarget = () => {
try {
const resolved = normalizeProxyTarget(buildOpenCodeUrl('/', ''));
if (resolved) {
return resolved;
const runtimeState = getRuntime();
// `buildOpenCodeUrl` throws while the port is unknown, and the port is
// nulled on several runtime paths (health-check failure, failed restart),
// not just cold start. Checking first keeps a degraded OpenCode from
// making every proxied request pay for a thrown-and-caught exception.
if (runtimeState.openCodePort) {
try {
const resolved = normalizeProxyTarget(buildOpenCodeUrl('/', ''));
if (resolved) {
return resolved;
}
} catch {
}
} catch {
}
const runtimeState = getRuntime();
const externalBase = normalizeProxyTarget(runtimeState.openCodeBaseUrl);
if (externalBase) {
return externalBase;
@@ -767,8 +867,18 @@ export const registerOpenCodeProxy = (app, deps) => {
});
// Generic proxy for non-SSE OpenCode API routes.
// The agent is exposed as a getter so its class is resolved per request, not
// at registration: the proxy is registered before OpenCode bootstraps, so an
// https target configured via OPENCODE_HOST is not yet visible here. Agents
// are memoized per scheme, so this is still one shared pool per scheme across
// `apiProxy` and `interactiveOAuthProxy`.
const resolveOpenCodeProxyAgent = createOpenCodeProxyAgentResolver(resolveProxyTarget);
const createApiProxy = (timeoutMs) => createProxyMiddleware({
target: resolveProxyTarget(),
get agent() {
return resolveOpenCodeProxyAgent();
},
changeOrigin: true,
pathRewrite: { '^/api': '' },
timeout: timeoutMs,
+158 -1
View File
@@ -1,6 +1,14 @@
import http from 'node:http';
import https from 'node:https';
import { createProxyMiddleware } from 'http-proxy-middleware';
import { describe, expect, it } from 'vitest';
import { createDirectoryQueryCanonicalizer, normalizeForwardedDirectoryHeaders } from './proxy.js';
import {
createDirectoryQueryCanonicalizer,
createOpenCodeProxyAgent,
normalizeForwardedDirectoryHeaders,
} from './proxy.js';
describe('createDirectoryQueryCanonicalizer', () => {
it('canonicalizes directory query params and preserves other params', async () => {
@@ -93,3 +101,152 @@ describe('normalizeForwardedDirectoryHeaders', () => {
});
});
});
const listen = (server) => new Promise((resolve, reject) => {
server.once('error', reject);
server.listen(0, '127.0.0.1', () => {
server.removeListener('error', reject);
resolve(server.address().port);
});
});
const closeServer = (server) => new Promise((resolve) => {
server.close(resolve);
});
const request = (port, agent) => new Promise((resolve, reject) => {
const req = http.request({ host: '127.0.0.1', port, path: '/', method: 'GET', agent }, (res) => {
res.resume();
res.on('end', resolve);
res.on('error', reject);
});
req.on('error', reject);
req.end();
});
/**
* Proxies two sequential requests through `createProxyMiddleware` and reports
* what the upstream server observed for each one.
*/
const proxyTwoRequests = async (proxyAgent) => {
const seen = [];
let middleware;
const upstream = http.createServer((req, res) => {
seen.push({ connection: req.headers.connection, remotePort: req.socket.remotePort });
res.end('ok');
});
const front = http.createServer((req, res) => {
middleware(req, res, () => {
res.statusCode = 502;
res.end();
});
});
const clientAgent = new http.Agent({ keepAlive: true });
try {
const upstreamPort = await listen(upstream);
middleware = createProxyMiddleware({
target: `http://127.0.0.1:${upstreamPort}`,
...(proxyAgent ? { agent: proxyAgent } : {}),
});
const frontPort = await listen(front);
await request(frontPort, clientAgent);
await request(frontPort, clientAgent);
} finally {
clientAgent.destroy();
proxyAgent?.destroy();
await closeServer(front);
await closeServer(upstream);
}
return seen;
};
describe('createOpenCodeProxyAgent', () => {
it('reuses a single upstream socket across sequential proxied requests', async () => {
const seen = await proxyTwoRequests(createOpenCodeProxyAgent('http://127.0.0.1'));
expect(seen).toHaveLength(2);
expect(seen[0].connection).not.toBe('close');
expect(seen[1].remotePort).toBe(seen[0].remotePort);
});
it('without an agent, http-proxy forces Connection: close and a new socket per request', async () => {
const seen = await proxyTwoRequests(null);
expect(seen).toHaveLength(2);
expect(seen[0].connection).toBe('close');
expect(seen[1].remotePort).not.toBe(seen[0].remotePort);
});
// http-proxy dispatches through `https.request` when the target protocol is
// `https:`, so an http.Agent would open a plaintext socket to a TLS port.
// External OpenCode servers can be configured over https via OPENCODE_HOST.
it('returns an https agent for https targets', () => {
const agent = createOpenCodeProxyAgent('https://opencode.example.com:4096');
expect(agent).toBeInstanceOf(https.Agent);
expect(agent.options.keepAlive).toBe(true);
});
it('returns a plain http agent for http targets', () => {
const agent = createOpenCodeProxyAgent('http://127.0.0.1:4096');
// https.Agent extends http.Agent, so the negative assertion is the load-bearing one.
expect(agent).toBeInstanceOf(http.Agent);
expect(agent).not.toBeInstanceOf(https.Agent);
expect(agent.options.keepAlive).toBe(true);
});
it('falls back to an http agent for missing or unparseable targets', () => {
expect(createOpenCodeProxyAgent(undefined)).not.toBeInstanceOf(https.Agent);
expect(createOpenCodeProxyAgent('not a url')).not.toBeInstanceOf(https.Agent);
});
// The cold-start fix relies on http-proxy-middleware rebuilding its per-request
// options via `Object.assign({}, this.proxyOptions)` in prepareProxyRequest,
// which invokes getters. If that ever changes to a cached or shallow-reference
// copy, the agent would freeze at its registration-time value and https targets
// would silently regress — so pin the behavior here against the real library.
it('http-proxy-middleware re-reads the agent option on every proxied request', async () => {
let reads = 0;
let middleware;
const agent = createOpenCodeProxyAgent('http://127.0.0.1');
const upstream = http.createServer((_req, res) => res.end('ok'));
const front = http.createServer((req, res) => {
middleware(req, res, () => {
res.statusCode = 502;
res.end();
});
});
const clientAgent = new http.Agent({ keepAlive: true });
try {
const upstreamPort = await listen(upstream);
middleware = createProxyMiddleware({
target: `http://127.0.0.1:${upstreamPort}`,
get agent() {
reads += 1;
return agent;
},
});
// Construction itself must not read the getter — otherwise the assertion
// below could be satisfied without any per-request resolution happening.
expect(reads).toBe(0);
const frontPort = await listen(front);
await request(frontPort, clientAgent);
expect(reads).toBe(1);
await request(frontPort, clientAgent);
expect(reads).toBe(2);
} finally {
clientAgent.destroy();
agent.destroy();
await closeServer(front);
await closeServer(upstream);
}
});
});