feat(knowledge): rebuild the project notes panel as Project knowledge (#2973)

The panel stored notes, todos and plans inside one shared JSON file that
six unrelated domains also wrote to, synchronised itself through window
CustomEvents, and could only read plans. It is now Project knowledge:
server-owned storage with explicit routes, a store with rollback, a
section sidebar, plans that open and edit in place, and search across
all of it.

Notes and plans the user pins travel with every message sent in that
project. Pinning is project state, not an attachment to one message, so
it holds until unpinned and the work status panel names what is riding
along and can detach it.

Agent memory is added alongside, in two scopes: what is true about the
user, and what is true about this codebase. The split is not cosmetic —
a wrong project fact costs one project and is noticed, while a wrong
global fact quietly shapes every session everywhere and the user has no
code to check it against. It stays separate from notes so an agent
mistake cannot land in what the user wrote. Sessions receive an index of
titles only; bodies are read on demand, because an index carrying full
text grows until it crowds out the conversation.

Deciding what a session must be told, and whether it has been told, now
lives on the server. The client owned it before, which meant sessions
started without a UI — scheduled tasks, sessions the agent dispatches —
received nothing at all, and a tab's record of what it had sent outlived
the conversation: after compaction the agent no longer held the block
while the tab went on believing it did. What was delivered is recorded
in the session's own metadata, and compaction restores it through the
runtime that already restores pinned messages, in the same turn.

Agent memory ships dark behind OPENCHAMBER_MEMORY_ENABLE: unset, there
is no tool, no routes, no session index, no settings row and no panel
tab. Absent rather than switched off, so nothing invites turning on a
feature that has not been announced. Pinned notes and plans are
unaffected and ship as normal.
This commit is contained in:
Bohdan Triapitsyn
2026-08-18 02:59:04 +03:00
committed by GitHub
parent 7611076436
commit 34e8a24b20
102 changed files with 10640 additions and 1630 deletions
@@ -0,0 +1,82 @@
# Session Knowledge
What a session must be told about the project — the user's pinned notes and
plans, and the index of what the agent has remembered — and whether it has been
told yet.
## Why it is here and not in the UI
The client used to own this: it assembled the text, decided when to send it, and
remembered what it had sent in a module-scoped map. Two consequences followed.
A session started without a UI got nothing at all. Scheduled tasks and sessions
the agent dispatches build their prompts on the server and never touch the
browser, so pinned context and the memory index simply did not exist for them.
And a tab's memory of what it sent survives compaction, while the conversation
does not. After a summary the agent no longer holds the block, but the tab goes
on believing it does and never sends it again.
## The contract
`session.metadata.openchamber.knowledge_context_delivered` holds the signature
of what the session is carrying. It lives with the session, so it survives the
tab closing and is visible to every sender, including the ones with no tab.
The signature covers content revisions, not just identity: editing a pinned note
must re-send it, not merely renaming one.
## Three moments, two deliveries
| Moment | Delivery |
|---|---|
| A message from the UI | synthetic part on that message |
| A scheduled task, a session the agent dispatched | synthetic part on that prompt |
| After compaction | its own `prompt_async`, alongside the pinned messages |
The first two attach to an outgoing message because there is one. Compaction has
none, which is why it re-sends on its own — and it travels with
`context-obligatory`'s pinned messages in a single turn, since two synthetic
messages back to back read as the agent being interrupted twice.
## Failure behaviour
Nothing here may fail a send. A message without its background costs the agent
some context; a failed send costs the user their message. Every caller treats an
error as "no block this time".
A source that will not load never blanks the rest: an unreadable memory store
still delivers the pinned notes. A memory scope that failed to load is left out
rather than indexed as empty, which would teach the agent to store again what it
already has.
Delivery is recorded only after the send is accepted. Recording it when the text
is handed over would leave a failed send believing the agent had context it never
received.
## Entries that read as instructions
Memory is the one place where text from outside can settle permanently. The
agent reads a page, decides a line is worth keeping, saves it — and from then on
it rides into every session in every project. An injection anywhere else lives
for one conversation.
`agent-memory/threat-patterns` scans on write and again on every read, so an
entry written before a pattern existed, or edited on disk since, is judged now.
A match never deletes: the entry is stored, flagged, kept out of what sessions
are told, and shown in the panel with a warning. Silently dropping it would hide
the attempt from the only person able to judge it.
Patterns, not a model — this runs on every index build. That buys the blunt
cases only, which is the honest expectation.
## Shipping dark
Agent memory is complete but unreleased. `OPENCHAMBER_MEMORY_ENABLE` decides
whether it exists in a given process at all: unset, there is no tool, no routes,
no session index, no settings row and no panel tab — absent rather than switched
off, which would invite turning on something never announced. The setting itself
also defaults to off, so setting the variable does not enable memory by itself.
Pinned notes and plans are unaffected: they ship as normal and travel with every
message whether or not memory exists.
@@ -0,0 +1,87 @@
/**
* What a session still owes in project knowledge, and the record that it was
* delivered.
*
* Two calls rather than one, because only the sender knows whether the message
* carrying the block actually went out. Handing over the text and recording it
* as delivered in the same request would leave a failed send believing the
* agent has context it never received.
*
* The body parser is attached per route: there is no global one, because the
* generic OpenCode proxy needs an unread request stream.
*/
import express from 'express';
const parseJsonBody = express.json({ limit: '1mb' });
const isRecord = (value) => Boolean(value) && typeof value === 'object' && !Array.isArray(value);
const asNonEmptyString = (value) => (
typeof value === 'string' && value.trim().length > 0 ? value.trim() : ''
);
export const registerSessionKnowledgeRoutes = (app, dependencies) => {
const { sessionKnowledgeRuntime } = dependencies;
/**
* Answers with the text to attach and the signature to report back once it
* has gone. An empty text means the session is already carrying it.
*/
app.get('/api/session-knowledge', async (req, res) => {
const directory = asNonEmptyString(req.query.directory);
const sessionId = asNonEmptyString(req.query.sessionId);
if (!directory) {
return res.status(400).json({ error: 'directory is required' });
}
try {
const pending = sessionId
? await sessionKnowledgeRuntime.resolvePendingForSession(sessionId, directory)
// A session that does not exist yet — a draft about to be created —
// has been told nothing, so everything is still owed.
: await sessionKnowledgeRuntime.resolvePending(directory, '');
return res.json(pending);
} catch (error) {
// Never fails the caller's send: a message without its background is far
// better than no message at all.
return res.json({ text: '', signature: '', unavailable: true, reason: error?.message ?? 'unknown' });
}
});
/** Counts and names for the work status panel; assembles no text. */
app.get('/api/session-knowledge/summary', async (req, res) => {
const directory = asNonEmptyString(req.query.directory);
if (!directory) {
return res.json({ notes: [], plans: [], memory: { global: 0, project: 0 } });
}
try {
return res.json(await sessionKnowledgeRuntime.collectSummary(directory));
} catch {
// A panel that cannot read this shows nothing rather than an error.
return res.json({ notes: [], plans: [], memory: { global: 0, project: 0 } });
}
});
app.post('/api/session-knowledge/delivered', parseJsonBody, async (req, res) => {
const body = req.body;
if (!isRecord(body)) {
return res.status(400).json({ error: 'Body must be an object' });
}
const sessionId = asNonEmptyString(body.sessionId);
const directory = asNonEmptyString(body.directory);
const signature = asNonEmptyString(body.signature);
if (!sessionId || !directory || !signature) {
return res.status(400).json({ error: 'sessionId, directory and signature are required' });
}
try {
await sessionKnowledgeRuntime.recordDelivered(sessionId, directory, signature);
return res.json({ recorded: true });
} catch (error) {
// The message is already sent; failing here only means the block may be
// sent once more, which is far better than reporting the send as failed.
return res.json({ recorded: false, reason: error?.message ?? 'unknown' });
}
});
};
@@ -0,0 +1,281 @@
/**
* What a session must be told about the project's knowledge, and whether it has
* been told yet.
*
* One owner, three moments. The block is attached to an outgoing prompt when
* there is one (a message from the UI, a scheduled task, a session the agent
* dispatched) and re-sent on its own after compaction, when there is no message
* to attach it to. The decision is the same in every case, so it lives here
* rather than in each sender — the client used to own it, which meant sessions
* started without a UI got nothing at all.
*
* What was delivered is recorded in the session's own metadata rather than in
* the browser. A signature held in a tab is lost when the tab closes, and worse,
* it survives compaction: the tab goes on believing the agent still has context
* that has just been summarised away.
*/
const KNOWLEDGE_METADATA_KEY = 'knowledge_context_delivered';
/** Total budget for the assembled block; anything past it is cut, loudly. */
const KNOWLEDGE_MAX_LENGTH = 8000;
const isRecord = (value) => Boolean(value && typeof value === 'object' && !Array.isArray(value));
const truncate = (value, budget) => (
value.length <= budget ? value : `${value.slice(0, Math.max(0, budget - 1))}`
);
/**
* Identity of everything the session should be carrying, content revisions
* included: editing a pinned note must re-send it, not merely renaming one.
*/
export const buildKnowledgeSignature = ({ notes, plans, memory }) => {
const parts = [
...notes.map((note) => `n:${note.id}:${note.updatedAt}`),
...plans.map((plan) => `p:${plan.id}:${plan.title}`),
...memory.global.map((entry) => `mg:${entry.id}:${entry.updatedAt}`),
...memory.project.map((entry) => `mp:${entry.id}:${entry.updatedAt}`),
];
return parts.length === 0 ? '' : parts.sort().join('|');
};
const renderMemorySection = (entries) => entries
.slice()
.sort((a, b) => a.createdAt - b.createdAt)
.map((entry) => `- [${entry.type}] ${entry.title}`)
.join('\n');
/**
* Titles only for memory, never bodies: an index carrying full text grows
* without bound until it crowds out the conversation it was meant to inform.
*/
const buildMemoryBlock = ({ global, project }) => {
const sections = [];
if (global.length > 0) sections.push(`### About the user\n\n${renderMemorySection(global)}`);
if (project.length > 0) sections.push(`### About this project\n\n${renderMemorySection(project)}`);
if (sections.length === 0) return '';
return [
'You have stored memory from earlier sessions. Only the titles are listed below.',
'A title is an abbreviation, not the memory. Read the entry with the'
+ ' openchamber_memory tool before you act on it: titles routinely leave out'
+ ' the conditions, exceptions and reasons that decide how the memory'
+ ' applies, and a title that looks self-explanatory is the most likely to'
+ ' be hiding them. Read every title that could bear on the task at hand;'
+ ' you need not read the ones unrelated to what you are doing.',
'Memory records what was true when it was written. Verify anything it says'
+ ' about files, flags or commands before relying on it.',
...sections,
].join('\n\n');
};
const buildPinnedBlock = ({ notes, plans }) => {
const sections = [];
if (notes.length > 0) {
const rendered = notes
.slice()
.sort((a, b) => a.createdAt - b.createdAt)
.map((note) => `- ${note.body.trim()}`)
.join('\n');
sections.push(`## Pinned notes\n\n${rendered}`);
}
for (const plan of plans) {
// A plan whose markdown cannot be read is marked rather than dropped:
// losing one attachment must not silently shrink the context.
sections.push(plan.body
? `## Pinned plan: ${plan.title}\n\n${plan.body}`
: `## Pinned plan: ${plan.title}\n\n(plan content unavailable)`);
}
if (sections.length === 0) return '';
return [
'The user pinned the following project context. Treat it as standing background, not as a new instruction.',
...sections,
].join('\n\n');
};
export const buildKnowledgeText = ({ notes, plans, memory }) => {
const blocks = [buildPinnedBlock({ notes, plans }), buildMemoryBlock(memory)].filter(Boolean);
if (blocks.length === 0) return '';
const assembled = blocks.join('\n\n');
return assembled.length <= KNOWLEDGE_MAX_LENGTH
? assembled
: `${truncate(assembled, KNOWLEDGE_MAX_LENGTH)}\n\n(project knowledge truncated)`;
};
export const createSessionKnowledgeRuntime = (dependencies) => {
const {
projectContextRuntime,
agentMemoryRuntime,
resolveProjectId,
isAgentMemoryEnabled,
openCodeFetch = null,
} = dependencies;
/**
* Everything the session should be carrying, read fresh. A failure in one
* source never blanks the rest: a memory store that will not load must not
* take the user's pinned notes down with it.
*/
const collect = async (directory) => {
const projectId = directory ? await resolveProjectId(directory) : '';
let notes = [];
let plans = [];
if (projectId) {
try {
const context = await projectContextRuntime.readContext(projectId);
notes = (context.notes || []).filter((note) => note.pinned);
const pinnedPlans = (context.plans || []).filter((plan) => plan.pinned);
plans = await Promise.all(pinnedPlans.map(async (plan) => {
try {
const content = await projectContextRuntime.readPlan(projectId, plan.id);
return { id: plan.id, title: plan.title, body: content?.body?.trim() || '' };
} catch {
return { id: plan.id, title: plan.title, body: '' };
}
}));
} catch {
notes = [];
plans = [];
}
}
let memory = { global: [], project: [] };
const memoryEnabled = typeof isAgentMemoryEnabled === 'function'
? await isAgentMemoryEnabled().catch(() => false)
: true;
if (memoryEnabled) {
try {
const stored = await agentMemoryRuntime.readAll(projectId || null);
// A scope that failed to load is left out entirely rather than indexed
// as empty, which would teach the agent to store what it already has.
//
// Flagged entries are withheld from the model but left in the store, so
// the user can see what was caught. Dropping them would hide the
// attempt from the only person able to judge it.
const visible = (entries) => entries.filter((entry) => !entry.flagged);
memory = {
global: stored.globalFailed ? [] : visible(stored.global),
project: stored.projectFailed ? [] : visible(stored.project),
};
} catch {
memory = { global: [], project: [] };
}
}
return { notes, plans, memory };
};
/**
* What the session is carrying, for display. Deliberately does not read plan
* bodies: the panel states counts and names, and reading every pinned plan
* off disk to show a number would make opening a panel cost what sending a
* message costs.
*/
const collectSummary = async (directory) => {
const projectId = directory ? await resolveProjectId(directory) : '';
const empty = { notes: [], plans: [], memory: { global: 0, project: 0 } };
if (!projectId) return empty;
let notes = [];
let plans = [];
try {
const context = await projectContextRuntime.readContext(projectId);
notes = (context.notes || []).filter((note) => note.pinned)
.map((note) => ({ id: note.id, body: note.body }));
plans = (context.plans || []).filter((plan) => plan.pinned)
.map((plan) => ({ id: plan.id, title: plan.title }));
} catch {
notes = [];
plans = [];
}
let memory = { global: 0, project: 0 };
const memoryEnabled = typeof isAgentMemoryEnabled === 'function'
? await isAgentMemoryEnabled().catch(() => false)
: true;
if (memoryEnabled) {
try {
const stored = await agentMemoryRuntime.readAll(projectId);
memory = {
global: stored.globalFailed ? 0 : stored.global.length,
project: stored.projectFailed ? 0 : stored.project.length,
};
} catch {
memory = { global: 0, project: 0 };
}
}
return { notes, plans, memory };
};
const readDeliveredSignature = (session) => {
const metadata = isRecord(session?.metadata) ? session.metadata : {};
const openchamber = isRecord(metadata.openchamber) ? metadata.openchamber : {};
const delivered = openchamber[KNOWLEDGE_METADATA_KEY];
return typeof delivered === 'string' ? delivered : '';
};
/**
* The text this session still owes, or an empty string when it is already
* carrying it. `deliveredSignature` comes from the session's metadata.
*/
const resolvePending = async (directory, deliveredSignature) => {
const collected = await collect(directory);
const signature = buildKnowledgeSignature(collected);
if (!signature || signature === deliveredSignature) {
return { text: '', signature };
}
return { text: buildKnowledgeText(collected), signature };
};
const readSession = async (sessionId, directory) => (
openCodeFetch(`/session/${encodeURIComponent(sessionId)}`, { directory })
);
/**
* What this session still owes, read from its own stored signature.
*/
const resolvePendingForSession = async (sessionId, directory) => {
const session = await readSession(sessionId, directory).catch(() => null);
return resolvePending(directory, readDeliveredSignature(session));
};
/**
* Recorded only once the message carrying it has actually gone out. Writing
* it when the text is handed over would leave a failed send believing the
* agent had context it never received.
*
* Merged onto a fresh read, because the session's metadata holds other
* OpenChamber state — pinned messages among it — and a blind write would
* drop whatever changed in between.
*/
const recordDelivered = async (sessionId, directory, signature) => {
const fresh = await readSession(sessionId, directory);
const metadata = isRecord(fresh?.metadata) ? fresh.metadata : {};
const openchamber = isRecord(metadata.openchamber) ? metadata.openchamber : {};
await openCodeFetch(`/session/${encodeURIComponent(sessionId)}`, {
directory,
method: 'PATCH',
body: {
metadata: {
...metadata,
openchamber: { ...openchamber, [KNOWLEDGE_METADATA_KEY]: signature },
},
},
});
};
return {
collect,
collectSummary,
resolvePending,
resolvePendingForSession,
recordDelivered,
readDeliveredSignature,
metadataKey: KNOWLEDGE_METADATA_KEY,
};
};
@@ -0,0 +1,240 @@
import { describe, expect, test } from 'bun:test';
import { buildKnowledgeSignature, buildKnowledgeText, createSessionKnowledgeRuntime } from './runtime.js';
const DIRECTORY = '/work/project';
const PROJECT_ID = 'path_project';
const note = (overrides = {}) => ({
id: 'n1', body: 'Pinned note body.', createdAt: 1, updatedAt: 1, pinned: true, source: 'manual', ...overrides,
});
const plan = (overrides = {}) => ({
id: 'p1', file: 'p1.md', title: 'Migration plan', createdAt: 1, pinned: true, ...overrides,
});
const memory = (overrides = {}) => ({
id: 'm1', title: 'Uses bun', body: 'Full text.', type: 'fact', createdAt: 1, updatedAt: 1, ...overrides,
});
const createRuntime = (overrides = {}) => createSessionKnowledgeRuntime({
resolveProjectId: async () => PROJECT_ID,
projectContextRuntime: {
readContext: async () => ({ notes: [note()], todos: [], plans: [plan()] }),
readPlan: async () => ({ body: 'Plan body.' }),
...overrides.projectContextRuntime,
},
agentMemoryRuntime: {
readAll: async () => ({ global: [memory()], project: [], globalFailed: false, projectFailed: false }),
...overrides.agentMemoryRuntime,
},
...('isAgentMemoryEnabled' in overrides ? { isAgentMemoryEnabled: overrides.isAgentMemoryEnabled } : {}),
});
describe('what the session is owed', () => {
test('carries pinned notes, pinned plan bodies, and the memory index', async () => {
const { text } = await createRuntime().resolvePending(DIRECTORY, '');
expect(text).toContain('Pinned note body.');
expect(text).toContain('Migration plan');
expect(text).toContain('Plan body.');
expect(text).toContain('Uses bun');
});
test('memory is indexed by title, never by body', async () => {
const { text } = await createRuntime().resolvePending(DIRECTORY, '');
expect(text).not.toContain('Full text.');
});
test('unpinned notes and plans stay out', async () => {
const runtime = createRuntime({
projectContextRuntime: {
readContext: async () => ({ notes: [note({ pinned: false })], todos: [], plans: [] }),
},
});
const { text } = await runtime.resolvePending(DIRECTORY, '');
expect(text).not.toContain('Pinned note body.');
});
test('nothing pinned and nothing remembered owes nothing', async () => {
const runtime = createRuntime({
projectContextRuntime: { readContext: async () => ({ notes: [], todos: [], plans: [] }) },
agentMemoryRuntime: {
readAll: async () => ({ global: [], project: [], globalFailed: false, projectFailed: false }),
},
});
const { text, signature } = await runtime.resolvePending(DIRECTORY, '');
expect(signature).toBe('');
expect(text).toBe('');
});
});
describe('what has already been delivered', () => {
test('owes nothing when the signature matches', async () => {
const runtime = createRuntime();
const first = await runtime.resolvePending(DIRECTORY, '');
const second = await runtime.resolvePending(DIRECTORY, first.signature);
expect(second.text).toBe('');
expect(second.signature).toBe(first.signature);
});
test('an edited note owes the block again', async () => {
const before = buildKnowledgeSignature({
notes: [note()], plans: [], memory: { global: [], project: [] },
});
const after = buildKnowledgeSignature({
notes: [note({ updatedAt: 2 })], plans: [], memory: { global: [], project: [] },
});
expect(after).not.toBe(before);
});
test('a memory saved mid-session owes the block again', async () => {
const before = buildKnowledgeSignature({
notes: [], plans: [], memory: { global: [memory()], project: [] },
});
const after = buildKnowledgeSignature({
notes: [], plans: [], memory: { global: [memory()], project: [memory({ id: 'm2' })] },
});
expect(after).not.toBe(before);
});
test('the same set in a different order is the same signature', () => {
const a = buildKnowledgeSignature({
notes: [note({ id: 'a' }), note({ id: 'b' })], plans: [], memory: { global: [], project: [] },
});
const b = buildKnowledgeSignature({
notes: [note({ id: 'b' }), note({ id: 'a' })], plans: [], memory: { global: [], project: [] },
});
expect(a).toBe(b);
});
});
describe('when a source will not load', () => {
test('a broken memory store still delivers the pinned notes', async () => {
const runtime = createRuntime({
agentMemoryRuntime: { readAll: async () => { throw new Error('unreadable'); } },
});
const { text } = await runtime.resolvePending(DIRECTORY, '');
expect(text).toContain('Pinned note body.');
});
test('a scope that failed to load is left out rather than indexed as empty', async () => {
const runtime = createRuntime({
agentMemoryRuntime: {
readAll: async () => ({ global: [memory()], project: [], globalFailed: true, projectFailed: false }),
},
});
const { text } = await runtime.resolvePending(DIRECTORY, '');
expect(text).not.toContain('Uses bun');
});
test('an unreadable plan is marked, not dropped', async () => {
const runtime = createRuntime({
projectContextRuntime: {
readContext: async () => ({ notes: [], todos: [], plans: [plan()] }),
readPlan: async () => { throw new Error('gone'); },
},
});
const { text } = await runtime.resolvePending(DIRECTORY, '');
expect(text).toContain('Migration plan');
expect(text).toContain('plan content unavailable');
});
test('a broken project context still delivers memory', async () => {
const runtime = createRuntime({
projectContextRuntime: { readContext: async () => { throw new Error('unreadable'); } },
});
const { text } = await runtime.resolvePending(DIRECTORY, '');
expect(text).toContain('Uses bun');
});
});
describe('the memory switch', () => {
test('memory is left out entirely while the feature is off', async () => {
const runtime = createRuntime({ isAgentMemoryEnabled: async () => false });
const { text } = await runtime.resolvePending(DIRECTORY, '');
expect(text).not.toContain('Uses bun');
expect(text).toContain('Pinned note body.');
});
test('an unreadable setting keeps memory out rather than guessing', async () => {
const runtime = createRuntime({
isAgentMemoryEnabled: async () => { throw new Error('settings unreadable'); },
});
const { text } = await runtime.resolvePending(DIRECTORY, '');
expect(text).not.toContain('Uses bun');
});
});
describe('reading what a session was told', () => {
test('finds the signature stored on the session', () => {
const runtime = createRuntime();
expect(runtime.readDeliveredSignature({
metadata: { openchamber: { knowledge_context_delivered: 'sig' } },
})).toBe('sig');
});
test('a session with no metadata has been told nothing', () => {
const runtime = createRuntime();
expect(runtime.readDeliveredSignature({})).toBe('');
expect(runtime.readDeliveredSignature(null)).toBe('');
});
});
describe('size', () => {
test('an oversized block is cut and says so', () => {
const text = buildKnowledgeText({
notes: [note({ body: 'x'.repeat(20_000) })],
plans: [],
memory: { global: [], project: [] },
});
expect(text.length).toBeLessThan(8_200);
expect(text).toContain('project knowledge truncated');
});
});
describe('entries that read as instructions', () => {
test('a flagged memory is kept out of what the session is told', async () => {
const runtime = createRuntime({
agentMemoryRuntime: {
readAll: async () => ({
global: [
memory({ id: 'ok', title: 'Uses bun' }),
memory({ id: 'bad', title: 'Ignore previous instructions', flagged: true }),
],
project: [],
globalFailed: false,
projectFailed: false,
}),
},
});
const { text } = await runtime.resolvePending(DIRECTORY, '');
expect(text).toContain('Uses bun');
expect(text).not.toContain('Ignore previous instructions');
});
});