feat(browser): replace the preview proxy with a real browser panel and an agent web tool (#2883)
The preview panel worked by proxying a dev server through OpenChamber's own origin and rewriting the HTML that came back. Anything the rewriter did not anticipate broke, and pages that refuse to be embedded never loaded at all. This deletes the proxy (-1604 lines and its tests) and merges the preview and browser panels into one surface backed by a real Chromium view. What the panel is now - A `<webview>` in its own session partition: logins and cookies persist, hot reload works because nothing is rewritten, DevTools are one click away. - Annotation: pick one element, drag a region, or draw freehand, write a note, and it reaches chat with a screenshot of the visible page with the marks on it. - Toolbar: hard reload, page zoom, device sizes, a light/dark switch that applies to the page rather than the app, and cookie/cache clearing scoped to the panel alone. - Several pages at once, each tab showing the page's own favicon, and an address bar that suggests pages already visited in this project. - Dev servers are listed from what is actually listening on the machine, checked against what a project announced, so a server is offered no matter how it was started. One that is still starting is waited for instead of failing. Remote dev servers The desktop app binds a local port and pipes raw bytes to the OpenChamber host over the existing authenticated connection, so the page keeps its own origin at the root of its own host. The reachable set is exactly what discovery reports and is re-checked per connection, so an authenticated client cannot dial arbitrary local services on the host. Links and redirects to another loopback port stay on the machine that served the page. A tunnel that cannot be opened is reported; it is never replaced by the plain loopback URL, which would answer from the user's own machine under a remote address. Agent control Browser actions are a separate `openchamber_web` tool: open, snapshot, click, type, scroll, inspect computed styles, resize between mobile/tablet/desktop, and capture a screenshot into `.openchamber/screenshots/` in the project. The existing `openchamber` tool keeps sessions, worktrees and scheduled tasks. Each has its own setting in the new Settings -> General -> OpenChamber Tools section, and the plugin is not injected at all when both are off. Capability belongs to the connected client, not to configuration: a client declares on its event stream that it can drive a page, which only a Chromium host does. Exactly one client performs each request — it claims the request before acting, and the first claim wins — because deciding by whose result arrives first would be too late for a click that already happened. No client listening is answered immediately with an explanation rather than a timeout. Runtime boundaries Web tabs get a plain iframe that can display a page but not inspect one. The VS Code extension no longer offers the surface at all, since nothing that makes the panel worth having works there. Mobile is unaffected. Native boundary Camera, microphone, location and device-picker requests from panel pages are denied — Electron grants them by default when no handler is set, and the panel loads whatever address the user types. Page capture, appearance emulation and storage clearing verify that their target belongs to the panel's own session instead of trusting a web-contents id from the renderer. Persisted state Stored `preview` tabs migrate to `browser` (v13 -> v14). Context panel tab limits are now per surface, so filling one surface no longer evicts another's tabs. Address history is stored per project and per runtime. Documentation `preview.mdx` and `desktop-browser.mdx` rewritten across all locales, the agent tool settings path corrected, new `DOCUMENTATION.md` for the browser-control broker and the dev tunnel, and the `ui-api-decoupling` skill updated where it still described the deleted proxy.
This commit is contained in:
Bohdan Triapitsyn
committed by
GitHub
GitHub
parent
50613bb170
commit
a5aa32446d
@@ -0,0 +1,76 @@
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
|
||||
import {
|
||||
ANNOTATION_TEARDOWN_SCRIPT,
|
||||
buildAnnotationOverlayScript,
|
||||
type BrowserAnnotationOverlayLabels,
|
||||
type BrowserAnnotationOverlayTheme,
|
||||
} from './annotationOverlay';
|
||||
|
||||
const theme: BrowserAnnotationOverlayTheme = {
|
||||
colorScheme: 'dark',
|
||||
primary: 'rgb(214, 93, 42)',
|
||||
primarySoft: 'rgba(214, 93, 42, 0.16)',
|
||||
primaryFaint: 'rgba(214, 93, 42, 0.1)',
|
||||
primaryContrast: 'rgb(255, 255, 255)',
|
||||
surface: 'rgb(10, 10, 10)',
|
||||
surfaceElevated: 'rgb(20, 20, 20)',
|
||||
glassSurface: 'rgba(20, 20, 20, 0.64)',
|
||||
glassFilter: 'blur(26px) saturate(1.16)',
|
||||
border: 'rgb(40, 40, 40)',
|
||||
text: 'rgb(240, 240, 240)',
|
||||
mutedText: 'rgb(160, 160, 160)',
|
||||
};
|
||||
|
||||
const labels: BrowserAnnotationOverlayLabels = {
|
||||
select: 'Element',
|
||||
marquee: 'Region',
|
||||
draw: 'Draw',
|
||||
commentPlaceholder: 'Describe the change...',
|
||||
submit: 'Attach',
|
||||
};
|
||||
|
||||
/**
|
||||
* The overlay ships as source text evaluated inside another page, so ordinary
|
||||
* type-checking never sees it. These are the failures that produced: a stray
|
||||
* backtick silently truncated the whole script, and a value interpolated
|
||||
* without escaping would end it early or run as code.
|
||||
*/
|
||||
const parses = (source: string): boolean => {
|
||||
try {
|
||||
new Function(source);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
describe('annotation overlay script', () => {
|
||||
const script = buildAnnotationOverlayScript(theme, labels);
|
||||
|
||||
test('parses as JavaScript', () => {
|
||||
expect(parses(`return ${script}`)).toBe(true);
|
||||
});
|
||||
|
||||
test('contains no backtick, which would terminate the template it lives in', () => {
|
||||
expect(script).not.toContain('`');
|
||||
});
|
||||
|
||||
test('carries the theme and labels through as data, not as concatenated code', () => {
|
||||
expect(script).toContain(JSON.stringify(theme.primarySoft));
|
||||
expect(script).toContain(JSON.stringify(labels.commentPlaceholder));
|
||||
});
|
||||
|
||||
test('escapes a label that would otherwise close the script', () => {
|
||||
const hostile = buildAnnotationOverlayScript(theme, {
|
||||
...labels,
|
||||
submit: '"); alert(1); ("',
|
||||
});
|
||||
expect(parses(`return ${hostile}`)).toBe(true);
|
||||
expect(hostile).not.toContain('alert(1); ("');
|
||||
});
|
||||
|
||||
test('the teardown script parses on its own', () => {
|
||||
expect(parses(ANNOTATION_TEARDOWN_SCRIPT)).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,610 @@
|
||||
/**
|
||||
* The annotation overlay that runs *inside* the previewed page.
|
||||
*
|
||||
* This module produces a self-contained script string. It is injected through
|
||||
* `webview.executeJavaScript`, so it cannot import anything, cannot reference
|
||||
* our theme variables (the page has its own `:root`), and must not disturb the
|
||||
* document it lands in. Consequences that drive the implementation:
|
||||
*
|
||||
* - All chrome lives in a **closed** shadow root on a single host element, so
|
||||
* page CSS cannot restyle it and page scripts cannot walk into it.
|
||||
* - Every color and every label is passed in from the host, already resolved
|
||||
* and already translated. Nothing user-facing is hardcoded here.
|
||||
*
|
||||
* The chrome is two small pieces rather than one bar: the tools sit at the top
|
||||
* of the page, and the comment box follows whatever was just marked. A single
|
||||
* bar pinned to one edge covers the part of the page people most often want to
|
||||
* point at, and accumulates into something that feels like an application.
|
||||
*
|
||||
* Lifecycle: the script resolves with a payload (or null when cancelled) and
|
||||
* tears down its own chrome *before* resolving, waiting for a repaint, so a
|
||||
* screenshot taken by the host contains the page and none of our UI.
|
||||
*/
|
||||
|
||||
export type BrowserAnnotationOverlayTheme = {
|
||||
readonly colorScheme: 'light' | 'dark';
|
||||
readonly primary: string;
|
||||
/** Translucent primary, already resolved to a concrete color by the host. */
|
||||
readonly primarySoft: string;
|
||||
/** Faint primary used for hover backgrounds. */
|
||||
readonly primaryFaint: string;
|
||||
readonly primaryContrast: string;
|
||||
readonly surface: string;
|
||||
readonly surfaceElevated: string;
|
||||
/** Translucent elevated surface, matching the app's floating panels. */
|
||||
readonly glassSurface: string;
|
||||
/** A ready `backdrop-filter` value; '' when the theme has no glass. */
|
||||
readonly glassFilter: string;
|
||||
readonly border: string;
|
||||
readonly text: string;
|
||||
readonly mutedText: string;
|
||||
};
|
||||
|
||||
export type BrowserAnnotationOverlayLabels = {
|
||||
readonly select: string;
|
||||
readonly marquee: string;
|
||||
readonly draw: string;
|
||||
readonly commentPlaceholder: string;
|
||||
readonly submit: string;
|
||||
};
|
||||
|
||||
const ANNOTATION_ACTIVE_FLAG = '__openchamberBrowserAnnotationActive';
|
||||
|
||||
/** Cancels an overlay left behind by an earlier session. Safe when none is active. */
|
||||
export const ANNOTATION_TEARDOWN_SCRIPT = `(() => {
|
||||
try {
|
||||
var host = document.querySelector('[data-openchamber-annotation]');
|
||||
if (host && host.parentNode) host.parentNode.removeChild(host);
|
||||
} catch (error) { /* page navigated away */ }
|
||||
try {
|
||||
var cursor = document.getElementById('openchamber-annotation-cursor');
|
||||
if (cursor && cursor.parentNode) cursor.parentNode.removeChild(cursor);
|
||||
} catch (error) { /* page navigated away */ }
|
||||
try { delete window['${ANNOTATION_ACTIVE_FLAG}']; } catch (error) { /* non-configurable */ }
|
||||
})();`;
|
||||
|
||||
export const buildAnnotationOverlayScript = (
|
||||
theme: BrowserAnnotationOverlayTheme,
|
||||
labels: BrowserAnnotationOverlayLabels,
|
||||
): string => {
|
||||
const config = JSON.stringify({ theme, labels });
|
||||
return String.raw`new Promise((resolve) => {
|
||||
var CONFIG = ${config};
|
||||
var THEME = CONFIG.theme;
|
||||
var LABELS = CONFIG.labels;
|
||||
var OVERLAY_ATTR = 'data-openchamber-annotation';
|
||||
var Z_OVERLAY = 2147483646;
|
||||
var MAX_TEXT = 400;
|
||||
var MIN_RECT = 3;
|
||||
var EDITOR_GAP = 10;
|
||||
|
||||
try {
|
||||
var stale = document.querySelector('[' + OVERLAY_ATTR + ']');
|
||||
if (stale && stale.parentNode) stale.parentNode.removeChild(stale);
|
||||
} catch (error) { /* nothing to clean */ }
|
||||
|
||||
window['${ANNOTATION_ACTIVE_FLAG}'] = true;
|
||||
|
||||
var counter = 0;
|
||||
var nextId = function (prefix) { counter += 1; return prefix + '-' + counter; };
|
||||
|
||||
var selected = null;
|
||||
var regions = [];
|
||||
var strokes = [];
|
||||
var tool = 'select';
|
||||
var settled = false;
|
||||
|
||||
// ---------------------------------------------------------------- geometry
|
||||
|
||||
var rectFrom = function (domRect) {
|
||||
return { x: domRect.left, y: domRect.top, width: domRect.width, height: domRect.height };
|
||||
};
|
||||
var normalizeRect = function (ax, ay, bx, by) {
|
||||
return { x: Math.min(ax, bx), y: Math.min(ay, by), width: Math.abs(bx - ax), height: Math.abs(by - ay) };
|
||||
};
|
||||
var usableRect = function (rect) { return rect.width >= MIN_RECT && rect.height >= MIN_RECT; };
|
||||
|
||||
// ------------------------------------------------------------- description
|
||||
|
||||
var isOverlayNode = function (element) {
|
||||
var node = element;
|
||||
while (node) {
|
||||
if (node.nodeType === 1 && node.hasAttribute && node.hasAttribute(OVERLAY_ATTR)) return true;
|
||||
node = node.parentNode || (node.host || null);
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
var elementFromPoint = function (x, y) {
|
||||
var found = document.elementFromPoint(x, y);
|
||||
if (!found || isOverlayNode(found)) return null;
|
||||
return found;
|
||||
};
|
||||
|
||||
var selectorPart = function (element) {
|
||||
var part = element.tagName.toLowerCase();
|
||||
if (element.id) return part + '#' + element.id;
|
||||
var className = typeof element.className === 'string' ? element.className.trim() : '';
|
||||
if (className) {
|
||||
var first = className.split(/\s+/).filter(Boolean).slice(0, 2).join('.');
|
||||
if (first) part += '.' + first;
|
||||
}
|
||||
return part;
|
||||
};
|
||||
|
||||
var buildSelector = function (element) {
|
||||
if (element.id) return '#' + element.id;
|
||||
var parts = [];
|
||||
var node = element;
|
||||
var depth = 0;
|
||||
while (node && node.nodeType === 1 && depth < 5) {
|
||||
var part = selectorPart(node);
|
||||
var parent = node.parentElement;
|
||||
if (parent) {
|
||||
var siblings = Array.prototype.filter.call(parent.children, function (child) {
|
||||
return child.tagName === node.tagName;
|
||||
});
|
||||
if (siblings.length > 1) part += ':nth-of-type(' + (siblings.indexOf(node) + 1) + ')';
|
||||
}
|
||||
parts.unshift(part);
|
||||
if (node.id) break;
|
||||
node = parent;
|
||||
depth += 1;
|
||||
}
|
||||
return parts.join(' > ');
|
||||
};
|
||||
|
||||
var INTERESTING_ATTRS = ['id', 'class', 'name', 'type', 'href', 'src', 'alt', 'title', 'role', 'placeholder', 'aria-label', 'data-testid'];
|
||||
var STYLE_PROPS = [
|
||||
'display', 'position', 'color', 'backgroundColor', 'fontSize', 'fontWeight', 'fontFamily',
|
||||
'lineHeight', 'padding', 'margin', 'border', 'borderRadius', 'width', 'height', 'opacity',
|
||||
'zIndex', 'flexDirection', 'justifyContent', 'alignItems', 'gap', 'textAlign'
|
||||
];
|
||||
|
||||
var describe = function (element) {
|
||||
var computed = window.getComputedStyle(element);
|
||||
var attributes = {};
|
||||
for (var i = 0; i < INTERESTING_ATTRS.length; i += 1) {
|
||||
var name = INTERESTING_ATTRS[i];
|
||||
var value = element.getAttribute(name);
|
||||
if (value) attributes[name] = String(value).slice(0, 200);
|
||||
}
|
||||
var computedStyle = {};
|
||||
for (var j = 0; j < STYLE_PROPS.length; j += 1) {
|
||||
var prop = STYLE_PROPS[j];
|
||||
computedStyle[prop] = String(computed[prop] == null ? '' : computed[prop]);
|
||||
}
|
||||
var ancestry = [];
|
||||
var node = element.parentElement;
|
||||
var depth = 0;
|
||||
while (node && node.nodeType === 1 && depth < 4) {
|
||||
var entry = { tag: node.tagName.toLowerCase(), selectorPart: selectorPart(node) };
|
||||
if (node.id) entry.id = node.id;
|
||||
var cls = typeof node.className === 'string' ? node.className.trim() : '';
|
||||
if (cls) entry.className = cls.slice(0, 200);
|
||||
ancestry.unshift(entry);
|
||||
node = node.parentElement;
|
||||
depth += 1;
|
||||
}
|
||||
var box = element.getBoundingClientRect();
|
||||
var text = (element.textContent == null ? '' : String(element.textContent)).replace(/\s+/g, ' ').trim();
|
||||
return {
|
||||
tag: element.tagName.toLowerCase(),
|
||||
text: text.slice(0, MAX_TEXT),
|
||||
selector: buildSelector(element),
|
||||
path: ancestry.map(function (e) { return e.selectorPart; }).concat([selectorPart(element)]).join(' > '),
|
||||
bounds: rectFrom(box),
|
||||
center: { x: box.left + box.width / 2, y: box.top + box.height / 2 },
|
||||
attributes: attributes,
|
||||
computedStyle: computedStyle,
|
||||
ancestry: ancestry
|
||||
};
|
||||
};
|
||||
|
||||
// ------------------------------------------------------------------ chrome
|
||||
|
||||
var host = document.createElement('div');
|
||||
host.setAttribute(OVERLAY_ATTR, '');
|
||||
host.style.cssText = 'position:fixed;inset:0;z-index:' + Z_OVERLAY + ';pointer-events:none';
|
||||
var shadow = host.attachShadow({ mode: 'closed' });
|
||||
|
||||
var style = document.createElement('style');
|
||||
style.textContent = [
|
||||
':host{all:initial}',
|
||||
'*{box-sizing:border-box;font-family:ui-sans-serif,system-ui,-apple-system,"Segoe UI",sans-serif}',
|
||||
'.layer{position:fixed;inset:0;pointer-events:none}',
|
||||
'.box{position:fixed;left:0;top:0;display:none;pointer-events:none;border:1.5px solid ' + THEME.primary + ';background:' + THEME.primarySoft + ';border-radius:2px}',
|
||||
'.label{position:fixed;left:0;top:0;display:none;pointer-events:none;padding:1px 6px;border-radius:4px;background:' + THEME.primary + ';color:' + THEME.primaryContrast + ';font-size:11px;line-height:17px;white-space:nowrap;font-weight:600}',
|
||||
'.tools{position:fixed;top:14px;left:50%;transform:translateX(-50%);display:flex;gap:2px;padding:4px;border-radius:999px;border:1px solid ' + THEME.border + ';background:' + THEME.glassSurface + ';-webkit-backdrop-filter:' + THEME.glassFilter + ';backdrop-filter:' + THEME.glassFilter + ';box-shadow:0 6px 20px rgba(0,0,0,.24);pointer-events:auto}',
|
||||
'.tools button{appearance:none;border:none;background:transparent;color:' + THEME.mutedText + ';border-radius:999px;padding:5px 14px;font-size:12px;line-height:18px;font-weight:500;cursor:pointer;white-space:nowrap}',
|
||||
'.tools button:hover{background:' + THEME.primaryFaint + '}',
|
||||
'.tools button[aria-pressed="true"]{background:' + THEME.primarySoft + ';color:' + THEME.primary + ';font-weight:600}',
|
||||
'.editor{position:fixed;left:0;top:0;display:none;align-items:center;gap:8px;width:min(420px,calc(100vw - 24px));padding:6px;padding-left:16px;border-radius:22px;border:1px solid ' + THEME.border + ';background:' + THEME.glassSurface + ';-webkit-backdrop-filter:' + THEME.glassFilter + ';backdrop-filter:' + THEME.glassFilter + ';box-shadow:0 8px 28px rgba(0,0,0,.3);pointer-events:auto}',
|
||||
'.editor textarea{flex:1;min-width:0;resize:none;border:none;background:transparent;color:' + THEME.text + ';font-size:13px;line-height:20px;outline:none;padding:6px 0;min-height:32px;max-height:104px;display:block}',
|
||||
'.editor textarea::placeholder{color:' + THEME.mutedText + '}',
|
||||
'.editor button{appearance:none;border:none;background:' + THEME.primary + ';color:' + THEME.primaryContrast + ';border-radius:999px;padding:8px 18px;font-size:12px;line-height:18px;font-weight:600;cursor:pointer;white-space:nowrap}',
|
||||
'.editor button[disabled]{opacity:.5;cursor:default}'
|
||||
].join('');
|
||||
shadow.appendChild(style);
|
||||
|
||||
var cursorStyle = document.createElement('style');
|
||||
cursorStyle.id = 'openchamber-annotation-cursor';
|
||||
document.head.appendChild(cursorStyle);
|
||||
var setCursor = function (value) {
|
||||
cursorStyle.textContent = value ? '*{cursor:' + value + ' !important}' : '';
|
||||
};
|
||||
|
||||
var svgNS = 'http://www.w3.org/2000/svg';
|
||||
var svg = document.createElementNS(svgNS, 'svg');
|
||||
svg.setAttribute('class', 'layer');
|
||||
svg.style.overflow = 'visible';
|
||||
shadow.appendChild(svg);
|
||||
|
||||
var hoverBox = document.createElement('div');
|
||||
hoverBox.className = 'box';
|
||||
var hoverLabel = document.createElement('div');
|
||||
hoverLabel.className = 'label';
|
||||
var marqueeBox = document.createElement('div');
|
||||
marqueeBox.className = 'box';
|
||||
shadow.append(hoverBox, hoverLabel, marqueeBox);
|
||||
|
||||
var positionBox = function (node, rect) {
|
||||
node.style.display = 'block';
|
||||
node.style.transform = 'translate(' + rect.x + 'px,' + rect.y + 'px)';
|
||||
node.style.width = rect.width + 'px';
|
||||
node.style.height = rect.height + 'px';
|
||||
};
|
||||
|
||||
var tools = document.createElement('div');
|
||||
tools.className = 'tools';
|
||||
shadow.appendChild(tools);
|
||||
|
||||
var toolButtons = {};
|
||||
[['select', LABELS.select], ['marquee', LABELS.marquee], ['draw', LABELS.draw]].forEach(function (entry) {
|
||||
var button = document.createElement('button');
|
||||
button.type = 'button';
|
||||
button.textContent = entry[1];
|
||||
button.addEventListener('click', function () { setTool(entry[0]); });
|
||||
toolButtons[entry[0]] = button;
|
||||
tools.appendChild(button);
|
||||
});
|
||||
|
||||
var editor = document.createElement('div');
|
||||
editor.className = 'editor';
|
||||
shadow.appendChild(editor);
|
||||
|
||||
var comment = document.createElement('textarea');
|
||||
comment.rows = 1;
|
||||
comment.placeholder = LABELS.commentPlaceholder;
|
||||
var submit = document.createElement('button');
|
||||
submit.type = 'button';
|
||||
submit.textContent = LABELS.submit;
|
||||
editor.append(comment, submit);
|
||||
|
||||
/**
|
||||
* Grows the box with its content.
|
||||
*
|
||||
* A hidden element reports scrollHeight 0, so measuring one collapses the
|
||||
* field to nothing — which is what left a sliver of a click target and an
|
||||
* invisible caret. Measuring only while visible, and never below the CSS
|
||||
* min-height, keeps it usable.
|
||||
*/
|
||||
var resizeComment = function () {
|
||||
if (editor.style.display === 'none') return;
|
||||
comment.style.height = 'auto';
|
||||
comment.style.height = Math.max(32, Math.min(comment.scrollHeight, 104)) + 'px';
|
||||
};
|
||||
comment.addEventListener('input', resizeComment);
|
||||
|
||||
// -------------------------------------------------------------- selection
|
||||
|
||||
var lastTargetRect = null;
|
||||
|
||||
/** Places the comment box under whatever was just marked, kept on screen. */
|
||||
var positionEditor = function () {
|
||||
if (!lastTargetRect) {
|
||||
editor.style.display = 'none';
|
||||
return;
|
||||
}
|
||||
var wasHidden = editor.style.display === 'none';
|
||||
editor.style.display = 'flex';
|
||||
// Measured now that it is laid out, not while it was still hidden.
|
||||
if (wasHidden) resizeComment();
|
||||
var box = editor.getBoundingClientRect();
|
||||
var left = lastTargetRect.x + lastTargetRect.width / 2 - box.width / 2;
|
||||
var top = lastTargetRect.y + lastTargetRect.height + EDITOR_GAP;
|
||||
// Above the target when there is no room below, so the box is never left
|
||||
// half off the bottom of the page.
|
||||
if (top + box.height > window.innerHeight - 8) {
|
||||
top = Math.max(8, lastTargetRect.y - box.height - EDITOR_GAP);
|
||||
}
|
||||
left = Math.max(8, Math.min(left, window.innerWidth - box.width - 8));
|
||||
editor.style.transform = 'translate(' + Math.round(left) + 'px,' + Math.round(top) + 'px)';
|
||||
};
|
||||
|
||||
/** Focus after layout: focusing a zero-height field puts the caret nowhere. */
|
||||
var focusComment = function () {
|
||||
requestAnimationFrame(function () {
|
||||
try {
|
||||
comment.focus({ preventScroll: true });
|
||||
comment.setSelectionRange(comment.value.length, comment.value.length);
|
||||
} catch (error) { /* field went away */ }
|
||||
});
|
||||
};
|
||||
|
||||
var hasTargets = function () {
|
||||
return Boolean(selected) || regions.length > 0 || strokes.length > 0;
|
||||
};
|
||||
|
||||
var syncChrome = function () {
|
||||
submit.disabled = !hasTargets();
|
||||
if (!hasTargets()) {
|
||||
lastTargetRect = null;
|
||||
editor.style.display = 'none';
|
||||
return;
|
||||
}
|
||||
positionEditor();
|
||||
};
|
||||
|
||||
var syncSelectionVisuals = function () {
|
||||
if (!selected) return;
|
||||
var rect = rectFrom(selected.element.getBoundingClientRect());
|
||||
if (!usableRect(rect)) {
|
||||
selected.outline.style.display = 'none';
|
||||
selected.badge.style.display = 'none';
|
||||
return;
|
||||
}
|
||||
positionBox(selected.outline, rect);
|
||||
selected.badge.style.display = 'block';
|
||||
selected.badge.style.transform = 'translate(' + Math.max(2, rect.x) + 'px,' + Math.max(2, rect.y - 19) + 'px)';
|
||||
lastTargetRect = rect;
|
||||
};
|
||||
|
||||
var dropSelection = function () {
|
||||
if (!selected) return;
|
||||
selected.outline.remove();
|
||||
selected.badge.remove();
|
||||
selected = null;
|
||||
};
|
||||
|
||||
var selectElement = function (element) {
|
||||
var wasSelected = selected && selected.element === element;
|
||||
dropSelection();
|
||||
if (wasSelected) {
|
||||
// Clicking the chosen element again clears it, so a misclick costs one
|
||||
// click rather than starting over.
|
||||
lastTargetRect = null;
|
||||
syncChrome();
|
||||
return;
|
||||
}
|
||||
var outline = document.createElement('div');
|
||||
outline.className = 'box';
|
||||
var badge = document.createElement('div');
|
||||
badge.className = 'label';
|
||||
badge.textContent = selectorPart(element);
|
||||
shadow.append(outline, badge);
|
||||
selected = { id: nextId('element'), element: element, outline: outline, badge: badge };
|
||||
syncSelectionVisuals();
|
||||
syncChrome();
|
||||
focusComment();
|
||||
};
|
||||
|
||||
var setTool = function (next) {
|
||||
tool = next;
|
||||
hoverBox.style.display = 'none';
|
||||
hoverLabel.style.display = 'none';
|
||||
marqueeBox.style.display = 'none';
|
||||
setCursor(next === 'select' ? '' : 'crosshair');
|
||||
for (var key in toolButtons) {
|
||||
if (Object.prototype.hasOwnProperty.call(toolButtons, key)) {
|
||||
toolButtons[key].setAttribute('aria-pressed', key === next ? 'true' : 'false');
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// ------------------------------------------------------------- interaction
|
||||
|
||||
var drag = null;
|
||||
|
||||
var onPointerMove = function (event) {
|
||||
if (drag) {
|
||||
if (drag.kind === 'marquee') {
|
||||
drag.rect = normalizeRect(drag.startX, drag.startY, event.clientX, event.clientY);
|
||||
positionBox(marqueeBox, drag.rect);
|
||||
} else if (drag.kind === 'draw') {
|
||||
drag.points.push({ x: event.clientX, y: event.clientY });
|
||||
drag.path.setAttribute('d', drag.points.map(function (point, index) {
|
||||
return (index === 0 ? 'M' : 'L') + point.x + ' ' + point.y;
|
||||
}).join(' '));
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (tool !== 'select') return;
|
||||
var element = elementFromPoint(event.clientX, event.clientY);
|
||||
if (!element) {
|
||||
hoverBox.style.display = 'none';
|
||||
hoverLabel.style.display = 'none';
|
||||
return;
|
||||
}
|
||||
var rect = rectFrom(element.getBoundingClientRect());
|
||||
positionBox(hoverBox, rect);
|
||||
hoverLabel.textContent = selectorPart(element);
|
||||
hoverLabel.style.display = 'block';
|
||||
hoverLabel.style.transform = 'translate(' + Math.max(2, rect.x) + 'px,' + Math.max(2, rect.y - 19) + 'px)';
|
||||
};
|
||||
|
||||
var onPointerDown = function (event) {
|
||||
if (event.button !== 0) return;
|
||||
if (isOverlayNode(event.target)) return;
|
||||
event.preventDefault();
|
||||
event.stopPropagation();
|
||||
|
||||
if (tool === 'select') {
|
||||
var element = elementFromPoint(event.clientX, event.clientY);
|
||||
if (element) selectElement(element);
|
||||
return;
|
||||
}
|
||||
|
||||
if (tool === 'marquee') {
|
||||
drag = { kind: 'marquee', startX: event.clientX, startY: event.clientY, rect: null };
|
||||
return;
|
||||
}
|
||||
|
||||
var path = document.createElementNS(svgNS, 'path');
|
||||
path.setAttribute('fill', 'none');
|
||||
path.setAttribute('stroke', THEME.primary);
|
||||
path.setAttribute('stroke-width', '2.5');
|
||||
path.setAttribute('stroke-linecap', 'round');
|
||||
path.setAttribute('stroke-linejoin', 'round');
|
||||
svg.appendChild(path);
|
||||
drag = { kind: 'draw', path: path, points: [{ x: event.clientX, y: event.clientY }] };
|
||||
};
|
||||
|
||||
var onPointerUp = function () {
|
||||
if (!drag) return;
|
||||
var finished = drag;
|
||||
drag = null;
|
||||
|
||||
if (finished.kind === 'marquee') {
|
||||
marqueeBox.style.display = 'none';
|
||||
var rect = finished.rect;
|
||||
if (!rect || !usableRect(rect)) return;
|
||||
// A region marks an area, not the elements inside it: expanding it into a
|
||||
// selection made the result depend on the page's markup rather than on
|
||||
// what was drawn.
|
||||
regions.push({ id: nextId('region'), rect: rect });
|
||||
var outline = document.createElementNS(svgNS, 'rect');
|
||||
outline.setAttribute('x', String(rect.x));
|
||||
outline.setAttribute('y', String(rect.y));
|
||||
outline.setAttribute('width', String(rect.width));
|
||||
outline.setAttribute('height', String(rect.height));
|
||||
outline.style.fill = THEME.primarySoft;
|
||||
outline.setAttribute('stroke', THEME.primary);
|
||||
outline.setAttribute('stroke-width', '1.5');
|
||||
outline.setAttribute('rx', '2');
|
||||
svg.appendChild(outline);
|
||||
lastTargetRect = rect;
|
||||
syncChrome();
|
||||
focusComment();
|
||||
return;
|
||||
}
|
||||
|
||||
var points = finished.points;
|
||||
if (points.length < 2) {
|
||||
if (finished.path.parentNode) finished.path.parentNode.removeChild(finished.path);
|
||||
return;
|
||||
}
|
||||
var minX = points[0].x, minY = points[0].y, maxX = points[0].x, maxY = points[0].y;
|
||||
for (var p = 1; p < points.length; p += 1) {
|
||||
minX = Math.min(minX, points[p].x);
|
||||
minY = Math.min(minY, points[p].y);
|
||||
maxX = Math.max(maxX, points[p].x);
|
||||
maxY = Math.max(maxY, points[p].y);
|
||||
}
|
||||
var bounds = { x: minX, y: minY, width: maxX - minX, height: maxY - minY };
|
||||
strokes.push({ id: nextId('stroke'), points: points, bounds: bounds });
|
||||
lastTargetRect = bounds;
|
||||
syncChrome();
|
||||
focusComment();
|
||||
};
|
||||
|
||||
/**
|
||||
* Swallows the page's own mouse handling while annotating.
|
||||
*
|
||||
* Blocking pointerdown is not enough: a click still follows it, so marking a
|
||||
* button pressed that button and marking a link navigated away from the page
|
||||
* being annotated. These are the events that reach page code, so these are
|
||||
* the ones that have to stop.
|
||||
*/
|
||||
var blockPageInteraction = function (event) {
|
||||
if (isOverlayNode(event.target)) return;
|
||||
event.preventDefault();
|
||||
event.stopPropagation();
|
||||
};
|
||||
var BLOCKED_EVENTS = ['click', 'auxclick', 'dblclick', 'mousedown', 'mouseup', 'contextmenu'];
|
||||
|
||||
var onScrollOrResize = function () {
|
||||
syncSelectionVisuals();
|
||||
positionEditor();
|
||||
};
|
||||
|
||||
var onKeyDown = function (event) {
|
||||
if (event.key === 'Escape') {
|
||||
event.preventDefault();
|
||||
event.stopImmediatePropagation();
|
||||
finish(null);
|
||||
return;
|
||||
}
|
||||
if (event.key === 'Enter' && !event.shiftKey && event.target === comment) {
|
||||
event.preventDefault();
|
||||
attach();
|
||||
}
|
||||
};
|
||||
|
||||
document.addEventListener('pointermove', onPointerMove, true);
|
||||
document.addEventListener('pointerdown', onPointerDown, true);
|
||||
document.addEventListener('pointerup', onPointerUp, true);
|
||||
BLOCKED_EVENTS.forEach(function (name) {
|
||||
window.addEventListener(name, blockPageInteraction, { capture: true, passive: false });
|
||||
});
|
||||
window.addEventListener('scroll', onScrollOrResize, true);
|
||||
window.addEventListener('resize', onScrollOrResize, true);
|
||||
window.addEventListener('keydown', onKeyDown, true);
|
||||
|
||||
// ------------------------------------------------------------------ finish
|
||||
|
||||
var teardown = function () {
|
||||
document.removeEventListener('pointermove', onPointerMove, true);
|
||||
document.removeEventListener('pointerdown', onPointerDown, true);
|
||||
document.removeEventListener('pointerup', onPointerUp, true);
|
||||
BLOCKED_EVENTS.forEach(function (name) {
|
||||
window.removeEventListener(name, blockPageInteraction, true);
|
||||
});
|
||||
window.removeEventListener('scroll', onScrollOrResize, true);
|
||||
window.removeEventListener('resize', onScrollOrResize, true);
|
||||
window.removeEventListener('keydown', onKeyDown, true);
|
||||
setCursor('');
|
||||
if (cursorStyle.parentNode) cursorStyle.parentNode.removeChild(cursorStyle);
|
||||
if (host.parentNode) host.parentNode.removeChild(host);
|
||||
try { delete window['${ANNOTATION_ACTIVE_FLAG}']; } catch (error) { /* non-configurable */ }
|
||||
};
|
||||
|
||||
var finish = function (payload) {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
teardown();
|
||||
// Removing the chrome is not enough: the host screenshots this page right
|
||||
// after we resolve, and the compositor can still hold a frame containing
|
||||
// our outlines. Yield two frames so the page has actually repainted.
|
||||
requestAnimationFrame(function () {
|
||||
requestAnimationFrame(function () { resolve(payload); });
|
||||
});
|
||||
};
|
||||
|
||||
var attach = function () {
|
||||
if (!hasTargets()) return;
|
||||
finish({
|
||||
id: 'annotation-' + Date.now(),
|
||||
pageUrl: String(location.href),
|
||||
pageTitle: String(document.title || ''),
|
||||
viewport: { width: window.innerWidth, height: window.innerHeight },
|
||||
devicePixelRatio: window.devicePixelRatio || 1,
|
||||
comment: comment.value,
|
||||
// Zero or one element, never more. Kept as a list so elements, regions
|
||||
// and strokes stay one uniform shape for everything downstream.
|
||||
elements: selected ? [{ id: selected.id, element: describe(selected.element) }] : [],
|
||||
regions: regions.map(function (entry) { return { id: entry.id, rect: entry.rect }; }),
|
||||
strokes: strokes.map(function (entry) {
|
||||
return { id: entry.id, points: entry.points, bounds: entry.bounds };
|
||||
})
|
||||
});
|
||||
};
|
||||
|
||||
submit.addEventListener('click', attach);
|
||||
|
||||
(document.body || document.documentElement).appendChild(host);
|
||||
setTool('select');
|
||||
syncChrome();
|
||||
});`;
|
||||
};
|
||||
@@ -0,0 +1,127 @@
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
|
||||
import { formatBrowserAnnotationPrompt } from './annotationPrompt';
|
||||
import type { BrowserAnnotationPayload, BrowserElementTarget } from './contract';
|
||||
|
||||
const element = (overrides: Partial<BrowserElementTarget> = {}): BrowserElementTarget => ({
|
||||
tag: 'button',
|
||||
text: ' Save changes ',
|
||||
selector: '#save',
|
||||
path: 'form > button#save',
|
||||
bounds: { x: 10.4, y: 20.6, width: 100, height: 40 },
|
||||
center: { x: 60, y: 40 },
|
||||
attributes: { id: 'save', 'aria-label': 'Save' },
|
||||
computedStyle: { display: 'flex', position: 'static', color: 'rgb(0, 0, 0)' },
|
||||
ancestry: [{ tag: 'form', selectorPart: 'form' }],
|
||||
...overrides,
|
||||
});
|
||||
|
||||
const basePayload = (overrides: Partial<BrowserAnnotationPayload> = {}): BrowserAnnotationPayload => ({
|
||||
id: 'annotation-1',
|
||||
pageUrl: 'http://localhost:5173/settings',
|
||||
pageTitle: 'Settings',
|
||||
viewport: { width: 1280, height: 800 },
|
||||
devicePixelRatio: 2,
|
||||
comment: '',
|
||||
elements: [],
|
||||
regions: [],
|
||||
strokes: [],
|
||||
...overrides,
|
||||
});
|
||||
|
||||
describe('annotation prompt', () => {
|
||||
test('describes each selected element with selector, ancestry and attributes', () => {
|
||||
const output = formatBrowserAnnotationPrompt({
|
||||
payload: basePayload({ elements: [{ id: 'e1', element: element() }] }),
|
||||
screenshotAttached: true,
|
||||
intro: 'Selected elements:',
|
||||
});
|
||||
|
||||
expect(output).toContain('Selected elements:');
|
||||
expect(output).toContain('Element 1: button');
|
||||
expect(output).toContain('- Selector: #save');
|
||||
expect(output).toContain('- Ancestry: form');
|
||||
expect(output).toContain('aria-label="Save"');
|
||||
expect(output).toContain('Screenshot: attached');
|
||||
});
|
||||
|
||||
test('collapses element text whitespace', () => {
|
||||
const output = formatBrowserAnnotationPrompt({
|
||||
payload: basePayload({ elements: [{ id: 'e1', element: element() }] }),
|
||||
screenshotAttached: false,
|
||||
intro: 'Selected',
|
||||
});
|
||||
expect(output).toContain('- Text: Save changes');
|
||||
});
|
||||
|
||||
test('rounds geometry so the prompt has no float noise', () => {
|
||||
const output = formatBrowserAnnotationPrompt({
|
||||
payload: basePayload({ elements: [{ id: 'e1', element: element() }] }),
|
||||
screenshotAttached: false,
|
||||
intro: 'Selected',
|
||||
});
|
||||
expect(output).toContain('- Bounds: x=10, y=21, width=100, height=40');
|
||||
});
|
||||
|
||||
test('reports a missing screenshot rather than staying silent about it', () => {
|
||||
const output = formatBrowserAnnotationPrompt({
|
||||
payload: basePayload({ elements: [{ id: 'e1', element: element() }] }),
|
||||
screenshotAttached: false,
|
||||
intro: 'Selected',
|
||||
});
|
||||
expect(output).toContain('Screenshot: not attached');
|
||||
});
|
||||
|
||||
test('numbers multiple elements, regions and drawings independently', () => {
|
||||
const output = formatBrowserAnnotationPrompt({
|
||||
payload: basePayload({
|
||||
elements: [{ id: 'e1', element: element() }, { id: 'e2', element: element({ selector: '#cancel' }) }],
|
||||
regions: [{ id: 'r1', rect: { x: 0, y: 0, width: 10, height: 10 } }],
|
||||
strokes: [{ id: 's1', points: [{ x: 0, y: 0 }, { x: 5, y: 5 }], bounds: { x: 0, y: 0, width: 5, height: 5 } }],
|
||||
}),
|
||||
screenshotAttached: true,
|
||||
intro: 'Selected',
|
||||
});
|
||||
|
||||
expect(output).toContain('Element 1: button');
|
||||
expect(output).toContain('Element 2: button');
|
||||
expect(output).toContain('Region 1: x=0, y=0, width=10, height=10');
|
||||
expect(output).toContain('Drawing 1: 2 points');
|
||||
});
|
||||
|
||||
test('includes the comment only when the user wrote one', () => {
|
||||
const withComment = formatBrowserAnnotationPrompt({
|
||||
payload: basePayload({ comment: ' needs more contrast ', elements: [{ id: 'e1', element: element() }] }),
|
||||
screenshotAttached: false,
|
||||
intro: 'Selected',
|
||||
});
|
||||
expect(withComment).toContain('Comment: needs more contrast');
|
||||
|
||||
const withoutComment = formatBrowserAnnotationPrompt({
|
||||
payload: basePayload({ elements: [{ id: 'e1', element: element() }] }),
|
||||
screenshotAttached: false,
|
||||
intro: 'Selected',
|
||||
});
|
||||
expect(withoutComment).not.toContain('Comment:');
|
||||
});
|
||||
|
||||
test('falls back to the url when the page has no title', () => {
|
||||
const output = formatBrowserAnnotationPrompt({
|
||||
payload: basePayload({ pageTitle: '' }),
|
||||
screenshotAttached: false,
|
||||
intro: 'Selected',
|
||||
});
|
||||
expect(output).toContain('Page: http://localhost:5173/settings');
|
||||
expect(output).not.toContain('URL:');
|
||||
});
|
||||
|
||||
test('keeps the url on its own line when a title is present', () => {
|
||||
const output = formatBrowserAnnotationPrompt({
|
||||
payload: basePayload(),
|
||||
screenshotAttached: false,
|
||||
intro: 'Selected',
|
||||
});
|
||||
expect(output).toContain('Page: Settings');
|
||||
expect(output).toContain('URL: http://localhost:5173/settings');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,83 @@
|
||||
/**
|
||||
* Renders a browser annotation into the text the agent actually reads.
|
||||
*
|
||||
* The body is deliberately English regardless of UI locale: it is prompt
|
||||
* content, not interface copy. Only the caller-supplied intro is translated,
|
||||
* because that line is echoed back to the user in the composer.
|
||||
*/
|
||||
import type {
|
||||
BrowserAnnotationPayload,
|
||||
BrowserAnnotationRegion,
|
||||
BrowserAnnotationStroke,
|
||||
BrowserElementTarget,
|
||||
} from './contract';
|
||||
|
||||
const round = (value: number): number => Math.round(value);
|
||||
|
||||
const describeElement = (target: BrowserElementTarget, index: number): string[] => {
|
||||
const text = target.text.trim();
|
||||
const attributes = Object.entries(target.attributes)
|
||||
.map(([key, value]) => `${key}="${value}"`)
|
||||
.join(' ');
|
||||
const ancestry = target.ancestry.map((entry) => entry.selectorPart).join(' > ');
|
||||
const styles = target.computedStyle;
|
||||
const bounds = target.bounds;
|
||||
|
||||
return [
|
||||
`Element ${index + 1}: ${target.tag}`,
|
||||
text ? `- Text: ${text}` : null,
|
||||
`- Selector: ${target.selector}`,
|
||||
`- Path: ${target.path}`,
|
||||
ancestry ? `- Ancestry: ${ancestry}` : null,
|
||||
attributes ? `- Attributes: ${attributes}` : null,
|
||||
`- Bounds: x=${round(bounds.x)}, y=${round(bounds.y)}, width=${round(bounds.width)}, height=${round(bounds.height)}`,
|
||||
`- Styles: display=${styles.display ?? ''}; position=${styles.position ?? ''}; font=${styles.fontWeight ?? ''} ${styles.fontSize ?? ''} / ${styles.lineHeight ?? ''} ${styles.fontFamily ?? ''}; color=${styles.color ?? ''}; background=${styles.backgroundColor ?? ''}`,
|
||||
].filter((line): line is string => typeof line === 'string');
|
||||
};
|
||||
|
||||
const describeRegion = (region: BrowserAnnotationRegion, index: number): string => {
|
||||
const rect = region.rect;
|
||||
return `Region ${index + 1}: x=${round(rect.x)}, y=${round(rect.y)}, width=${round(rect.width)}, height=${round(rect.height)}`;
|
||||
};
|
||||
|
||||
const describeStroke = (stroke: BrowserAnnotationStroke, index: number): string => {
|
||||
const bounds = stroke.bounds;
|
||||
return `Drawing ${index + 1}: ${stroke.points.length} points over x=${round(bounds.x)}, y=${round(bounds.y)}, width=${round(bounds.width)}, height=${round(bounds.height)}`;
|
||||
};
|
||||
|
||||
export const formatBrowserAnnotationPrompt = ({
|
||||
payload,
|
||||
screenshotAttached,
|
||||
intro,
|
||||
}: {
|
||||
payload: BrowserAnnotationPayload;
|
||||
screenshotAttached: boolean;
|
||||
intro: string;
|
||||
}): string => {
|
||||
const introLabel = intro.replace(/[.:]+$/g, '');
|
||||
const comment = payload.comment.trim();
|
||||
const lines: Array<string | null> = [
|
||||
`${introLabel}:`,
|
||||
`Page: ${payload.pageTitle.trim() || payload.pageUrl || 'browser'}`,
|
||||
payload.pageTitle.trim() && payload.pageUrl ? `URL: ${payload.pageUrl}` : null,
|
||||
`Viewport: ${round(payload.viewport.width)}x${round(payload.viewport.height)}, DPR ${payload.devicePixelRatio}`,
|
||||
`Screenshot: ${screenshotAttached ? 'attached' : 'not attached'}`,
|
||||
comment ? `Comment: ${comment}` : null,
|
||||
];
|
||||
|
||||
for (const [index, entry] of payload.elements.entries()) {
|
||||
lines.push('', ...describeElement(entry.element, index));
|
||||
}
|
||||
|
||||
if (payload.regions.length > 0) {
|
||||
lines.push('');
|
||||
payload.regions.forEach((region, index) => lines.push(describeRegion(region, index)));
|
||||
}
|
||||
|
||||
if (payload.strokes.length > 0) {
|
||||
lines.push('');
|
||||
payload.strokes.forEach((stroke, index) => lines.push(describeStroke(stroke, index)));
|
||||
}
|
||||
|
||||
return lines.filter((line): line is string => typeof line === 'string').join('\n');
|
||||
};
|
||||
@@ -0,0 +1,118 @@
|
||||
/**
|
||||
* Turns a native page capture plus an annotation payload into the image the
|
||||
* agent receives.
|
||||
*
|
||||
* The image is the whole visible page, with the marked targets drawn on top —
|
||||
* not a crop around them. A tight crop answers "what does this element look
|
||||
* like" but destroys the answer to "where is it and what is it next to", which
|
||||
* is usually the more useful half of pointing at something.
|
||||
*
|
||||
* The capture arrives in device pixels while every rectangle in the payload is
|
||||
* in CSS pixels, so everything is scaled by the ratio between the two rather
|
||||
* than by `devicePixelRatio` — the page may be zoomed, and the measured ratio
|
||||
* stays correct when it is.
|
||||
*/
|
||||
import type { BrowserAnnotationPayload, BrowserRect } from './contract';
|
||||
|
||||
const MAX_OUTPUT_WIDTH = 1600;
|
||||
|
||||
const loadImage = (src: string): Promise<HTMLImageElement> => new Promise((resolve, reject) => {
|
||||
const image = new Image();
|
||||
image.onload = () => resolve(image);
|
||||
image.onerror = () => reject(new Error('Failed to decode browser page capture'));
|
||||
image.src = src;
|
||||
});
|
||||
|
||||
export const renderAnnotationScreenshot = async ({
|
||||
base64,
|
||||
mime,
|
||||
captureWidth,
|
||||
captureHeight,
|
||||
cssWidth,
|
||||
cssHeight,
|
||||
payload,
|
||||
accentColor,
|
||||
accentFill,
|
||||
}: {
|
||||
base64: string;
|
||||
mime: string;
|
||||
captureWidth: number;
|
||||
captureHeight: number;
|
||||
cssWidth: number;
|
||||
cssHeight: number;
|
||||
payload: BrowserAnnotationPayload;
|
||||
/** Solid outline color. */
|
||||
accentColor: string;
|
||||
/**
|
||||
* Translucent fill, resolved by the caller. Never derive this by appending an
|
||||
* alpha suffix to `accentColor`: theme colors are not necessarily hex, and an
|
||||
* invalid value leaves canvas on its default opaque black, which paints over
|
||||
* the very element the screenshot is meant to show.
|
||||
*/
|
||||
accentFill: string;
|
||||
}): Promise<File | null> => {
|
||||
if (!base64) return null;
|
||||
|
||||
try {
|
||||
const image = await loadImage(`data:${mime};base64,${base64}`);
|
||||
const pixelWidth = Math.max(1, image.naturalWidth || captureWidth);
|
||||
const pixelHeight = Math.max(1, image.naturalHeight || captureHeight);
|
||||
const scaleX = pixelWidth / Math.max(1, cssWidth || pixelWidth);
|
||||
const scaleY = pixelHeight / Math.max(1, cssHeight || pixelHeight);
|
||||
|
||||
const outputScale = Math.min(1, MAX_OUTPUT_WIDTH / pixelWidth);
|
||||
const canvas = document.createElement('canvas');
|
||||
canvas.width = Math.max(1, Math.floor(pixelWidth * outputScale));
|
||||
canvas.height = Math.max(1, Math.floor(pixelHeight * outputScale));
|
||||
const context = canvas.getContext('2d');
|
||||
if (!context) return null;
|
||||
|
||||
context.scale(outputScale, outputScale);
|
||||
context.drawImage(image, 0, 0, pixelWidth, pixelHeight);
|
||||
|
||||
/** CSS pixels to capture pixels; the canvas transform handles the rest. */
|
||||
const toCanvas = (rect: BrowserRect): BrowserRect => ({
|
||||
x: rect.x * scaleX,
|
||||
y: rect.y * scaleY,
|
||||
width: rect.width * scaleX,
|
||||
height: rect.height * scaleY,
|
||||
});
|
||||
|
||||
// Scale the outline with the image so it stays visible once a full page is
|
||||
// shrunk to the output width; a hairline on a 1600px-wide page is lost.
|
||||
const outlineWidth = Math.max(2, 2.5 * scaleX / Math.max(outputScale, 0.2));
|
||||
context.strokeStyle = accentColor;
|
||||
context.fillStyle = accentFill;
|
||||
|
||||
const markRect = (rect: BrowserRect) => {
|
||||
const box = toCanvas(rect);
|
||||
context.lineWidth = outlineWidth;
|
||||
context.fillRect(box.x, box.y, box.width, box.height);
|
||||
context.strokeRect(box.x, box.y, box.width, box.height);
|
||||
};
|
||||
|
||||
for (const entry of payload.elements) markRect(entry.element.bounds);
|
||||
for (const region of payload.regions) markRect(region.rect);
|
||||
|
||||
for (const stroke of payload.strokes) {
|
||||
if (stroke.points.length < 2) continue;
|
||||
context.beginPath();
|
||||
stroke.points.forEach((point, index) => {
|
||||
const x = point.x * scaleX;
|
||||
const y = point.y * scaleY;
|
||||
if (index === 0) context.moveTo(x, y);
|
||||
else context.lineTo(x, y);
|
||||
});
|
||||
context.lineCap = 'round';
|
||||
context.lineJoin = 'round';
|
||||
context.lineWidth = outlineWidth * 1.4;
|
||||
context.stroke();
|
||||
}
|
||||
|
||||
const blob = await new Promise<Blob | null>((resolve) => canvas.toBlob(resolve, 'image/jpeg', 0.85));
|
||||
if (!blob) return null;
|
||||
return new File([blob], `browser-annotation-${Date.now()}.jpg`, { type: 'image/jpeg' });
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,136 @@
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
|
||||
import { cancelAnnotationSession, runAnnotationSession, type AnnotationHost, type PageCapture } from './annotationSession';
|
||||
import type { BrowserAnnotationOverlayLabels, BrowserAnnotationOverlayTheme } from './annotationOverlay';
|
||||
|
||||
const theme: BrowserAnnotationOverlayTheme = {
|
||||
colorScheme: 'dark',
|
||||
primary: 'rgb(1, 2, 3)',
|
||||
primarySoft: 'rgba(1, 2, 3, 0.16)',
|
||||
primaryFaint: 'rgba(1, 2, 3, 0.1)',
|
||||
primaryContrast: 'rgb(255, 255, 255)',
|
||||
surface: 'rgb(10, 10, 10)',
|
||||
surfaceElevated: 'rgb(20, 20, 20)',
|
||||
glassSurface: 'rgba(20, 20, 20, 0.64)',
|
||||
glassFilter: 'blur(26px) saturate(1.16)',
|
||||
border: 'rgb(30, 30, 30)',
|
||||
text: 'rgb(240, 240, 240)',
|
||||
mutedText: 'rgb(160, 160, 160)',
|
||||
};
|
||||
|
||||
const labels = {
|
||||
select: 'Element', marquee: 'Region', draw: 'Draw',
|
||||
commentPlaceholder: 'Describe', submit: 'Attach',
|
||||
} satisfies BrowserAnnotationOverlayLabels;
|
||||
|
||||
const validPayload = {
|
||||
id: 'annotation-1',
|
||||
pageUrl: 'http://localhost:5173/',
|
||||
pageTitle: 'App',
|
||||
viewport: { width: 1000, height: 700 },
|
||||
devicePixelRatio: 1,
|
||||
comment: 'tighten this',
|
||||
elements: [{
|
||||
id: 'element-1',
|
||||
element: {
|
||||
tag: 'div',
|
||||
text: 'Hi',
|
||||
selector: '#hero',
|
||||
path: 'main > div#hero',
|
||||
bounds: { x: 0, y: 0, width: 100, height: 50 },
|
||||
center: { x: 50, y: 25 },
|
||||
attributes: {},
|
||||
computedStyle: {},
|
||||
ancestry: [],
|
||||
},
|
||||
}],
|
||||
regions: [],
|
||||
strokes: [],
|
||||
};
|
||||
|
||||
const capture: PageCapture = { mime: 'image/jpeg', base64: 'AAAA', width: 1000, height: 700 };
|
||||
|
||||
type Call = { code: string; gesture?: boolean };
|
||||
|
||||
const createHost = (options: {
|
||||
overlayResult: unknown;
|
||||
capturePage?: () => Promise<PageCapture | null>;
|
||||
}): { host: AnnotationHost; calls: Call[] } => {
|
||||
const calls: Call[] = [];
|
||||
const host: AnnotationHost = {
|
||||
executeJavaScript: async (code: string, gesture?: boolean) => {
|
||||
calls.push({ code, gesture });
|
||||
if (code.includes('new Promise')) return options.overlayResult;
|
||||
if (code.includes('window.innerWidth')) return { width: 1000, height: 700 };
|
||||
return undefined;
|
||||
},
|
||||
capturePage: options.capturePage ?? (async () => capture),
|
||||
};
|
||||
return { host, calls };
|
||||
};
|
||||
|
||||
|
||||
describe('annotation session', () => {
|
||||
test('returns null when the user cancels inside the page', async () => {
|
||||
const { host } = createHost({ overlayResult: null });
|
||||
expect(await runAnnotationSession({ host, theme, labels })).toBeNull();
|
||||
});
|
||||
|
||||
test('does not capture anything when the overlay was cancelled', async () => {
|
||||
let captures = 0;
|
||||
const { host } = createHost({
|
||||
overlayResult: null,
|
||||
capturePage: async () => { captures += 1; return null; },
|
||||
});
|
||||
await runAnnotationSession({ host, theme, labels });
|
||||
expect(captures).toBe(0);
|
||||
});
|
||||
|
||||
test('discards a malformed payload and tears the overlay down', async () => {
|
||||
const { host, calls } = createHost({ overlayResult: { id: 'x', elements: 'not-an-array' } });
|
||||
const result = await runAnnotationSession({ host, theme, labels });
|
||||
expect(result).toBeNull();
|
||||
expect(calls.some((call) => call.code.includes('data-openchamber-annotation'))).toBe(true);
|
||||
});
|
||||
|
||||
test('returns the annotation when the capture succeeds', async () => {
|
||||
const { host } = createHost({ overlayResult: validPayload });
|
||||
const result = await runAnnotationSession({ host, theme, labels });
|
||||
expect(result?.payload.id).toBe('annotation-1');
|
||||
});
|
||||
|
||||
test('keeps the annotation when the capture throws', async () => {
|
||||
// A failed screenshot degrades the annotation; it must not discard it.
|
||||
const { host } = createHost({
|
||||
overlayResult: validPayload,
|
||||
capturePage: async () => { throw new Error('capture failed'); },
|
||||
});
|
||||
|
||||
const result = await runAnnotationSession({ host, theme, labels });
|
||||
|
||||
expect(result?.payload.id).toBe('annotation-1');
|
||||
expect(result?.screenshot).toBeNull();
|
||||
});
|
||||
|
||||
test('keeps the annotation when capture returns nothing', async () => {
|
||||
const { host } = createHost({ overlayResult: validPayload, capturePage: async () => null });
|
||||
const result = await runAnnotationSession({ host, theme, labels });
|
||||
expect(result?.payload.comment).toBe('tighten this');
|
||||
expect(result?.screenshot).toBeNull();
|
||||
});
|
||||
|
||||
test('runs the overlay with a user gesture so the page treats it as interactive', async () => {
|
||||
const { host, calls } = createHost({ overlayResult: null });
|
||||
await runAnnotationSession({ host, theme, labels });
|
||||
expect(calls[0]?.gesture).toBe(true);
|
||||
});
|
||||
|
||||
test('cancelling a stale session tolerates a destroyed page', async () => {
|
||||
const host: AnnotationHost = {
|
||||
executeJavaScript: async () => { throw new Error('webview destroyed'); },
|
||||
capturePage: async () => null,
|
||||
};
|
||||
// Resolving at all is the contract: a destroyed page must not throw.
|
||||
expect(await cancelAnnotationSession(host)).toBeFalsy();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,106 @@
|
||||
/**
|
||||
* Drives one annotation session end to end.
|
||||
*
|
||||
* The overlay resolves only after tearing its own chrome down and waiting for a
|
||||
* repaint, so the capture that follows shows the page and none of our UI. The
|
||||
* page itself is never modified: annotation marks what is there, it does not
|
||||
* edit it.
|
||||
*/
|
||||
import {
|
||||
ANNOTATION_TEARDOWN_SCRIPT,
|
||||
buildAnnotationOverlayScript,
|
||||
type BrowserAnnotationOverlayLabels,
|
||||
type BrowserAnnotationOverlayTheme,
|
||||
} from './annotationOverlay';
|
||||
import { isBrowserAnnotationPayload, type BrowserAnnotationPayload } from './contract';
|
||||
import { renderAnnotationScreenshot } from './annotationScreenshot';
|
||||
|
||||
export type PageCapture = {
|
||||
readonly mime: string;
|
||||
readonly base64: string;
|
||||
readonly width: number;
|
||||
readonly height: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* The capabilities an annotation session needs from its host, named so the
|
||||
* session can be exercised without a live `<webview>`.
|
||||
*/
|
||||
export type AnnotationHost = {
|
||||
readonly executeJavaScript: (code: string, userGesture?: boolean) => Promise<unknown>;
|
||||
readonly capturePage: () => Promise<PageCapture | null>;
|
||||
};
|
||||
|
||||
export type AnnotationSessionResult = {
|
||||
readonly payload: BrowserAnnotationPayload;
|
||||
readonly screenshot: File | null;
|
||||
};
|
||||
|
||||
const VIEWPORT_SCRIPT = '({ width: window.innerWidth, height: window.innerHeight })';
|
||||
|
||||
const readViewport = async (host: AnnotationHost): Promise<{ width: number; height: number } | null> => {
|
||||
try {
|
||||
const value = await host.executeJavaScript(VIEWPORT_SCRIPT, true);
|
||||
if (!value || typeof value !== 'object') return null;
|
||||
const record = value as { width?: unknown; height?: unknown };
|
||||
if (typeof record.width !== 'number' || typeof record.height !== 'number') return null;
|
||||
if (!Number.isFinite(record.width) || !Number.isFinite(record.height)) return null;
|
||||
return { width: record.width, height: record.height };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
/** Best-effort cleanup of an overlay left behind by a previous session. */
|
||||
export const cancelAnnotationSession = async (host: AnnotationHost): Promise<void> => {
|
||||
try {
|
||||
await host.executeJavaScript(ANNOTATION_TEARDOWN_SCRIPT, false);
|
||||
} catch {
|
||||
// The page navigated or was destroyed; the overlay went with it.
|
||||
}
|
||||
};
|
||||
|
||||
export const runAnnotationSession = async ({
|
||||
host,
|
||||
theme,
|
||||
labels,
|
||||
}: {
|
||||
host: AnnotationHost;
|
||||
theme: BrowserAnnotationOverlayTheme;
|
||||
labels: BrowserAnnotationOverlayLabels;
|
||||
}): Promise<AnnotationSessionResult | null> => {
|
||||
const script = buildAnnotationOverlayScript(theme, labels);
|
||||
const raw = await host.executeJavaScript(script, true);
|
||||
|
||||
// Cancelled from inside the page.
|
||||
if (raw === null || raw === undefined) return null;
|
||||
|
||||
if (!isBrowserAnnotationPayload(raw)) {
|
||||
await cancelAnnotationSession(host);
|
||||
return null;
|
||||
}
|
||||
const payload = raw;
|
||||
|
||||
try {
|
||||
const viewport = await readViewport(host) ?? payload.viewport;
|
||||
const capture = await host.capturePage();
|
||||
if (!capture || !capture.base64) {
|
||||
return { payload, screenshot: null };
|
||||
}
|
||||
|
||||
const screenshot = await renderAnnotationScreenshot({
|
||||
base64: capture.base64,
|
||||
mime: capture.mime || 'image/jpeg',
|
||||
captureWidth: capture.width,
|
||||
captureHeight: capture.height,
|
||||
cssWidth: viewport.width,
|
||||
cssHeight: viewport.height,
|
||||
payload,
|
||||
accentColor: theme.primary,
|
||||
accentFill: theme.primarySoft,
|
||||
});
|
||||
return { payload, screenshot };
|
||||
} catch {
|
||||
return { payload, screenshot: null };
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,53 @@
|
||||
import React from 'react';
|
||||
|
||||
/**
|
||||
* Addresses the servers of one directory announced when they started.
|
||||
*
|
||||
* Auto-discovery starts a command and watches what it prints. When several
|
||||
* servers announce themselves — a gateway and the apps behind it, an API beside
|
||||
* a site — there is no honest way to pick one, so the candidates are parked
|
||||
* here and the browser panel offers them.
|
||||
*
|
||||
* These beat port discovery when both are available: an app served under a base
|
||||
* path announces that path, and a listening socket cannot reveal it.
|
||||
*
|
||||
* Kept in memory only. They describe one run of one command; a stored copy is
|
||||
* exactly the kind of stale address that sent the panel to the wrong page.
|
||||
*/
|
||||
const announcedByDirectory = new Map<string, readonly string[]>();
|
||||
const listeners = new Set<() => void>();
|
||||
|
||||
const emit = (): void => {
|
||||
for (const listener of listeners) listener();
|
||||
};
|
||||
|
||||
export const setAnnouncedDevServers = (directory: string, urls: readonly string[]): void => {
|
||||
const key = directory.trim();
|
||||
if (!key) return;
|
||||
if (urls.length === 0) announcedByDirectory.delete(key);
|
||||
else announcedByDirectory.set(key, [...urls]);
|
||||
emit();
|
||||
};
|
||||
|
||||
export const clearAnnouncedDevServers = (directory: string): void => {
|
||||
if (announcedByDirectory.delete(directory.trim())) emit();
|
||||
};
|
||||
|
||||
const EMPTY: readonly string[] = [];
|
||||
|
||||
const getAnnouncedDevServers = (directory: string): readonly string[] => (
|
||||
announcedByDirectory.get(directory.trim()) ?? EMPTY
|
||||
);
|
||||
|
||||
const subscribe = (listener: () => void): (() => void) => {
|
||||
listeners.add(listener);
|
||||
return () => { listeners.delete(listener); };
|
||||
};
|
||||
|
||||
export const useAnnouncedDevServers = (directory: string): readonly string[] => (
|
||||
React.useSyncExternalStore(
|
||||
subscribe,
|
||||
() => getAnnouncedDevServers(directory),
|
||||
() => EMPTY,
|
||||
)
|
||||
);
|
||||
@@ -0,0 +1,94 @@
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
|
||||
import {
|
||||
annotationTargetCount,
|
||||
isBrowserAnnotationPayload,
|
||||
isBrowserElementTarget,
|
||||
navStatusUrl,
|
||||
type BrowserAnnotationPayload,
|
||||
type BrowserElementTarget,
|
||||
} from './contract';
|
||||
|
||||
const element: BrowserElementTarget = {
|
||||
tag: 'button',
|
||||
text: 'Save',
|
||||
selector: '#save',
|
||||
path: 'main > form > button#save',
|
||||
bounds: { x: 10, y: 20, width: 100, height: 40 },
|
||||
center: { x: 60, y: 40 },
|
||||
attributes: { id: 'save' },
|
||||
computedStyle: { display: 'flex' },
|
||||
ancestry: [{ tag: 'form', selectorPart: 'form' }],
|
||||
};
|
||||
|
||||
const payload: BrowserAnnotationPayload = {
|
||||
id: 'annotation-1',
|
||||
pageUrl: 'http://localhost:5173/settings',
|
||||
pageTitle: 'Settings',
|
||||
viewport: { width: 1280, height: 800 },
|
||||
devicePixelRatio: 2,
|
||||
comment: 'Make this primary',
|
||||
elements: [{ id: 'element-1', element }],
|
||||
regions: [{ id: 'region-1', rect: { x: 200, y: 0, width: 50, height: 50 } }],
|
||||
strokes: [],
|
||||
};
|
||||
|
||||
describe('navigation status', () => {
|
||||
test('idle carries no url; the other states carry the one they describe', () => {
|
||||
expect(navStatusUrl({ kind: 'idle' })).toBe('');
|
||||
expect(navStatusUrl({ kind: 'loading', url: 'http://a/' })).toBe('http://a/');
|
||||
expect(navStatusUrl({ kind: 'ready', url: 'http://a/', title: 'A' })).toBe('http://a/');
|
||||
expect(navStatusUrl({ kind: 'failed', url: 'http://a/', code: -6, description: 'FILE_NOT_FOUND' }))
|
||||
.toBe('http://a/');
|
||||
});
|
||||
});
|
||||
|
||||
describe('element target validation', () => {
|
||||
test('accepts a fully-formed target', () => {
|
||||
expect(isBrowserElementTarget(element)).toBe(true);
|
||||
});
|
||||
|
||||
test('rejects payloads that would only fail later, at prompt or draw time', () => {
|
||||
expect(isBrowserElementTarget({ ...element, attributes: { id: 3 } })).toBe(false);
|
||||
expect(isBrowserElementTarget({ ...element, computedStyle: { display: null } })).toBe(false);
|
||||
expect(isBrowserElementTarget({ ...element, ancestry: [{ tag: 'form' }] })).toBe(false);
|
||||
expect(isBrowserElementTarget({ ...element, center: { x: 1 } })).toBe(false);
|
||||
expect(isBrowserElementTarget({ ...element, bounds: { x: 1, y: 2, width: 3 } })).toBe(false);
|
||||
});
|
||||
|
||||
test('rejects non-finite geometry rather than passing NaN downstream', () => {
|
||||
expect(isBrowserElementTarget({ ...element, bounds: { x: Number.NaN, y: 0, width: 1, height: 1 } })).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('annotation payload validation', () => {
|
||||
test('accepts a complete payload', () => {
|
||||
expect(isBrowserAnnotationPayload(payload)).toBe(true);
|
||||
});
|
||||
|
||||
test('accepts a payload with nothing marked but a comment', () => {
|
||||
expect(isBrowserAnnotationPayload({ ...payload, elements: [], regions: [], strokes: [] }))
|
||||
.toBe(true);
|
||||
});
|
||||
|
||||
test('rejects a payload whose nested element is malformed', () => {
|
||||
expect(isBrowserAnnotationPayload({
|
||||
...payload,
|
||||
elements: [{ id: 'element-1', element: { ...element, selector: 12 } }],
|
||||
})).toBe(false);
|
||||
});
|
||||
|
||||
test('rejects a malformed stroke', () => {
|
||||
expect(isBrowserAnnotationPayload({
|
||||
...payload,
|
||||
strokes: [{ id: 's', points: [{ x: 1 }], bounds: { x: 0, y: 0, width: 1, height: 1 } }],
|
||||
})).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('target geometry', () => {
|
||||
test('counts every kind of target', () => {
|
||||
expect(annotationTargetCount(payload)).toBe(2);
|
||||
expect(annotationTargetCount({ ...payload, elements: [], regions: [], strokes: [] })).toBe(0);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,191 @@
|
||||
/**
|
||||
* Browser surface contract.
|
||||
*
|
||||
* One shared vocabulary for the in-app browser across every runtime. The
|
||||
* surface renders a real Chromium `<webview>` on desktop and a plain iframe
|
||||
* everywhere else; both report their state through the types below, so the
|
||||
* panel never has to ask which transport it is talking to.
|
||||
*
|
||||
* Navigation is a tagged union rather than a bag of booleans: `loading` and
|
||||
* `failed` carry the URL they describe, which is what makes a late event from
|
||||
* a superseded navigation discardable instead of ambiguous.
|
||||
*/
|
||||
|
||||
export type BrowserNavStatus =
|
||||
| { readonly kind: 'idle' }
|
||||
| { readonly kind: 'loading'; readonly url: string }
|
||||
| { readonly kind: 'ready'; readonly url: string; readonly title: string }
|
||||
| {
|
||||
readonly kind: 'failed';
|
||||
readonly url: string;
|
||||
readonly code: number;
|
||||
readonly description: string;
|
||||
/** The page's renderer died rather than the load failing; worth saying so. */
|
||||
readonly crashed?: boolean;
|
||||
};
|
||||
|
||||
export const IDLE_NAV_STATUS: BrowserNavStatus = { kind: 'idle' };
|
||||
|
||||
/** The URL a nav status refers to, or '' when idle. */
|
||||
export const navStatusUrl = (status: BrowserNavStatus): string => (
|
||||
status.kind === 'idle' ? '' : status.url
|
||||
);
|
||||
|
||||
export type BrowserRect = {
|
||||
readonly x: number;
|
||||
readonly y: number;
|
||||
readonly width: number;
|
||||
readonly height: number;
|
||||
};
|
||||
|
||||
export type BrowserPoint = { readonly x: number; readonly y: number };
|
||||
|
||||
export type BrowserElementAncestor = {
|
||||
readonly tag: string;
|
||||
readonly id?: string;
|
||||
readonly className?: string;
|
||||
readonly selectorPart: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* A single DOM element described well enough for an agent to find it again in
|
||||
* source: a selector, a readable ancestry path, its own box, and the computed
|
||||
* styles that usually matter when someone is asking for a visual change.
|
||||
*/
|
||||
export type BrowserElementTarget = {
|
||||
readonly tag: string;
|
||||
readonly text: string;
|
||||
readonly selector: string;
|
||||
readonly path: string;
|
||||
readonly bounds: BrowserRect;
|
||||
readonly center: BrowserPoint;
|
||||
readonly attributes: Readonly<Record<string, string>>;
|
||||
readonly computedStyle: Readonly<Record<string, string>>;
|
||||
readonly ancestry: ReadonlyArray<BrowserElementAncestor>;
|
||||
};
|
||||
|
||||
export type BrowserAnnotationElement = {
|
||||
readonly id: string;
|
||||
readonly element: BrowserElementTarget;
|
||||
};
|
||||
|
||||
/** A free-form rectangle the user dragged over a part of the page. */
|
||||
export type BrowserAnnotationRegion = {
|
||||
readonly id: string;
|
||||
readonly rect: BrowserRect;
|
||||
};
|
||||
|
||||
/** A free-hand stroke drawn over the page. */
|
||||
export type BrowserAnnotationStroke = {
|
||||
readonly id: string;
|
||||
readonly points: ReadonlyArray<BrowserPoint>;
|
||||
readonly bounds: BrowserRect;
|
||||
};
|
||||
|
||||
/**
|
||||
* The complete result of one annotation session: everything the user marked,
|
||||
* everything they restyled, and what they said about it. Emitted once, on
|
||||
* submit — never per-click.
|
||||
*/
|
||||
export type BrowserAnnotationPayload = {
|
||||
readonly id: string;
|
||||
readonly pageUrl: string;
|
||||
readonly pageTitle: string;
|
||||
readonly viewport: { readonly width: number; readonly height: number };
|
||||
readonly devicePixelRatio: number;
|
||||
readonly comment: string;
|
||||
readonly elements: ReadonlyArray<BrowserAnnotationElement>;
|
||||
readonly regions: ReadonlyArray<BrowserAnnotationRegion>;
|
||||
readonly strokes: ReadonlyArray<BrowserAnnotationStroke>;
|
||||
};
|
||||
|
||||
export const annotationTargetCount = (payload: BrowserAnnotationPayload): number => (
|
||||
payload.elements.length + payload.regions.length + payload.strokes.length
|
||||
);
|
||||
|
||||
const isRecord = (value: unknown): value is Record<string, unknown> => (
|
||||
typeof value === 'object' && value !== null
|
||||
);
|
||||
|
||||
const isFiniteNumber = (value: unknown): value is number => (
|
||||
typeof value === 'number' && Number.isFinite(value)
|
||||
);
|
||||
|
||||
const isStringRecord = (value: unknown): value is Record<string, string> => (
|
||||
isRecord(value) && Object.values(value).every((entry) => typeof entry === 'string')
|
||||
);
|
||||
|
||||
const isBrowserRect = (value: unknown): value is BrowserRect => (
|
||||
isRecord(value)
|
||||
&& isFiniteNumber(value.x)
|
||||
&& isFiniteNumber(value.y)
|
||||
&& isFiniteNumber(value.width)
|
||||
&& isFiniteNumber(value.height)
|
||||
);
|
||||
|
||||
const isBrowserPoint = (value: unknown): value is BrowserPoint => (
|
||||
isRecord(value) && isFiniteNumber(value.x) && isFiniteNumber(value.y)
|
||||
);
|
||||
|
||||
const isAncestor = (value: unknown): value is BrowserElementAncestor => (
|
||||
isRecord(value)
|
||||
&& typeof value.tag === 'string'
|
||||
&& typeof value.selectorPart === 'string'
|
||||
&& (value.id === undefined || typeof value.id === 'string')
|
||||
&& (value.className === undefined || typeof value.className === 'string')
|
||||
);
|
||||
|
||||
/**
|
||||
* Annotation payloads cross a trust boundary: they are produced by a script
|
||||
* running inside a page we do not control. Every field a consumer dereferences
|
||||
* is validated here, not just the ones that are convenient to check — a
|
||||
* partially-valid payload would otherwise throw far away from its origin, at
|
||||
* prompt-formatting or screenshot-crop time.
|
||||
*/
|
||||
export const isBrowserElementTarget = (value: unknown): value is BrowserElementTarget => (
|
||||
isRecord(value)
|
||||
&& typeof value.tag === 'string'
|
||||
&& typeof value.text === 'string'
|
||||
&& typeof value.selector === 'string'
|
||||
&& typeof value.path === 'string'
|
||||
&& isBrowserRect(value.bounds)
|
||||
&& isBrowserPoint(value.center)
|
||||
&& isStringRecord(value.attributes)
|
||||
&& isStringRecord(value.computedStyle)
|
||||
&& Array.isArray(value.ancestry)
|
||||
&& value.ancestry.every(isAncestor)
|
||||
);
|
||||
|
||||
const isAnnotationElement = (value: unknown): value is BrowserAnnotationElement => (
|
||||
isRecord(value) && typeof value.id === 'string' && isBrowserElementTarget(value.element)
|
||||
);
|
||||
|
||||
const isAnnotationRegion = (value: unknown): value is BrowserAnnotationRegion => (
|
||||
isRecord(value) && typeof value.id === 'string' && isBrowserRect(value.rect)
|
||||
);
|
||||
|
||||
const isAnnotationStroke = (value: unknown): value is BrowserAnnotationStroke => (
|
||||
isRecord(value)
|
||||
&& typeof value.id === 'string'
|
||||
&& isBrowserRect(value.bounds)
|
||||
&& Array.isArray(value.points)
|
||||
&& value.points.every(isBrowserPoint)
|
||||
);
|
||||
|
||||
export const isBrowserAnnotationPayload = (value: unknown): value is BrowserAnnotationPayload => (
|
||||
isRecord(value)
|
||||
&& typeof value.id === 'string'
|
||||
&& typeof value.pageUrl === 'string'
|
||||
&& typeof value.pageTitle === 'string'
|
||||
&& typeof value.comment === 'string'
|
||||
&& isRecord(value.viewport)
|
||||
&& isFiniteNumber(value.viewport.width)
|
||||
&& isFiniteNumber(value.viewport.height)
|
||||
&& isFiniteNumber(value.devicePixelRatio)
|
||||
&& Array.isArray(value.elements)
|
||||
&& value.elements.every(isAnnotationElement)
|
||||
&& Array.isArray(value.regions)
|
||||
&& value.regions.every(isAnnotationRegion)
|
||||
&& Array.isArray(value.strokes)
|
||||
&& value.strokes.every(isAnnotationStroke)
|
||||
);
|
||||
@@ -0,0 +1,133 @@
|
||||
import { afterEach, beforeEach, describe, expect, mock, test } from 'bun:test';
|
||||
|
||||
type Listener = (event: { type: string; requestId: string; action: string; parameters: Record<string, unknown> }) => void;
|
||||
|
||||
const posted: Array<{ requestId: string; ok: boolean; data?: unknown; error?: string }> = [];
|
||||
const claims: string[] = [];
|
||||
/** Flipped to false to play the client that lost the race for a request. */
|
||||
let grantClaims = true;
|
||||
let listener: Listener | null = null;
|
||||
|
||||
mock.module('@/lib/runtime-fetch', () => ({
|
||||
runtimeFetch: mock(async (path: string, init?: { body?: string }) => {
|
||||
const body = JSON.parse(init?.body ?? '{}');
|
||||
if (path.endsWith('/claim')) {
|
||||
claims.push(body.requestId);
|
||||
return { ok: true, status: 200, json: async () => ({ granted: grantClaims }) };
|
||||
}
|
||||
posted.push(body);
|
||||
return { ok: true, status: 200 };
|
||||
}),
|
||||
}));
|
||||
mock.module('@/lib/openchamberEvents', () => ({
|
||||
subscribeOpenchamberEvents: (handler: Listener) => {
|
||||
listener = handler;
|
||||
return () => { listener = null; };
|
||||
},
|
||||
}));
|
||||
|
||||
const { registerBrowserController, registerBrowserOpener } = await import('./controlClient');
|
||||
|
||||
/** Registrations are module-global, so every test unwinds its own. */
|
||||
const cleanups: Array<() => void> = [];
|
||||
|
||||
const emitOpen = (parameters: Record<string, unknown>): void => {
|
||||
listener?.({ type: 'browser-control-request', requestId: 'req-1', action: 'browser.open', parameters });
|
||||
};
|
||||
|
||||
const wait = (ms: number): Promise<void> => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
|
||||
describe('opening a page before any view exists', () => {
|
||||
beforeEach(() => {
|
||||
posted.length = 0;
|
||||
claims.length = 0;
|
||||
grantClaims = true;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
while (cleanups.length > 0) cleanups.pop()?.();
|
||||
});
|
||||
|
||||
test('lets the view that the open created apply the layout that was asked for', async () => {
|
||||
const opened: string[] = [];
|
||||
const ran: Array<{ action: string; parameters: Record<string, unknown> }> = [];
|
||||
|
||||
cleanups.push(registerBrowserOpener((url) => {
|
||||
opened.push(url);
|
||||
// The pane mounts a moment after the tab is created, as it does in the app.
|
||||
setTimeout(() => {
|
||||
cleanups.push(registerBrowserController({
|
||||
run: async (action, parameters) => {
|
||||
ran.push({ action, parameters });
|
||||
return { viewport: { mode: 'mobile', width: 390, height: 844 } };
|
||||
},
|
||||
}));
|
||||
}, 120);
|
||||
}));
|
||||
|
||||
emitOpen({ url: 'https://example.test', viewport: 'mobile' });
|
||||
await wait(400);
|
||||
|
||||
expect(opened).toEqual(['https://example.test']);
|
||||
expect(ran).toEqual([{ action: 'browser.resize', parameters: { viewport: 'mobile' } }]);
|
||||
expect(posted[0]?.data).toEqual({
|
||||
url: 'https://example.test',
|
||||
opened: true,
|
||||
viewportApplied: true,
|
||||
viewport: { mode: 'mobile', width: 390, height: 844 },
|
||||
});
|
||||
});
|
||||
|
||||
test('does nothing at all when another client was granted the request', async () => {
|
||||
grantClaims = false;
|
||||
const opened: string[] = [];
|
||||
const ran: string[] = [];
|
||||
cleanups.push(registerBrowserOpener((url) => { opened.push(url); }));
|
||||
cleanups.push(registerBrowserController({
|
||||
run: async (action) => { ran.push(action); return {}; },
|
||||
}));
|
||||
|
||||
emitOpen({ url: 'https://example.test' });
|
||||
await wait(50);
|
||||
|
||||
expect(claims).toEqual(['req-1']);
|
||||
// The losing client must not act: a late result cannot undo a click.
|
||||
expect(ran).toEqual([]);
|
||||
expect(opened).toEqual([]);
|
||||
expect(posted).toEqual([]);
|
||||
});
|
||||
|
||||
test('claims the request before touching a page', async () => {
|
||||
const ran: string[] = [];
|
||||
cleanups.push(registerBrowserController({
|
||||
run: async (action) => { ran.push(action); return {}; },
|
||||
}));
|
||||
|
||||
listener?.({ type: 'browser-control-request', requestId: 'req-1', action: 'browser.click', parameters: { selector: 'button' } });
|
||||
await wait(50);
|
||||
|
||||
expect(claims).toEqual(['req-1']);
|
||||
expect(ran).toEqual(['browser.click']);
|
||||
});
|
||||
|
||||
test('does not wait for a view when no layout was requested', async () => {
|
||||
cleanups.push(registerBrowserOpener(() => {}));
|
||||
|
||||
emitOpen({ url: 'https://example.test' });
|
||||
await wait(20);
|
||||
|
||||
expect(posted[0]?.data).toEqual({ url: 'https://example.test', opened: true });
|
||||
});
|
||||
|
||||
test('says the layout was not applied when no view ever appears', async () => {
|
||||
cleanups.push(registerBrowserOpener(() => {}));
|
||||
|
||||
emitOpen({ url: 'https://example.test', viewport: 'mobile' });
|
||||
// Past the client's own attach deadline.
|
||||
await wait(2_400);
|
||||
|
||||
const data = posted[0]?.data as { viewportApplied?: boolean; note?: string };
|
||||
expect(data.viewportApplied).toBe(false);
|
||||
expect(typeof data.note).toBe('string');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,215 @@
|
||||
/**
|
||||
* Client half of agent browser control.
|
||||
*
|
||||
* The server broadcasts a browser request to every connected client, because it
|
||||
* cannot know which one is showing the browser panel. More than one may be able
|
||||
* to serve it, so a client asks the server for the request before doing
|
||||
* anything, and acts only if it is granted. Deciding by whose result arrives
|
||||
* first would be too late — by then every client has already clicked.
|
||||
*
|
||||
* `browser.open` is the exception: it is handled even with no view attached,
|
||||
* since opening a tab is precisely what creates one. The view it creates then
|
||||
* takes over the rest of that same request, so asking for a layout while
|
||||
* opening does not cost the agent a second call.
|
||||
*/
|
||||
import { runtimeFetch } from '@/lib/runtime-fetch';
|
||||
import { subscribeOpenchamberEvents } from '@/lib/openchamberEvents';
|
||||
|
||||
type BrowserControlRequest = {
|
||||
readonly requestId: string;
|
||||
readonly action: string;
|
||||
readonly parameters: Record<string, unknown>;
|
||||
};
|
||||
|
||||
/** Implemented by the mounted browser pane. */
|
||||
export type BrowserController = {
|
||||
/** Runs one action and resolves with its JSON-serializable result. */
|
||||
readonly run: (action: string, parameters: Record<string, unknown>) => Promise<unknown>;
|
||||
};
|
||||
|
||||
/** Opens a URL when no browser view exists yet. */
|
||||
export type BrowserOpener = (url: string) => void;
|
||||
|
||||
/**
|
||||
* How long a freshly opened tab is given to mount its view. A pane appears
|
||||
* within a frame or two; this is slack for a busy renderer, not a wait anyone
|
||||
* should ever notice.
|
||||
*/
|
||||
const VIEW_ATTACH_TIMEOUT_MS = 2_000;
|
||||
const VIEW_ATTACH_POLL_MS = 50;
|
||||
|
||||
let activeController: BrowserController | null = null;
|
||||
let opener: BrowserOpener | null = null;
|
||||
let unsubscribe: (() => void) | null = null;
|
||||
|
||||
/**
|
||||
* Delivers a result to the server.
|
||||
*
|
||||
* A dropped result is indistinguishable from an unreachable browser on the
|
||||
* agent's side, so a failure here is reported rather than swallowed — that
|
||||
* silence is what once turned a missing body parser into an unexplained
|
||||
* twenty-second timeout.
|
||||
*/
|
||||
/**
|
||||
* Asks for the exclusive right to perform a request.
|
||||
*
|
||||
* A refusal is the normal outcome for a client that lost the race, and so is a
|
||||
* failure to ask at all: acting without a grant is what this exists to prevent.
|
||||
*/
|
||||
const claimRequest = async (requestId: string): Promise<boolean> => {
|
||||
try {
|
||||
const response = await runtimeFetch('/api/browser-control/claim', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ requestId }),
|
||||
});
|
||||
if (!response.ok) return false;
|
||||
const body = await response.json() as { granted?: boolean };
|
||||
return body?.granted === true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
const postResult = async (requestId: string, outcome: { ok: boolean; data?: unknown; error?: string }): Promise<void> => {
|
||||
try {
|
||||
const response = await runtimeFetch('/api/browser-control/result', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ requestId, ...outcome }),
|
||||
});
|
||||
if (!response.ok) {
|
||||
console.warn(
|
||||
`[browser-control] the server rejected the result for ${requestId} (HTTP ${response.status}); `
|
||||
+ 'the agent will see this action time out',
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn(`[browser-control] could not deliver the result for ${requestId}:`, error);
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Waits for a browser view to register itself, or gives up.
|
||||
*
|
||||
* Polls rather than subscribes because registration is a plain assignment made
|
||||
* by whichever pane mounts; a callback would have to be maintained by every
|
||||
* caller of `registerBrowserController` for one waiter.
|
||||
*/
|
||||
const waitForController = async (
|
||||
timeoutMs = VIEW_ATTACH_TIMEOUT_MS,
|
||||
): Promise<BrowserController | null> => {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
while (!activeController && Date.now() < deadline) {
|
||||
await new Promise((resolve) => setTimeout(resolve, VIEW_ATTACH_POLL_MS));
|
||||
}
|
||||
return activeController;
|
||||
};
|
||||
|
||||
const handleRequest = async (request: BrowserControlRequest): Promise<void> => {
|
||||
const isOpen = request.action === 'browser.open';
|
||||
const controller = activeController;
|
||||
|
||||
if (!controller && !(isOpen && opener)) return;
|
||||
|
||||
// Nothing below this line may touch a page without the server's grant.
|
||||
if (!await claimRequest(request.requestId)) return;
|
||||
|
||||
try {
|
||||
if (isOpen && !controller) {
|
||||
const url = typeof request.parameters.url === 'string' ? request.parameters.url : '';
|
||||
if (!url) {
|
||||
await postResult(request.requestId, { ok: false, error: 'url is required' });
|
||||
return;
|
||||
}
|
||||
opener?.(url);
|
||||
|
||||
const requestedViewport = typeof request.parameters.viewport === 'string'
|
||||
? request.parameters.viewport
|
||||
: '';
|
||||
if (!requestedViewport || requestedViewport === 'fill') {
|
||||
await postResult(request.requestId, { ok: true, data: { url, opened: true } });
|
||||
return;
|
||||
}
|
||||
|
||||
// The tab was just created, so its view is a few frames away. Waiting for
|
||||
// it lets the layout the agent asked for be applied to the page it is
|
||||
// opening, rather than to the next call it has to make.
|
||||
const attached = await waitForController();
|
||||
if (!attached) {
|
||||
// Still no view. Reporting a plain success here would leave the agent
|
||||
// believing a size it asked for was applied to a page nobody is showing.
|
||||
await postResult(request.requestId, {
|
||||
ok: true,
|
||||
data: {
|
||||
url,
|
||||
opened: true,
|
||||
viewportApplied: false,
|
||||
note: 'The panel had no browser view yet, so the viewport was not applied. Call browser.resize now that one exists.',
|
||||
},
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
const resized = await attached.run('browser.resize', { viewport: requestedViewport });
|
||||
const viewport = resized && typeof resized === 'object'
|
||||
? (resized as { viewport?: unknown }).viewport ?? null
|
||||
: null;
|
||||
await postResult(request.requestId, {
|
||||
ok: true,
|
||||
data: { url, opened: true, viewportApplied: true, viewport },
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
const data = await controller!.run(request.action, request.parameters);
|
||||
await postResult(request.requestId, { ok: true, data });
|
||||
} catch (error) {
|
||||
await postResult(request.requestId, {
|
||||
ok: false,
|
||||
error: error instanceof Error ? error.message : String(error),
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
const ensureSubscribed = (): void => {
|
||||
if (unsubscribe) return;
|
||||
unsubscribe = subscribeOpenchamberEvents((event) => {
|
||||
if (event.type !== 'browser-control-request') return;
|
||||
void handleRequest({
|
||||
requestId: event.requestId,
|
||||
action: event.action,
|
||||
parameters: event.parameters,
|
||||
});
|
||||
});
|
||||
};
|
||||
|
||||
const releaseIfIdle = (): void => {
|
||||
if (activeController || opener || !unsubscribe) return;
|
||||
unsubscribe();
|
||||
unsubscribe = null;
|
||||
};
|
||||
|
||||
/**
|
||||
* Registers the mounted browser view. The most recently mounted view wins;
|
||||
* unregistering only clears the registry when it still points at the caller,
|
||||
* so a stale unmount cannot detach a newer view.
|
||||
*/
|
||||
export const registerBrowserController = (controller: BrowserController): (() => void) => {
|
||||
activeController = controller;
|
||||
ensureSubscribed();
|
||||
return () => {
|
||||
if (activeController === controller) activeController = null;
|
||||
releaseIfIdle();
|
||||
};
|
||||
};
|
||||
|
||||
/** Registers the app-level fallback that can open a browser tab on demand. */
|
||||
export const registerBrowserOpener = (open: BrowserOpener): (() => void) => {
|
||||
opener = open;
|
||||
ensureSubscribed();
|
||||
return () => {
|
||||
if (opener === open) opener = null;
|
||||
releaseIfIdle();
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,55 @@
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
|
||||
import {
|
||||
CRASH_RECOVERY_BASE_DELAY_MS,
|
||||
CRASH_RECOVERY_MAX_ATTEMPTS,
|
||||
CRASH_RECOVERY_WINDOW_MS,
|
||||
INITIAL_CRASH_RECOVERY_STATE,
|
||||
planCrashRecovery,
|
||||
} from './crashRecovery';
|
||||
|
||||
describe('crash recovery', () => {
|
||||
test('recovers from the first crash immediately enough to feel automatic', () => {
|
||||
const plan = planCrashRecovery(INITIAL_CRASH_RECOVERY_STATE, 1_000);
|
||||
expect(plan?.delayMs).toBe(CRASH_RECOVERY_BASE_DELAY_MS);
|
||||
expect(plan?.state).toEqual({ attempts: 1, windowStartedAt: 1_000 });
|
||||
});
|
||||
|
||||
test('waits longer after each attempt instead of reloading in a tight loop', () => {
|
||||
let state = INITIAL_CRASH_RECOVERY_STATE;
|
||||
const delays: number[] = [];
|
||||
for (let index = 0; index < CRASH_RECOVERY_MAX_ATTEMPTS; index += 1) {
|
||||
const plan = planCrashRecovery(state, 1_000 + index);
|
||||
expect(plan === null).toBe(false);
|
||||
delays.push(plan!.delayMs);
|
||||
state = plan!.state;
|
||||
}
|
||||
expect(delays).toEqual([250, 500, 1000]);
|
||||
});
|
||||
|
||||
test('gives up once the attempts in this window are spent', () => {
|
||||
let state = INITIAL_CRASH_RECOVERY_STATE;
|
||||
for (let index = 0; index < CRASH_RECOVERY_MAX_ATTEMPTS; index += 1) {
|
||||
state = planCrashRecovery(state, 1_000)!.state;
|
||||
}
|
||||
expect(planCrashRecovery(state, 1_000)).toBeNull();
|
||||
});
|
||||
|
||||
test('a crash long after the last one starts over rather than staying given up', () => {
|
||||
let state = INITIAL_CRASH_RECOVERY_STATE;
|
||||
for (let index = 0; index < CRASH_RECOVERY_MAX_ATTEMPTS; index += 1) {
|
||||
state = planCrashRecovery(state, 1_000)!.state;
|
||||
}
|
||||
const later = 1_000 + CRASH_RECOVERY_WINDOW_MS;
|
||||
const plan = planCrashRecovery(state, later);
|
||||
expect(plan?.delayMs).toBe(CRASH_RECOVERY_BASE_DELAY_MS);
|
||||
expect(plan?.state).toEqual({ attempts: 1, windowStartedAt: later });
|
||||
});
|
||||
|
||||
test('crashes inside the window keep counting against the same window', () => {
|
||||
const first = planCrashRecovery(INITIAL_CRASH_RECOVERY_STATE, 1_000)!;
|
||||
const second = planCrashRecovery(first.state, 1_000 + CRASH_RECOVERY_WINDOW_MS - 1)!;
|
||||
expect(second.state.windowStartedAt).toBe(1_000);
|
||||
expect(second.state.attempts).toBe(2);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,66 @@
|
||||
/**
|
||||
* Recovery policy for a page whose renderer died.
|
||||
*
|
||||
* A `<webview>` runs the page in its own process, and that process can be lost
|
||||
* — out of memory, a hung tab killed by the system, a page that crashes itself.
|
||||
* Nothing else in the view's lifecycle reports it: no load fails and no
|
||||
* navigation happens, so without this the panel simply stays blank forever with
|
||||
* no way back other than closing the tab.
|
||||
*
|
||||
* Reloading is usually right, because most crashes are transient. Reloading
|
||||
* without limit is not: a page that crashes on load would be reloaded until the
|
||||
* machine gives up. So attempts are bounded within a window, and once they run
|
||||
* out the panel says so instead of trying again.
|
||||
*
|
||||
* The policy is a pure function of the previous state and the current time so
|
||||
* it can be reasoned about and tested without crashing a real renderer.
|
||||
*/
|
||||
|
||||
export const CRASH_RECOVERY_WINDOW_MS = 30_000;
|
||||
export const CRASH_RECOVERY_MAX_ATTEMPTS = 3;
|
||||
export const CRASH_RECOVERY_BASE_DELAY_MS = 250;
|
||||
|
||||
export type CrashRecoveryState = {
|
||||
readonly attempts: number;
|
||||
/** When the current window opened, or null before the first crash. */
|
||||
readonly windowStartedAt: number | null;
|
||||
};
|
||||
|
||||
export const INITIAL_CRASH_RECOVERY_STATE: CrashRecoveryState = {
|
||||
attempts: 0,
|
||||
windowStartedAt: null,
|
||||
};
|
||||
|
||||
type CrashRecoveryPlan = {
|
||||
/** How long to wait before reloading. */
|
||||
readonly delayMs: number;
|
||||
readonly state: CrashRecoveryState;
|
||||
};
|
||||
|
||||
/**
|
||||
* Decides whether to reload after a crash, and how long to wait first.
|
||||
*
|
||||
* Returns null when the attempts in this window are spent — the caller should
|
||||
* then report the crash rather than retry. Each attempt waits longer than the
|
||||
* last, so a page that crashes immediately is not reloaded in a tight loop.
|
||||
*/
|
||||
export const planCrashRecovery = (
|
||||
state: CrashRecoveryState,
|
||||
now: number,
|
||||
): CrashRecoveryPlan | null => {
|
||||
// A crash long after the previous one is a new problem, not a continuation
|
||||
// of an old one; counting it against a stale window would refuse to recover
|
||||
// from the first crash of an otherwise healthy session.
|
||||
const startsNewWindow = state.windowStartedAt === null
|
||||
|| now - state.windowStartedAt >= CRASH_RECOVERY_WINDOW_MS;
|
||||
const attempts = startsNewWindow ? 0 : state.attempts;
|
||||
if (attempts >= CRASH_RECOVERY_MAX_ATTEMPTS) return null;
|
||||
|
||||
return {
|
||||
delayMs: CRASH_RECOVERY_BASE_DELAY_MS * 2 ** attempts,
|
||||
state: {
|
||||
attempts: attempts + 1,
|
||||
windowStartedAt: startsNewWindow ? now : state.windowStartedAt,
|
||||
},
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,65 @@
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
|
||||
import { mergeDevServerCandidates } from './devServers';
|
||||
|
||||
const discovered = [
|
||||
{ port: 3000, url: 'http://localhost:3000/', command: 'node' },
|
||||
{ port: 4321, url: 'http://localhost:4321/', command: 'node' },
|
||||
{ port: 4323, url: 'http://localhost:4323/', command: 'node' },
|
||||
];
|
||||
|
||||
describe('dev server candidates', () => {
|
||||
test('takes the announced address, which carries the base path', () => {
|
||||
const merged = mergeDevServerCandidates({
|
||||
announced: ['http://localhost:4323/__analytics'],
|
||||
discovered,
|
||||
});
|
||||
expect(merged.find((entry) => entry.port === 4323)?.url).toBe('http://localhost:4323/__analytics');
|
||||
});
|
||||
|
||||
test('keeps a listening server whose announcement was mangled', () => {
|
||||
// A terminal wrapping ".../localhost:3000" mid-port yields port 300, which
|
||||
// parses fine and points nowhere.
|
||||
const merged = mergeDevServerCandidates({
|
||||
announced: ['http://localhost:300'],
|
||||
discovered,
|
||||
});
|
||||
expect(merged.map((entry) => entry.port)).toContain(3000);
|
||||
expect(merged.map((entry) => entry.port)).not.toContain(300);
|
||||
});
|
||||
|
||||
test('drops an announced address with nothing listening behind it', () => {
|
||||
const merged = mergeDevServerCandidates({
|
||||
announced: ['http://localhost:9999/app'],
|
||||
discovered,
|
||||
});
|
||||
expect(merged.map((entry) => entry.port)).not.toContain(9999);
|
||||
});
|
||||
|
||||
test('lists servers that never announced themselves', () => {
|
||||
const merged = mergeDevServerCandidates({ announced: [], discovered });
|
||||
expect(merged.map((entry) => entry.port)).toEqual([3000, 4321, 4323]);
|
||||
expect(merged.every((entry) => entry.announced === false)).toBe(true);
|
||||
});
|
||||
|
||||
test('puts the servers this run announced first', () => {
|
||||
const merged = mergeDevServerCandidates({
|
||||
announced: ['http://localhost:4323/__analytics'],
|
||||
discovered,
|
||||
});
|
||||
expect(merged[0]?.port).toBe(4323);
|
||||
});
|
||||
|
||||
test('falls back to announcements when discovery is unavailable', () => {
|
||||
const merged = mergeDevServerCandidates({
|
||||
announced: ['http://localhost:4323/__analytics', 'http://localhost:3000'],
|
||||
discovered: null,
|
||||
});
|
||||
expect(merged.map((entry) => entry.port)).toEqual([3000, 4323]);
|
||||
});
|
||||
|
||||
test('is empty when neither source has anything', () => {
|
||||
expect(mergeDevServerCandidates({ announced: [], discovered: null })).toEqual([]);
|
||||
expect(mergeDevServerCandidates({ announced: [], discovered: [] })).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,138 @@
|
||||
/**
|
||||
* Client for dev-server discovery.
|
||||
*
|
||||
* The result is a tagged union rather than an array, because "no dev server is
|
||||
* running" and "we could not look" lead to different UI and must not collapse
|
||||
* into the same empty list.
|
||||
*/
|
||||
import { runtimeFetch } from '@/lib/runtime-fetch';
|
||||
|
||||
export type DiscoveredDevServer = {
|
||||
readonly port: number;
|
||||
readonly url: string;
|
||||
readonly command: string;
|
||||
};
|
||||
|
||||
export type DevServerDiscovery =
|
||||
| { readonly kind: 'loading' }
|
||||
| { readonly kind: 'ready'; readonly servers: ReadonlyArray<DiscoveredDevServer> }
|
||||
| { readonly kind: 'unavailable' };
|
||||
|
||||
const isDiscoveredServer = (value: unknown): value is DiscoveredDevServer => {
|
||||
if (!value || typeof value !== 'object') return false;
|
||||
const record = value as Record<string, unknown>;
|
||||
return typeof record.port === 'number'
|
||||
&& Number.isFinite(record.port)
|
||||
&& typeof record.url === 'string'
|
||||
&& record.url.length > 0
|
||||
&& typeof record.command === 'string';
|
||||
};
|
||||
|
||||
export const fetchDevServers = async (signal?: AbortSignal): Promise<DevServerDiscovery> => {
|
||||
try {
|
||||
const response = await runtimeFetch('/api/dev-servers', { signal });
|
||||
if (!response.ok) return { kind: 'unavailable' };
|
||||
|
||||
const body: unknown = await response.json();
|
||||
if (!body || typeof body !== 'object') return { kind: 'unavailable' };
|
||||
|
||||
const servers = (body as { servers?: unknown }).servers;
|
||||
if (!Array.isArray(servers)) return { kind: 'unavailable' };
|
||||
|
||||
return { kind: 'ready', servers: servers.filter(isDiscoveredServer) };
|
||||
} catch {
|
||||
return { kind: 'unavailable' };
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Asks the server for the HTTP status a loopback URL currently returns.
|
||||
*
|
||||
* A dev server fronted by a gateway answers requests before the app behind it
|
||||
* is up, returning a 5xx page. That is a successful load as far as the browser
|
||||
* is concerned, so it produces no navigation failure — the status is the only
|
||||
* honest signal, short of reading the page and guessing from its contents.
|
||||
*
|
||||
* Returns null when the status could not be established.
|
||||
*/
|
||||
export const probeLoopbackStatus = async (url: string): Promise<number | null> => {
|
||||
try {
|
||||
const response = await runtimeFetch('/api/system/probe-url', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ url }),
|
||||
});
|
||||
if (!response.ok) return null;
|
||||
const body: unknown = await response.json();
|
||||
if (!body || typeof body !== 'object') return null;
|
||||
const status = (body as { status?: unknown }).status;
|
||||
return typeof status === 'number' && Number.isFinite(status) ? status : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
export type DevServerCandidate = {
|
||||
readonly url: string;
|
||||
readonly port: number;
|
||||
/** Present when a server announced this address itself. */
|
||||
readonly announced: boolean;
|
||||
};
|
||||
|
||||
const portOf = (url: string): number | null => {
|
||||
try {
|
||||
const parsed = new URL(url);
|
||||
const port = Number.parseInt(parsed.port || (parsed.protocol === 'https:' ? '443' : '80'), 10);
|
||||
return Number.isInteger(port) && port > 0 ? port : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Combines what servers said with what is actually listening.
|
||||
*
|
||||
* Each source knows something the other cannot. An announcement carries the base
|
||||
* path an app is served under, which a socket cannot reveal. A listening port is
|
||||
* ground truth, which an announcement is not: terminals wrap long lines, and a
|
||||
* URL split mid-port reads as a perfectly plausible address on a port where
|
||||
* nothing is running.
|
||||
*
|
||||
* So discovery decides which servers exist and announcements supply their paths.
|
||||
* When discovery is unavailable the announcements stand on their own — offering
|
||||
* something unverified beats offering nothing.
|
||||
*/
|
||||
export const mergeDevServerCandidates = ({
|
||||
announced,
|
||||
discovered,
|
||||
}: {
|
||||
announced: ReadonlyArray<string>;
|
||||
discovered: ReadonlyArray<DiscoveredDevServer> | null;
|
||||
}): DevServerCandidate[] => {
|
||||
const announcedByPort = new Map<number, string>();
|
||||
for (const url of announced) {
|
||||
const port = portOf(url);
|
||||
if (port !== null && !announcedByPort.has(port)) announcedByPort.set(port, url);
|
||||
}
|
||||
|
||||
if (!discovered) {
|
||||
return [...announcedByPort.entries()]
|
||||
.map(([port, url]) => ({ url, port, announced: true }))
|
||||
.sort((left, right) => left.port - right.port);
|
||||
}
|
||||
|
||||
return discovered
|
||||
.map((server) => {
|
||||
const announcedUrl = announcedByPort.get(server.port);
|
||||
return {
|
||||
url: announcedUrl ?? server.url,
|
||||
port: server.port,
|
||||
announced: announcedUrl !== undefined,
|
||||
};
|
||||
})
|
||||
.sort((left, right) => {
|
||||
// Servers this run announced come first: they are the ones just started.
|
||||
if (left.announced !== right.announced) return left.announced ? -1 : 1;
|
||||
return left.port - right.port;
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,94 @@
|
||||
import { afterEach, beforeEach, describe, expect, mock, test } from 'bun:test';
|
||||
|
||||
let apiBaseUrl = 'https://remote.example.test';
|
||||
|
||||
let tunnelResult: unknown = { localPort: 52418, reused: false };
|
||||
mock.module('@/lib/desktopNative', () => ({
|
||||
invokeDesktopCommand: mock(async () => {
|
||||
if (tunnelResult instanceof Error) throw tunnelResult;
|
||||
return tunnelResult;
|
||||
}),
|
||||
}));
|
||||
mock.module('@/lib/runtime-auth', () => ({
|
||||
getRuntimeBearerTokenSync: () => 'token',
|
||||
getRuntimeExtraHeadersSync: () => ({}),
|
||||
}));
|
||||
mock.module('@/lib/runtime-switch', () => ({
|
||||
getRuntimeApiBaseUrl: () => apiBaseUrl,
|
||||
subscribeRuntimeEndpointChanged: () => () => {},
|
||||
}));
|
||||
|
||||
const {
|
||||
DevTunnelUnavailableError,
|
||||
resolveBrowsableUrl,
|
||||
shouldTunnelLoopbackUrl,
|
||||
toDisplayUrl,
|
||||
} = await import('./devTunnel');
|
||||
|
||||
const globalScope = globalThis as unknown as { window?: unknown };
|
||||
|
||||
const asDesktop = (value: boolean) => {
|
||||
globalScope.window = value
|
||||
? { __OPENCHAMBER_ELECTRON__: true, location: { href: 'http://127.0.0.1:3901/' } }
|
||||
: { location: { href: 'http://127.0.0.1:3901/' } };
|
||||
};
|
||||
|
||||
describe('loopback navigations against a remote instance', () => {
|
||||
beforeEach(() => {
|
||||
apiBaseUrl = 'https://remote.example.test';
|
||||
tunnelResult = { localPort: 52418, reused: false };
|
||||
asDesktop(true);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
delete globalScope.window;
|
||||
});
|
||||
|
||||
test('a page reached through a tunnel keeps its other ports on the host', () => {
|
||||
expect(shouldTunnelLoopbackUrl('http://localhost:4322/docs/')).toBe(true);
|
||||
});
|
||||
|
||||
test('a tunnel port is this machine on purpose and is left alone', async () => {
|
||||
const tunneled = await resolveBrowsableUrl('http://localhost:3000/');
|
||||
expect(tunneled).toBe('http://127.0.0.1:52418/');
|
||||
// Following a link inside the tunnelled page must not tunnel the tunnel.
|
||||
expect(shouldTunnelLoopbackUrl(tunneled)).toBe(false);
|
||||
// And the address bar still shows what was asked for.
|
||||
expect(toDisplayUrl(tunneled)).toBe('http://localhost:3000/');
|
||||
});
|
||||
|
||||
test('a public address is not loopback at all', () => {
|
||||
expect(shouldTunnelLoopbackUrl('https://openchamber.dev/docs/')).toBe(false);
|
||||
});
|
||||
|
||||
test('an implicit port is the port the scheme means, not nothing', () => {
|
||||
// http://localhost/ is port 80 on the host, and must be tunnelled like any
|
||||
// other. Reading it as 0 would send the view to this machine instead.
|
||||
expect(shouldTunnelLoopbackUrl('http://localhost/')).toBe(true);
|
||||
expect(shouldTunnelLoopbackUrl('https://localhost/')).toBe(true);
|
||||
});
|
||||
|
||||
test('a failed tunnel is reported, never answered by this machine', async () => {
|
||||
tunnelResult = new Error('discovery unavailable');
|
||||
let failed = false;
|
||||
try {
|
||||
// A port no earlier test opened: a successful tunnel is cached per target.
|
||||
await resolveBrowsableUrl('http://localhost:3100/');
|
||||
} catch (error) {
|
||||
failed = error instanceof DevTunnelUnavailableError;
|
||||
}
|
||||
// Falling back to the plain loopback URL would show whatever runs on that
|
||||
// port here, under the address of a server on another machine.
|
||||
expect(failed).toBe(true);
|
||||
});
|
||||
|
||||
test('a local instance resolves its own loopback correctly', () => {
|
||||
apiBaseUrl = 'http://127.0.0.1:3901';
|
||||
expect(shouldTunnelLoopbackUrl('http://localhost:4322/docs/')).toBe(false);
|
||||
});
|
||||
|
||||
test('nothing is tunneled outside the desktop shell', () => {
|
||||
asDesktop(false);
|
||||
expect(shouldTunnelLoopbackUrl('http://localhost:4322/docs/')).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,187 @@
|
||||
/**
|
||||
* Makes a remote dev server browsable from the desktop app.
|
||||
*
|
||||
* A URL like `http://localhost:5173` means "this machine" to whoever resolves
|
||||
* it. When OpenChamber is running on another host, that is the wrong machine:
|
||||
* the dev server is on the host, the browser is here. The desktop shell binds
|
||||
* an equivalent local port and pipes it to the host, so the same page loads
|
||||
* from a real local origin with nothing rewritten.
|
||||
*
|
||||
* Everywhere else — local runtime, web, mobile — the URL is already correct and
|
||||
* is returned untouched.
|
||||
*/
|
||||
import { invokeDesktopCommand } from '@/lib/desktopNative';
|
||||
import { getRuntimeBearerTokenSync, getRuntimeExtraHeadersSync } from '@/lib/runtime-auth';
|
||||
import { getRuntimeApiBaseUrl, subscribeRuntimeEndpointChanged } from '@/lib/runtime-switch';
|
||||
import { isLoopbackUrl } from './url';
|
||||
|
||||
type TunnelResult = { localPort: number; reused: boolean; url: string };
|
||||
|
||||
/** Keyed by `${baseUrl}|${port}`; the shell owns the real lifetime. */
|
||||
const localPortByTarget = new Map<string, number>();
|
||||
/** Reverse map, so a tunnel port never leaks into the address bar or storage. */
|
||||
const originByLocalPort = new Map<number, string>();
|
||||
|
||||
const isDesktopRuntime = (): boolean => (
|
||||
typeof window !== 'undefined' && Boolean(window.__OPENCHAMBER_ELECTRON__)
|
||||
);
|
||||
|
||||
/**
|
||||
* True when the app is talking to an OpenChamber on another machine. A local
|
||||
* runtime resolves loopback URLs correctly on its own and must not be tunneled,
|
||||
* which would only add a hop.
|
||||
*/
|
||||
const isRemoteRuntime = (baseUrl: string): boolean => {
|
||||
if (!baseUrl) return false;
|
||||
try {
|
||||
const parsed = new URL(baseUrl, typeof window !== 'undefined' ? window.location.href : undefined);
|
||||
const localOrigin = typeof window !== 'undefined' ? window.__OPENCHAMBER_LOCAL_ORIGIN__ : '';
|
||||
if (localOrigin && parsed.origin === localOrigin) return false;
|
||||
return !isLoopbackUrl(parsed.toString());
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* The port a loopback URL addresses, including the one it leaves implicit.
|
||||
* Both callers must agree on this: an omitted port is 80 or 443, not nothing.
|
||||
*/
|
||||
const loopbackPort = (url: string): number => {
|
||||
try {
|
||||
const parsed = new URL(url);
|
||||
const port = Number.parseInt(parsed.port || (parsed.protocol === 'https:' ? '443' : '80'), 10);
|
||||
return Number.isInteger(port) && port > 0 ? port : 0;
|
||||
} catch {
|
||||
return 0;
|
||||
}
|
||||
};
|
||||
|
||||
const rewriteToLocalPort = (url: string, localPort: number): string => {
|
||||
try {
|
||||
const parsed = new URL(url);
|
||||
parsed.protocol = 'http:';
|
||||
parsed.hostname = '127.0.0.1';
|
||||
parsed.port = String(localPort);
|
||||
return parsed.toString();
|
||||
} catch {
|
||||
return url;
|
||||
}
|
||||
};
|
||||
|
||||
/** Thrown when a remote dev server exists but could not be reached from here. */
|
||||
export class DevTunnelUnavailableError extends Error {
|
||||
constructor(message: string) {
|
||||
super(message);
|
||||
this.name = 'DevTunnelUnavailableError';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the URL the browser view should actually load.
|
||||
*
|
||||
* A failure to tunnel is reported rather than papered over. Loading the
|
||||
* original loopback URL instead would not be "the same outcome without this
|
||||
* mechanism": on a remote instance it changes which machine answers, so the
|
||||
* user would be shown whatever happens to run on that port here — possibly a
|
||||
* different application — under the address they asked for. The refusal is
|
||||
* often authoritative, too: discovery unavailable, port not offered,
|
||||
* authentication rejected. None of that should look like a page.
|
||||
*/
|
||||
export const resolveBrowsableUrl = async (url: string): Promise<string> => {
|
||||
if (!url || !isDesktopRuntime() || !isLoopbackUrl(url)) return url;
|
||||
|
||||
const baseUrl = getRuntimeApiBaseUrl();
|
||||
if (!isRemoteRuntime(baseUrl)) return url;
|
||||
|
||||
const port = loopbackPort(url);
|
||||
if (!port) return url;
|
||||
|
||||
const key = `${baseUrl}|${port}`;
|
||||
const cached = localPortByTarget.get(key);
|
||||
if (cached) {
|
||||
try {
|
||||
originByLocalPort.set(cached, new URL(url).origin);
|
||||
} catch {
|
||||
// Unparseable input never reaches here; nothing to record.
|
||||
}
|
||||
return rewriteToLocalPort(url, cached);
|
||||
}
|
||||
|
||||
try {
|
||||
const result = await invokeDesktopCommand<TunnelResult>('desktop_dev_tunnel_open', {
|
||||
baseUrl,
|
||||
port,
|
||||
clientToken: getRuntimeBearerTokenSync(),
|
||||
requestHeaders: getRuntimeExtraHeadersSync(),
|
||||
});
|
||||
if (!result || !Number.isInteger(result.localPort) || result.localPort <= 0) {
|
||||
throw new DevTunnelUnavailableError(url);
|
||||
}
|
||||
localPortByTarget.set(key, result.localPort);
|
||||
try {
|
||||
originByLocalPort.set(result.localPort, new URL(url).origin);
|
||||
} catch {
|
||||
// Unparseable input never reaches here; nothing to record.
|
||||
}
|
||||
return rewriteToLocalPort(url, result.localPort);
|
||||
} catch (error) {
|
||||
if (error instanceof DevTunnelUnavailableError) throw error;
|
||||
throw new DevTunnelUnavailableError(url);
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* True when a loopback URL belongs to the machine OpenChamber runs on rather
|
||||
* than to this one.
|
||||
*
|
||||
* A page served through a tunnel can send the browser to another local port —
|
||||
* a docs server behind a dev gateway, an API on its own port — and that
|
||||
* navigation happens inside the view, where nothing resolved it. Without this
|
||||
* the address would be looked for on the user's own machine, where it is either
|
||||
* nothing at all or, worse, a different application.
|
||||
*
|
||||
* A URL already pointing at a tunnel's local port is not retargeted; that one
|
||||
* is this machine, deliberately.
|
||||
*/
|
||||
export const shouldTunnelLoopbackUrl = (url: string): boolean => {
|
||||
if (!url || !isDesktopRuntime() || !isLoopbackUrl(url)) return false;
|
||||
if (!isRemoteRuntime(getRuntimeApiBaseUrl())) return false;
|
||||
const port = loopbackPort(url);
|
||||
return port > 0 && !originByLocalPort.has(port);
|
||||
};
|
||||
|
||||
/**
|
||||
* Maps a URL the view actually loaded back to the address the user asked for.
|
||||
*
|
||||
* Without this the tunnel's random local port would show up in the address bar
|
||||
* and, worse, be persisted as the tab's target — a port that means nothing
|
||||
* after a restart.
|
||||
*/
|
||||
export const toDisplayUrl = (url: string): string => {
|
||||
if (!url) return url;
|
||||
try {
|
||||
const parsed = new URL(url);
|
||||
if (parsed.hostname !== '127.0.0.1') return url;
|
||||
const origin = originByLocalPort.get(Number.parseInt(parsed.port || '0', 10));
|
||||
if (!origin) return url;
|
||||
return `${origin}${parsed.pathname}${parsed.search}${parsed.hash}`;
|
||||
} catch {
|
||||
return url;
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Forgets cached tunnels. Cache entries are keyed by runtime base URL, so a
|
||||
* switch does not make them wrong — but the shell's listeners belong to the
|
||||
* previous endpoint, and holding their ports would keep resolving URLs to a
|
||||
* host the user has left.
|
||||
*/
|
||||
const resetDevTunnelCache = (): void => {
|
||||
localPortByTarget.clear();
|
||||
originByLocalPort.clear();
|
||||
};
|
||||
|
||||
if (typeof window !== 'undefined') {
|
||||
subscribeRuntimeEndpointChanged(resetDevTunnelCache);
|
||||
}
|
||||
@@ -0,0 +1,104 @@
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
|
||||
import {
|
||||
MAX_HISTORY_ENTRIES,
|
||||
forgetVisit,
|
||||
historyUrl,
|
||||
recordVisit,
|
||||
suggestFromHistory,
|
||||
type BrowserHistoryEntry,
|
||||
} from './history';
|
||||
|
||||
const entry = (url: string, title: string, lastVisitedAt: number): BrowserHistoryEntry => (
|
||||
{ url, title, lastVisitedAt }
|
||||
);
|
||||
|
||||
describe('what is worth remembering', () => {
|
||||
test('accepts a typed address the way the panel opens it', () => {
|
||||
expect(historyUrl('localhost:3000')).toBe('http://localhost:3000/');
|
||||
});
|
||||
|
||||
test('refuses the resting state and documents with no address', () => {
|
||||
expect(historyUrl('about:blank')).toBe('');
|
||||
expect(historyUrl('data:text/html,<p>hi</p>')).toBe('');
|
||||
expect(historyUrl('')).toBe('');
|
||||
});
|
||||
|
||||
test('refuses an address too long to be one', () => {
|
||||
expect(historyUrl(`http://example.test/${'a'.repeat(3000)}`)).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('recording visits', () => {
|
||||
test('keeps places rather than events', () => {
|
||||
let entries = recordVisit([], { url: 'http://localhost:3000/', title: 'App', at: 1 });
|
||||
entries = recordVisit(entries, { url: 'http://localhost:5173/', title: 'Docs', at: 2 });
|
||||
entries = recordVisit(entries, { url: 'http://localhost:3000/', title: 'App', at: 3 });
|
||||
|
||||
expect(entries.map((item) => item.url)).toEqual(['http://localhost:3000/', 'http://localhost:5173/']);
|
||||
expect(entries[0]?.lastVisitedAt).toBe(3);
|
||||
});
|
||||
|
||||
test('a later visit with no title keeps the name already known', () => {
|
||||
let entries = recordVisit([], { url: 'http://localhost:3000/', title: 'App', at: 1 });
|
||||
entries = recordVisit(entries, { url: 'http://localhost:3000/', at: 2 });
|
||||
expect(entries[0]?.title).toBe('App');
|
||||
});
|
||||
|
||||
test('ignores a visit that is not a page', () => {
|
||||
const entries = recordVisit([], { url: 'about:blank', at: 1 });
|
||||
expect(entries).toEqual([]);
|
||||
});
|
||||
|
||||
test('drops the oldest rather than growing without end', () => {
|
||||
let entries: BrowserHistoryEntry[] = [];
|
||||
for (let index = 0; index < MAX_HISTORY_ENTRIES + 10; index += 1) {
|
||||
entries = recordVisit(entries, { url: `http://example.test/${index}`, at: index });
|
||||
}
|
||||
expect(entries).toHaveLength(MAX_HISTORY_ENTRIES);
|
||||
expect(entries[0]?.url).toBe(`http://example.test/${MAX_HISTORY_ENTRIES + 9}`);
|
||||
});
|
||||
|
||||
test('forgets one address without touching the rest', () => {
|
||||
const entries = [entry('http://a.test/', 'A', 2), entry('http://b.test/', 'B', 1)];
|
||||
expect(forgetVisit(entries, 'http://a.test').map((item) => item.url)).toEqual(['http://b.test/']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('suggestions', () => {
|
||||
const entries = [
|
||||
entry('http://localhost:3000/', 'Storefront', 1),
|
||||
entry('http://localhost:5173/docs', 'Docs', 3),
|
||||
entry('https://staging.example.test/', 'Staging', 2),
|
||||
];
|
||||
|
||||
test('an empty address bar offers the most recent places', () => {
|
||||
expect(suggestFromHistory(entries, '').map((item) => item.url)).toEqual([
|
||||
'http://localhost:5173/docs',
|
||||
'https://staging.example.test/',
|
||||
'http://localhost:3000/',
|
||||
]);
|
||||
});
|
||||
|
||||
test('matches a port or a fragment of the path, not just a prefix', () => {
|
||||
expect(suggestFromHistory(entries, '5173').map((item) => item.url)).toEqual(['http://localhost:5173/docs']);
|
||||
expect(suggestFromHistory(entries, 'docs').map((item) => item.url)).toEqual(['http://localhost:5173/docs']);
|
||||
});
|
||||
|
||||
test('matches the page title as well as the address', () => {
|
||||
expect(suggestFromHistory(entries, 'storefront').map((item) => item.url)).toEqual(['http://localhost:3000/']);
|
||||
});
|
||||
|
||||
test('ignores the scheme the user did or did not type', () => {
|
||||
expect(suggestFromHistory(entries, 'http://staging').map((item) => item.url))
|
||||
.toEqual(['https://staging.example.test/']);
|
||||
});
|
||||
|
||||
test('does not offer back the address already typed in full', () => {
|
||||
expect(suggestFromHistory(entries, 'http://localhost:3000/')).toEqual([]);
|
||||
});
|
||||
|
||||
test('never offers more than it was asked for', () => {
|
||||
expect(suggestFromHistory(entries, '', 2)).toHaveLength(2);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,105 @@
|
||||
/**
|
||||
* Address history for the browser panel.
|
||||
*
|
||||
* The panel is used to return to the same handful of addresses — a dev server,
|
||||
* a staging URL, one page of the app being worked on — so typing them out every
|
||||
* time is the wrong default. History is per project, because the addresses that
|
||||
* matter belong to whatever is being worked on, not to the app as a whole.
|
||||
*
|
||||
* The ranking and matching live here as pure functions so the behaviour can be
|
||||
* reasoned about without a store, a list, or a keyboard in the way.
|
||||
*/
|
||||
import { normalizeBrowserUrl } from './url';
|
||||
|
||||
export type BrowserHistoryEntry = {
|
||||
readonly url: string;
|
||||
readonly title: string;
|
||||
readonly lastVisitedAt: number;
|
||||
};
|
||||
|
||||
/** Enough to cover a project's real addresses without becoming a list nobody reads. */
|
||||
export const MAX_HISTORY_ENTRIES = 50;
|
||||
const MAX_HISTORY_SUGGESTIONS = 6;
|
||||
const MAX_URL_LENGTH = 2048;
|
||||
const MAX_TITLE_LENGTH = 200;
|
||||
|
||||
/**
|
||||
* The stored form of an address.
|
||||
*
|
||||
* Returns '' for anything that is not a page worth remembering. `about:blank`
|
||||
* is the view's resting state, and a data URL is a document with no address to
|
||||
* return to.
|
||||
*/
|
||||
export const historyUrl = (value: string): string => {
|
||||
const normalized = normalizeBrowserUrl(String(value ?? '').trim());
|
||||
if (!normalized || normalized.length > MAX_URL_LENGTH) return '';
|
||||
try {
|
||||
const parsed = new URL(normalized);
|
||||
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') return '';
|
||||
return parsed.toString();
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Records a visit, newest first.
|
||||
*
|
||||
* Revisiting an address moves it up rather than adding a second copy — the list
|
||||
* is places, not events. A visit with no title keeps the title already known,
|
||||
* because a page often reports its address before it reports its name.
|
||||
*/
|
||||
export const recordVisit = (
|
||||
entries: readonly BrowserHistoryEntry[],
|
||||
visit: { url: string; title?: string; at: number },
|
||||
): BrowserHistoryEntry[] => {
|
||||
const url = historyUrl(visit.url);
|
||||
if (!url) return entries as BrowserHistoryEntry[];
|
||||
|
||||
const previous = entries.find((entry) => entry.url === url);
|
||||
const title = String(visit.title ?? '').trim().slice(0, MAX_TITLE_LENGTH) || previous?.title || '';
|
||||
const next: BrowserHistoryEntry = { url, title, lastVisitedAt: visit.at };
|
||||
|
||||
return [next, ...entries.filter((entry) => entry.url !== url)].slice(0, MAX_HISTORY_ENTRIES);
|
||||
};
|
||||
|
||||
export const forgetVisit = (
|
||||
entries: readonly BrowserHistoryEntry[],
|
||||
url: string,
|
||||
): BrowserHistoryEntry[] => {
|
||||
const target = historyUrl(url);
|
||||
return entries.filter((entry) => entry.url !== target);
|
||||
};
|
||||
|
||||
/** What a query matches against: the address without its scheme, plus the title. */
|
||||
const searchableText = (entry: BrowserHistoryEntry): string => (
|
||||
`${entry.url.replace(/^https?:\/\//, '')} ${entry.title}`.toLowerCase()
|
||||
);
|
||||
|
||||
/**
|
||||
* Suggests addresses for what has been typed so far.
|
||||
*
|
||||
* An empty query offers the most recent addresses, which is what an empty
|
||||
* address bar is asking for. A query matches anywhere in the address or title,
|
||||
* since a port or a path fragment is often all anyone remembers.
|
||||
*/
|
||||
export const suggestFromHistory = (
|
||||
entries: readonly BrowserHistoryEntry[],
|
||||
query: string,
|
||||
limit = MAX_HISTORY_SUGGESTIONS,
|
||||
): BrowserHistoryEntry[] => {
|
||||
const needle = String(query ?? '').trim().toLowerCase();
|
||||
const ordered = [...entries].sort((a, b) => b.lastVisitedAt - a.lastVisitedAt);
|
||||
if (!needle) return ordered.slice(0, limit);
|
||||
|
||||
const scheme = needle.replace(/^https?:\/\//, '');
|
||||
return ordered
|
||||
.filter((entry) => {
|
||||
const text = searchableText(entry);
|
||||
// An address the user has fully typed is not a suggestion, it is what
|
||||
// they already have; offering it back is noise.
|
||||
if (entry.url === needle || text === scheme) return false;
|
||||
return text.includes(scheme);
|
||||
})
|
||||
.slice(0, limit);
|
||||
};
|
||||
@@ -0,0 +1,94 @@
|
||||
/**
|
||||
* Resolves OpenChamber theme tokens into concrete color strings.
|
||||
*
|
||||
* The annotation overlay renders inside a page we do not control, so it cannot
|
||||
* reference our CSS variables — that page has its own `:root`. Theme tokens can
|
||||
* also be authored in any color space, so string-concatenating an alpha suffix
|
||||
* onto them is not safe. Both problems go away by letting the browser resolve
|
||||
* the values here: a probe element carries the token, and the computed style is
|
||||
* always a concrete color the overlay can use verbatim.
|
||||
*/
|
||||
import type { BrowserAnnotationOverlayTheme } from './annotationOverlay';
|
||||
|
||||
const FALLBACK: BrowserAnnotationOverlayTheme = {
|
||||
colorScheme: 'dark',
|
||||
primary: 'rgb(59, 130, 246)',
|
||||
primarySoft: 'rgba(59, 130, 246, 0.16)',
|
||||
primaryFaint: 'rgba(59, 130, 246, 0.10)',
|
||||
primaryContrast: 'rgb(255, 255, 255)',
|
||||
surface: 'rgb(24, 24, 27)',
|
||||
surfaceElevated: 'rgb(32, 32, 36)',
|
||||
glassSurface: 'rgba(32, 32, 36, 0.64)',
|
||||
glassFilter: 'blur(26px) saturate(1.16)',
|
||||
border: 'rgba(255, 255, 255, 0.14)',
|
||||
text: 'rgb(244, 244, 245)',
|
||||
mutedText: 'rgba(244, 244, 245, 0.62)',
|
||||
};
|
||||
|
||||
type Probe = {
|
||||
readonly read: (value: string) => string;
|
||||
readonly readVariable: (name: string) => string;
|
||||
readonly dispose: () => void;
|
||||
};
|
||||
|
||||
const createProbe = (): Probe | null => {
|
||||
if (typeof document === 'undefined' || !document.body) return null;
|
||||
const element = document.createElement('div');
|
||||
element.setAttribute('aria-hidden', 'true');
|
||||
element.style.cssText = 'position:fixed;left:-9999px;top:-9999px;width:1px;height:1px;pointer-events:none';
|
||||
document.body.appendChild(element);
|
||||
return {
|
||||
read: (value: string): string => {
|
||||
element.style.backgroundColor = '';
|
||||
element.style.backgroundColor = value;
|
||||
const resolved = window.getComputedStyle(element).backgroundColor;
|
||||
return resolved && resolved !== 'rgba(0, 0, 0, 0)' ? resolved : '';
|
||||
},
|
||||
readVariable: (name: string): string => (
|
||||
window.getComputedStyle(document.documentElement).getPropertyValue(name).trim()
|
||||
),
|
||||
dispose: () => element.remove(),
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* Reads the live theme. Returns a usable palette even when probing fails, so a
|
||||
* missing token can never leave the overlay invisible against the page.
|
||||
*/
|
||||
export const resolveAnnotationOverlayTheme = (colorScheme: 'light' | 'dark'): BrowserAnnotationOverlayTheme => {
|
||||
const probe = createProbe();
|
||||
if (!probe) return { ...FALLBACK, colorScheme };
|
||||
|
||||
try {
|
||||
const read = (token: string, fallback: string): string => probe.read(`var(${token})`) || fallback;
|
||||
const mix = (token: string, percent: number, fallback: string): string => (
|
||||
probe.read(`color-mix(in srgb, var(${token}) ${percent}%, transparent)`) || fallback
|
||||
);
|
||||
|
||||
// The same recipe the app's tooltips and popovers use, resolved here
|
||||
// because the overlay cannot reach our stylesheet from inside the page.
|
||||
const glassOpacity = probe.readVariable('--oc-glass-tooltip-opacity') || '62%';
|
||||
const blur = probe.readVariable('--oc-glass-blur') || '26px';
|
||||
const saturation = probe.readVariable('--oc-glass-saturation') || '1.16';
|
||||
|
||||
return {
|
||||
colorScheme,
|
||||
glassSurface: probe.read(`color-mix(in srgb, var(--surface-elevated) ${glassOpacity}, transparent)`)
|
||||
|| FALLBACK.glassSurface,
|
||||
glassFilter: `blur(${blur}) saturate(${saturation})`,
|
||||
primary: read('--primary', FALLBACK.primary),
|
||||
primarySoft: mix('--primary', 16, FALLBACK.primarySoft),
|
||||
primaryFaint: mix('--primary', 10, FALLBACK.primaryFaint),
|
||||
primaryContrast: read('--primary-foreground', FALLBACK.primaryContrast),
|
||||
surface: read('--surface-background', FALLBACK.surface),
|
||||
surfaceElevated: read('--surface-elevated', FALLBACK.surfaceElevated),
|
||||
border: read('--border', FALLBACK.border),
|
||||
text: read('--foreground', FALLBACK.text),
|
||||
mutedText: read('--muted-foreground', FALLBACK.mutedText),
|
||||
};
|
||||
} catch {
|
||||
return { ...FALLBACK, colorScheme };
|
||||
} finally {
|
||||
probe.dispose();
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,70 @@
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
|
||||
import {
|
||||
buildClickScript,
|
||||
buildInspectScript,
|
||||
buildScrollScript,
|
||||
buildSnapshotScript,
|
||||
buildTypeScript,
|
||||
} from './pageActions';
|
||||
|
||||
/**
|
||||
* These scripts are source text evaluated inside another page: nothing
|
||||
* type-checks them, and a value interpolated without escaping either breaks the
|
||||
* script or runs as code.
|
||||
*/
|
||||
const parses = (source: string): boolean => {
|
||||
try {
|
||||
new Function(source);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
describe('page action scripts', () => {
|
||||
test('every script parses', () => {
|
||||
expect(parses(buildSnapshotScript())).toBe(true);
|
||||
expect(parses(buildSnapshotScript({ selector: '#main' }))).toBe(true);
|
||||
expect(parses(buildClickScript({ selector: '#save' }))).toBe(true);
|
||||
expect(parses(buildClickScript({ text: 'Save' }))).toBe(true);
|
||||
expect(parses(buildTypeScript({ selector: '#q', value: 'hello', submit: true }))).toBe(true);
|
||||
expect(parses(buildInspectScript({ selector: '#save' }))).toBe(true);
|
||||
expect(parses(buildScrollScript({ direction: 'bottom' }))).toBe(true);
|
||||
expect(parses(buildScrollScript({ selector: 'footer' }))).toBe(true);
|
||||
});
|
||||
|
||||
test('a hostile selector is embedded as data, not as code', () => {
|
||||
const hostile = `'); window.__owned = true; ('`;
|
||||
const script = buildClickScript({ selector: hostile });
|
||||
expect(parses(script)).toBe(true);
|
||||
// Present only inside a quoted literal: that is what makes it inert.
|
||||
expect(script).toContain(JSON.stringify(hostile));
|
||||
});
|
||||
|
||||
test('a typed value is embedded as data too', () => {
|
||||
const value = '"); alert(1); ("';
|
||||
const script = buildTypeScript({ selector: '#q', value, submit: false });
|
||||
expect(parses(script)).toBe(true);
|
||||
expect(script).toContain(JSON.stringify(value));
|
||||
});
|
||||
|
||||
test('whitespace regexes survive interpolation', () => {
|
||||
// A doubled backslash here would produce a literal backslash-s and match
|
||||
// nothing, silently collapsing no whitespace at all.
|
||||
expect(buildSnapshotScript()).toContain('replace(/\\s+/g');
|
||||
});
|
||||
|
||||
test('scrolling asks for instant behaviour, not the page preference', () => {
|
||||
// A page with scroll-behavior: smooth would otherwise still be animating
|
||||
// when the position is read.
|
||||
expect(buildScrollScript({ direction: 'bottom' })).toContain("behavior: 'instant'");
|
||||
expect(buildScrollScript({ selector: 'footer' })).toContain("behavior: 'instant'");
|
||||
});
|
||||
|
||||
test('a scoped snapshot reads only the subtree it was given', () => {
|
||||
const script = buildSnapshotScript({ selector: '#changelog' });
|
||||
expect(script).toContain('"#changelog"');
|
||||
expect(script).toContain('root.querySelectorAll');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,379 @@
|
||||
/**
|
||||
* Scripts the agent's browser actions run inside the page.
|
||||
*
|
||||
* Each builder returns a self-contained expression evaluated in the page's own
|
||||
* context, so none of them may reference anything from this module at runtime.
|
||||
* Inputs are embedded with `JSON.stringify`, which is what keeps a selector or
|
||||
* a typed value from terminating the expression and becoming code.
|
||||
*
|
||||
* Every script resolves to `{ ok, ... }` instead of throwing, so a failed match
|
||||
* comes back as an explainable result rather than an opaque evaluation error.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Budget caps. The snapshot cost is bounded by these, not by the size of the
|
||||
* page: a document with ten thousand nodes returns the same shape as one with
|
||||
* two hundred, because only visible interactive elements are collected and both
|
||||
* lists are cut off here. What the caps drop is always reported, so a partial
|
||||
* answer never reads as a complete one.
|
||||
*/
|
||||
const MAX_TEXT_CHARS = 6_000;
|
||||
const MAX_ELEMENTS = 120;
|
||||
/** Enough to recognise a control; full labels are what made entries expensive. */
|
||||
const MAX_LABEL_CHARS = 80;
|
||||
|
||||
/**
|
||||
* Shared helpers, injected into each script. `describe` builds the same kind of
|
||||
* selector the other actions accept, so a snapshot result is directly usable as
|
||||
* input to click or type.
|
||||
*/
|
||||
const HELPERS = `
|
||||
var MAX_ELEMENTS = ${MAX_ELEMENTS};
|
||||
var MAX_LABEL_CHARS = ${MAX_LABEL_CHARS};
|
||||
var visible = function (element) {
|
||||
var rect = element.getBoundingClientRect();
|
||||
if (rect.width < 1 || rect.height < 1) return false;
|
||||
var style = window.getComputedStyle(element);
|
||||
return style.visibility !== 'hidden' && style.display !== 'none' && Number(style.opacity) !== 0;
|
||||
};
|
||||
var label = function (element) {
|
||||
var aria = element.getAttribute('aria-label');
|
||||
if (aria) return aria.trim();
|
||||
var value = element.getAttribute('value');
|
||||
var text = (element.innerText || element.textContent || '').replace(/\\s+/g, ' ').trim();
|
||||
if (text) return text.slice(0, MAX_LABEL_CHARS);
|
||||
if (value) return String(value).slice(0, MAX_LABEL_CHARS);
|
||||
var placeholder = element.getAttribute('placeholder');
|
||||
return placeholder ? placeholder.trim().slice(0, MAX_LABEL_CHARS) : '';
|
||||
};
|
||||
var isUnique = function (selector) {
|
||||
try {
|
||||
return document.querySelectorAll(selector).length === 1;
|
||||
} catch (error) {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Names an element by what it is, falling back to where it sits.
|
||||
*
|
||||
* A positional chain like 'main > section:nth-of-type(3) > div > a' survives
|
||||
* only until the markup shifts, and says nothing about what it points at.
|
||||
* Anything the page states about identity — an id, a test id, an accessible
|
||||
* name — outlives edits and reads as the thing it selects. The chain remains
|
||||
* as the last resort, because something always has to work.
|
||||
*/
|
||||
var cssPath = function (element) {
|
||||
var tag = element.tagName.toLowerCase();
|
||||
|
||||
if (element.id) {
|
||||
var byId = '#' + CSS.escape(element.id);
|
||||
if (isUnique(byId)) return byId;
|
||||
}
|
||||
var stableAttrs = ['data-testid', 'data-test-id', 'data-test', 'name', 'aria-label'];
|
||||
for (var a = 0; a < stableAttrs.length; a += 1) {
|
||||
var value = element.getAttribute(stableAttrs[a]);
|
||||
if (!value) continue;
|
||||
var raw = String(value);
|
||||
// A value containing a quote would need escaping for no real gain: such
|
||||
// attributes are rare, and the positional chain still covers them.
|
||||
if (raw.indexOf('"') !== -1) continue;
|
||||
var byAttr = tag + '[' + stableAttrs[a] + '="' + raw + '"]';
|
||||
if (isUnique(byAttr)) return byAttr;
|
||||
}
|
||||
var className = typeof element.className === 'string' ? element.className.trim() : '';
|
||||
if (className) {
|
||||
var classes = className.split(/\\s+/).filter(Boolean);
|
||||
for (var c = 0; c < classes.length; c += 1) {
|
||||
var byClass = tag + '.' + CSS.escape(classes[c]);
|
||||
if (isUnique(byClass)) return byClass;
|
||||
}
|
||||
}
|
||||
|
||||
var parts = [];
|
||||
var node = element;
|
||||
var depth = 0;
|
||||
while (node && node.nodeType === 1 && depth < 6) {
|
||||
var part = node.tagName.toLowerCase();
|
||||
var parent = node.parentElement;
|
||||
if (!parent) { parts.unshift(part); break; }
|
||||
var siblings = Array.prototype.filter.call(parent.children, function (child) {
|
||||
return child.tagName === node.tagName;
|
||||
});
|
||||
if (siblings.length > 1) part += ':nth-of-type(' + (siblings.indexOf(node) + 1) + ')';
|
||||
parts.unshift(part);
|
||||
if (node.id) { parts[0] = '#' + CSS.escape(node.id); break; }
|
||||
node = parent;
|
||||
depth += 1;
|
||||
}
|
||||
return parts.join(' > ');
|
||||
};
|
||||
|
||||
/** What a screen reader would announce, or '' when there is nothing to say. */
|
||||
var accessibleName = function (element) {
|
||||
var aria = element.getAttribute('aria-label');
|
||||
if (aria && aria.trim()) return aria.trim();
|
||||
var labelled = element.getAttribute('aria-labelledby');
|
||||
if (labelled) {
|
||||
var source = document.getElementById(labelled.split(/\\s+/)[0]);
|
||||
if (source && (source.innerText || '').trim()) return source.innerText.trim();
|
||||
}
|
||||
var title = element.getAttribute('title');
|
||||
if (title && title.trim()) return title.trim();
|
||||
var alt = element.getAttribute('alt');
|
||||
if (alt && alt.trim()) return alt.trim();
|
||||
var text = (element.innerText || element.textContent || '').replace(/\\s+/g, ' ').trim();
|
||||
if (text) return text;
|
||||
var value = element.getAttribute('value');
|
||||
return value && String(value).trim() ? String(value).trim() : '';
|
||||
};
|
||||
var findByText = function (needle) {
|
||||
var wanted = String(needle).replace(/\\s+/g, ' ').trim().toLowerCase();
|
||||
var candidates = document.querySelectorAll('a, button, [role="button"], [role="link"], input[type="submit"], input[type="button"], summary, label');
|
||||
var exact = null;
|
||||
var partial = null;
|
||||
for (var i = 0; i < candidates.length; i += 1) {
|
||||
var element = candidates[i];
|
||||
if (!visible(element)) continue;
|
||||
var text = label(element).toLowerCase();
|
||||
if (!text) continue;
|
||||
if (text === wanted) { exact = element; break; }
|
||||
if (!partial && text.indexOf(wanted) !== -1) partial = element;
|
||||
}
|
||||
return exact || partial;
|
||||
};
|
||||
`;
|
||||
|
||||
const wrap = (body: string): string => `(() => {\n${HELPERS}\n${body}\n})()`;
|
||||
|
||||
/**
|
||||
* `selector` narrows the snapshot to one subtree.
|
||||
*
|
||||
* A long page truncates against the caps no matter how they are tuned, which
|
||||
* leaves the agent hunting. Scoping answers that directly: ask about the part
|
||||
* you mean and the caps stop mattering.
|
||||
*/
|
||||
export const buildSnapshotScript = ({ selector }: { selector?: string } = {}): string => wrap(`
|
||||
var scopeSelector = ${JSON.stringify(selector ?? '')};
|
||||
var root = document;
|
||||
if (scopeSelector) {
|
||||
try { root = document.querySelector(scopeSelector); }
|
||||
catch (error) { return { ok: false, error: 'Invalid selector: ' + scopeSelector }; }
|
||||
if (!root) return { ok: false, error: 'No element matches ' + scopeSelector };
|
||||
}
|
||||
var interactive = root.querySelectorAll('a[href], button, input, select, textarea, [role="button"], [role="link"], [role="tab"], [contenteditable="true"]');
|
||||
var elements = [];
|
||||
var visibleTotal = 0;
|
||||
for (var i = 0; i < interactive.length; i += 1) {
|
||||
var element = interactive[i];
|
||||
if (!visible(element)) continue;
|
||||
visibleTotal += 1;
|
||||
if (elements.length >= MAX_ELEMENTS) continue;
|
||||
|
||||
var rect = element.getBoundingClientRect();
|
||||
// Empty and default-valued fields are left out rather than serialized as
|
||||
// "" and false. Repeated across a hundred entries that overhead dwarfed
|
||||
// the information it carried.
|
||||
var entry = {
|
||||
selector: cssPath(element),
|
||||
tag: element.tagName.toLowerCase(),
|
||||
bounds: { x: Math.round(rect.x), y: Math.round(rect.y), width: Math.round(rect.width), height: Math.round(rect.height) }
|
||||
};
|
||||
// The list covers the whole document so anything can be clicked without
|
||||
// scrolling to it first, which means bounds alone do not say what is on
|
||||
// screen — a negative y reads as a bug otherwise.
|
||||
if (rect.bottom > 0 && rect.top < window.innerHeight) entry.inViewport = true;
|
||||
var type = element.getAttribute('type');
|
||||
if (type) entry.type = type;
|
||||
var role = element.getAttribute('role');
|
||||
if (role) entry.role = role;
|
||||
var labelText = label(element);
|
||||
if (labelText) entry.label = labelText;
|
||||
if (element.disabled === true) entry.disabled = true;
|
||||
// Flagged rather than described: reporting the accessible name of every
|
||||
// element would cost more than it tells, while its absence on something
|
||||
// clickable is a defect worth naming.
|
||||
if (!accessibleName(element)) entry.missingAccessibleName = true;
|
||||
elements.push(entry);
|
||||
}
|
||||
var body = document.body ? (document.body.innerText || '') : '';
|
||||
var text = body.replace(/\\n{3,}/g, '\\n\\n').trim();
|
||||
var docEl = document.documentElement;
|
||||
var result = {
|
||||
ok: true,
|
||||
url: String(location.href),
|
||||
title: String(document.title || ''),
|
||||
scope: scopeSelector || 'document',
|
||||
scrollY: Math.round(window.scrollY),
|
||||
maxScrollY: Math.max(0, Math.round(docEl.scrollHeight - window.innerHeight)),
|
||||
text: text.slice(0, ${MAX_TEXT_CHARS}),
|
||||
elements: elements
|
||||
};
|
||||
// State what was dropped. A capped list that reports only its own length
|
||||
// reads as the whole page, and the agent acts as if it had seen everything.
|
||||
if (text.length > ${MAX_TEXT_CHARS}) {
|
||||
result.textTruncated = true;
|
||||
result.textTotalChars = text.length;
|
||||
}
|
||||
if (visibleTotal > elements.length) {
|
||||
result.elementsTruncated = true;
|
||||
result.interactiveElementsOnPage = visibleTotal;
|
||||
}
|
||||
return result;
|
||||
`);
|
||||
|
||||
export const buildClickScript = ({ selector, text }: { selector?: string; text?: string }): string => wrap(`
|
||||
var selector = ${JSON.stringify(selector ?? '')};
|
||||
var text = ${JSON.stringify(text ?? '')};
|
||||
var target = null;
|
||||
if (selector) {
|
||||
try { target = document.querySelector(selector); }
|
||||
catch (error) { return { ok: false, error: 'Invalid selector: ' + selector }; }
|
||||
if (!target) return { ok: false, error: 'No element matches ' + selector };
|
||||
} else {
|
||||
target = findByText(text);
|
||||
if (!target) return { ok: false, error: 'No clickable element has the label ' + text };
|
||||
}
|
||||
if (target.disabled === true) return { ok: false, error: 'Element is disabled' };
|
||||
target.scrollIntoView({ block: 'center', inline: 'center' });
|
||||
target.click();
|
||||
return { ok: true, clicked: cssPath(target), label: label(target), url: String(location.href) };
|
||||
`);
|
||||
|
||||
export const buildTypeScript = ({ selector, value, submit }: { selector: string; value: string; submit: boolean }): string => wrap(`
|
||||
var selector = ${JSON.stringify(selector)};
|
||||
var value = ${JSON.stringify(value)};
|
||||
var target = null;
|
||||
try { target = document.querySelector(selector); }
|
||||
catch (error) { return { ok: false, error: 'Invalid selector: ' + selector }; }
|
||||
if (!target) return { ok: false, error: 'No element matches ' + selector };
|
||||
|
||||
var editable = target.tagName === 'INPUT' || target.tagName === 'TEXTAREA' || target.isContentEditable;
|
||||
if (!editable) return { ok: false, error: selector + ' is not a text field' };
|
||||
if (target.disabled === true || target.readOnly === true) return { ok: false, error: 'Field is not editable' };
|
||||
|
||||
target.scrollIntoView({ block: 'center' });
|
||||
target.focus();
|
||||
if (target.isContentEditable) {
|
||||
target.textContent = value;
|
||||
} else {
|
||||
// Frameworks track the value through the native setter; assigning the
|
||||
// property directly leaves React and friends unaware of the change.
|
||||
var prototype = target.tagName === 'TEXTAREA' ? HTMLTextAreaElement.prototype : HTMLInputElement.prototype;
|
||||
var setter = Object.getOwnPropertyDescriptor(prototype, 'value');
|
||||
if (setter && setter.set) setter.set.call(target, value);
|
||||
else target.value = value;
|
||||
}
|
||||
target.dispatchEvent(new Event('input', { bubbles: true }));
|
||||
target.dispatchEvent(new Event('change', { bubbles: true }));
|
||||
|
||||
if (${submit ? 'true' : 'false'}) {
|
||||
var enter = { key: 'Enter', code: 'Enter', keyCode: 13, which: 13, bubbles: true };
|
||||
target.dispatchEvent(new KeyboardEvent('keydown', enter));
|
||||
target.dispatchEvent(new KeyboardEvent('keyup', enter));
|
||||
var form = target.form;
|
||||
if (form && typeof form.requestSubmit === 'function') form.requestSubmit();
|
||||
}
|
||||
return { ok: true, selector: cssPath(target), url: String(location.href) };
|
||||
`);
|
||||
|
||||
/**
|
||||
* Scrolling, reported after it has actually happened.
|
||||
*
|
||||
* Two things made this lie. Pages commonly set `scroll-behavior: smooth`, which
|
||||
* turns a programmatic scroll into an animation — so the position read straight
|
||||
* afterwards is the position before the scroll, and the result said nothing
|
||||
* moved. And a scroll that is already at the end is indistinguishable from one
|
||||
* that failed unless the limits are reported too. An agent reading "scrollY: 0"
|
||||
* from a scroll that worked learns a superstition it will apply for the rest of
|
||||
* the session.
|
||||
*/
|
||||
export const buildScrollScript = ({ selector, direction }: { selector?: string; direction?: string }): string => wrap(`
|
||||
var selector = ${JSON.stringify(selector ?? '')};
|
||||
var direction = ${JSON.stringify(direction ?? '')};
|
||||
|
||||
var settle = function (extra) {
|
||||
return new Promise(function (resolve) {
|
||||
requestAnimationFrame(function () {
|
||||
requestAnimationFrame(function () {
|
||||
var doc = document.documentElement;
|
||||
var maxScrollY = Math.max(0, doc.scrollHeight - window.innerHeight);
|
||||
var scrollY = Math.round(window.scrollY);
|
||||
var result = { ok: true, scrollY: scrollY, maxScrollY: Math.round(maxScrollY) };
|
||||
result.atTop = scrollY <= 1;
|
||||
result.atBottom = scrollY >= maxScrollY - 1;
|
||||
for (var key in extra) {
|
||||
if (Object.prototype.hasOwnProperty.call(extra, key)) result[key] = extra[key];
|
||||
}
|
||||
resolve(result);
|
||||
});
|
||||
});
|
||||
});
|
||||
};
|
||||
|
||||
if (selector) {
|
||||
var target = null;
|
||||
try { target = document.querySelector(selector); }
|
||||
catch (error) { return { ok: false, error: 'Invalid selector: ' + selector }; }
|
||||
if (!target) return { ok: false, error: 'No element matches ' + selector };
|
||||
// Instant on purpose: the page's own smooth scrolling would still be
|
||||
// animating when the next action runs against it.
|
||||
target.scrollIntoView({ block: 'center', behavior: 'instant' });
|
||||
return settle({ scrolledTo: cssPath(target) });
|
||||
}
|
||||
|
||||
var doc = document.documentElement;
|
||||
var page = Math.round(window.innerHeight * 0.85);
|
||||
var bottom = Math.max(0, doc.scrollHeight - window.innerHeight);
|
||||
if (direction === 'down') window.scrollTo({ top: window.scrollY + page, behavior: 'instant' });
|
||||
else if (direction === 'up') window.scrollTo({ top: window.scrollY - page, behavior: 'instant' });
|
||||
else if (direction === 'top') window.scrollTo({ top: 0, behavior: 'instant' });
|
||||
else if (direction === 'bottom') window.scrollTo({ top: bottom, behavior: 'instant' });
|
||||
else return { ok: false, error: 'Unknown scroll direction: ' + direction };
|
||||
return settle({ direction: direction });
|
||||
`);
|
||||
|
||||
/** Properties that answer "how does this look", without dumping the whole cascade. */
|
||||
const INSPECTED_STYLE_PROPS = [
|
||||
'color', 'background-color', 'background-image', 'opacity',
|
||||
'font-family', 'font-size', 'font-weight', 'line-height', 'letter-spacing', 'text-align',
|
||||
'border-radius', 'border-width', 'border-style', 'border-color', 'box-shadow',
|
||||
'display', 'position', 'width', 'height', 'padding', 'margin', 'gap',
|
||||
'flex-direction', 'justify-content', 'align-items', 'z-index', 'overflow', 'visibility',
|
||||
];
|
||||
|
||||
/**
|
||||
* Reads how one element actually renders.
|
||||
*
|
||||
* The snapshot describes structure, which leaves questions of appearance
|
||||
* answerable only by reading the source and hoping the build agrees. Computed
|
||||
* styles come from the live page, so a colour reported from here is the colour
|
||||
* on screen — and unlike a screenshot it is readable by an agent that cannot
|
||||
* see images.
|
||||
*/
|
||||
export const buildInspectScript = ({ selector }: { selector: string }): string => wrap(`
|
||||
var selector = ${JSON.stringify(selector)};
|
||||
var target = null;
|
||||
try { target = document.querySelector(selector); }
|
||||
catch (error) { return { ok: false, error: 'Invalid selector: ' + selector }; }
|
||||
if (!target) return { ok: false, error: 'No element matches ' + selector };
|
||||
|
||||
var computed = window.getComputedStyle(target);
|
||||
var styles = {};
|
||||
var props = ${JSON.stringify(INSPECTED_STYLE_PROPS)};
|
||||
for (var i = 0; i < props.length; i += 1) {
|
||||
var value = computed.getPropertyValue(props[i]);
|
||||
if (value) styles[props[i]] = String(value).trim();
|
||||
}
|
||||
|
||||
var rect = target.getBoundingClientRect();
|
||||
return {
|
||||
ok: true,
|
||||
selector: cssPath(target),
|
||||
tag: target.tagName.toLowerCase(),
|
||||
label: label(target),
|
||||
bounds: { x: Math.round(rect.x), y: Math.round(rect.y), width: Math.round(rect.width), height: Math.round(rect.height) },
|
||||
inViewport: rect.bottom > 0 && rect.top < window.innerHeight,
|
||||
styles: styles
|
||||
};
|
||||
`);
|
||||
@@ -0,0 +1,80 @@
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
|
||||
import { BLANK_URL, browserUrlLabel, isLoopbackUrl, isStartingServerFailure, normalizeBrowserUrl } from './url';
|
||||
|
||||
describe('normalizeBrowserUrl', () => {
|
||||
test('keeps an explicit scheme', () => {
|
||||
expect(normalizeBrowserUrl('http://example.com/a')).toBe('http://example.com/a');
|
||||
expect(normalizeBrowserUrl('https://example.com/a')).toBe('https://example.com/a');
|
||||
});
|
||||
|
||||
test('defaults a public host to https', () => {
|
||||
expect(normalizeBrowserUrl('example.com')).toBe('https://example.com/');
|
||||
});
|
||||
|
||||
test('defaults a loopback authority to http, since dev servers speak plain http', () => {
|
||||
expect(normalizeBrowserUrl('localhost:5173')).toBe('http://localhost:5173/');
|
||||
expect(normalizeBrowserUrl('127.0.0.1:3000/app')).toBe('http://127.0.0.1:3000/app');
|
||||
expect(normalizeBrowserUrl('localhost')).toBe('http://localhost/');
|
||||
});
|
||||
|
||||
test('does not mistake a public host that merely starts with a loopback-looking label', () => {
|
||||
expect(normalizeBrowserUrl('localhost.example.com')).toBe('https://localhost.example.com/');
|
||||
});
|
||||
|
||||
test('rejects non-http schemes rather than handing them to the browser', () => {
|
||||
expect(normalizeBrowserUrl('file:///etc/passwd')).toBe(BLANK_URL);
|
||||
expect(normalizeBrowserUrl('javascript://alert(1)')).toBe(BLANK_URL);
|
||||
expect(normalizeBrowserUrl('data://text/html,x')).toBe(BLANK_URL);
|
||||
});
|
||||
|
||||
test('treats empty and unparseable input as blank', () => {
|
||||
expect(normalizeBrowserUrl('')).toBe(BLANK_URL);
|
||||
expect(normalizeBrowserUrl(' ')).toBe(BLANK_URL);
|
||||
expect(normalizeBrowserUrl('http://')).toBe(BLANK_URL);
|
||||
});
|
||||
});
|
||||
|
||||
describe('isLoopbackUrl', () => {
|
||||
test('recognizes loopback hosts', () => {
|
||||
expect(isLoopbackUrl('http://localhost:5173/')).toBe(true);
|
||||
expect(isLoopbackUrl('http://127.0.0.1/')).toBe(true);
|
||||
});
|
||||
|
||||
test('rejects remote hosts and garbage', () => {
|
||||
expect(isLoopbackUrl('https://example.com/')).toBe(false);
|
||||
expect(isLoopbackUrl('nonsense')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('browserUrlLabel', () => {
|
||||
test('shows host and port', () => {
|
||||
expect(browserUrlLabel('http://localhost:5173/a/b')).toBe('localhost:5173');
|
||||
});
|
||||
|
||||
test('is empty for a blank page', () => {
|
||||
expect(browserUrlLabel(BLANK_URL)).toBe('');
|
||||
expect(browserUrlLabel('')).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('isStartingServerFailure', () => {
|
||||
test('retries a loopback connection refusal, the usual "server not up yet"', () => {
|
||||
expect(isStartingServerFailure(-102, 'http://localhost:3000/')).toBe(true);
|
||||
expect(isStartingServerFailure(-104, 'http://127.0.0.1:5173/')).toBe(true);
|
||||
});
|
||||
|
||||
test('does not retry a public host that refused the connection', () => {
|
||||
expect(isStartingServerFailure(-102, 'https://example.com/')).toBe(false);
|
||||
});
|
||||
|
||||
test('does not retry a real page-level failure', () => {
|
||||
// ERR_ABORTED and certificate errors are not "not up yet".
|
||||
expect(isStartingServerFailure(-3, 'http://localhost:3000/')).toBe(false);
|
||||
expect(isStartingServerFailure(-201, 'http://localhost:3000/')).toBe(false);
|
||||
});
|
||||
|
||||
test('does not retry an unparseable url', () => {
|
||||
expect(isStartingServerFailure(-102, 'not-a-url')).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,88 @@
|
||||
/**
|
||||
* Address-bar input handling for the browser surface.
|
||||
*/
|
||||
|
||||
const LOOPBACK_HOSTNAMES = new Set(['localhost', '127.0.0.1', '0.0.0.0', '::1', '[::1]']);
|
||||
|
||||
/** `localhost:5173`, `127.0.0.1:3000` — an authority with no scheme. */
|
||||
const isLoopbackAuthority = (value: string): boolean => {
|
||||
const host = value.split('/')[0]?.split('?')[0] ?? '';
|
||||
const hostname = host.startsWith('[')
|
||||
? host.slice(0, host.indexOf(']') + 1)
|
||||
: host.split(':')[0] ?? '';
|
||||
return LOOPBACK_HOSTNAMES.has(hostname.toLowerCase());
|
||||
};
|
||||
|
||||
export const BLANK_URL = 'about:blank';
|
||||
|
||||
/**
|
||||
* Normalizes what the user typed into a URL the browser can load.
|
||||
*
|
||||
* Schemeless input defaults to `https:`, except for loopback authorities:
|
||||
* dev servers overwhelmingly speak plain HTTP, and defaulting `localhost:5173`
|
||||
* to HTTPS turns the single most common address in this panel into a
|
||||
* connection error.
|
||||
*/
|
||||
export const normalizeBrowserUrl = (value: string): string => {
|
||||
const trimmed = value.trim();
|
||||
if (!trimmed) return BLANK_URL;
|
||||
|
||||
const withScheme = trimmed.includes('://')
|
||||
? trimmed
|
||||
: `${isLoopbackAuthority(trimmed) ? 'http' : 'https'}://${trimmed}`;
|
||||
|
||||
try {
|
||||
const parsed = new URL(withScheme);
|
||||
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') return BLANK_URL;
|
||||
return parsed.toString();
|
||||
} catch {
|
||||
return BLANK_URL;
|
||||
}
|
||||
};
|
||||
|
||||
/** True when a URL points at the machine the page is loaded from. */
|
||||
export const isLoopbackUrl = (value: string): boolean => {
|
||||
try {
|
||||
return LOOPBACK_HOSTNAMES.has(new URL(value).hostname.toLowerCase());
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
/** Short label for a tab or header: host plus port, falling back to the raw value. */
|
||||
export const browserUrlLabel = (value: string): string => {
|
||||
if (!value || value === BLANK_URL) return '';
|
||||
try {
|
||||
return new URL(value).host || value;
|
||||
} catch {
|
||||
return value;
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Chromium network errors that mean "nothing is answering there yet".
|
||||
*
|
||||
* A dev server is routinely opened the moment its address appears in the
|
||||
* terminal, before it accepts connections, so the first load fails as a matter
|
||||
* of course. Treating that as a dead end — and making the user press reload —
|
||||
* is the single most common way this panel feels broken.
|
||||
*/
|
||||
const RETRYABLE_LOAD_ERROR_CODES = new Set([
|
||||
-2, // FAILED (generic; Electron reports this for some early refusals)
|
||||
-7, // TIMED_OUT
|
||||
-101, // CONNECTION_RESET
|
||||
-102, // CONNECTION_REFUSED
|
||||
-104, // CONNECTION_FAILED
|
||||
-109, // ADDRESS_UNREACHABLE
|
||||
-118, // CONNECTION_TIMED_OUT
|
||||
-324, // EMPTY_RESPONSE
|
||||
]);
|
||||
|
||||
/**
|
||||
* Whether a failed load is worth retrying. Restricted to loopback: retrying a
|
||||
* public site that refused us is just hammering somebody else's server, and a
|
||||
* remote dev server is reached through a local tunnel port anyway.
|
||||
*/
|
||||
export const isStartingServerFailure = (code: number, url: string): boolean => (
|
||||
RETRYABLE_LOAD_ERROR_CODES.has(code) && isLoopbackUrl(url)
|
||||
);
|
||||
@@ -0,0 +1,124 @@
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
|
||||
import {
|
||||
FILL_VIEWPORT,
|
||||
MAX_VIEWPORT_SIZE,
|
||||
MIN_VIEWPORT_SIZE,
|
||||
VIEWPORT_PRESETS,
|
||||
clampViewportSize,
|
||||
describeViewport,
|
||||
fitViewport,
|
||||
presetViewport,
|
||||
isViewportMode,
|
||||
rotateViewport,
|
||||
viewportForMode,
|
||||
viewportSize,
|
||||
viewportSummary,
|
||||
} from './viewport';
|
||||
|
||||
describe('viewport size', () => {
|
||||
test('fill has no size of its own', () => {
|
||||
expect(viewportSize(FILL_VIEWPORT)).toBeNull();
|
||||
expect(fitViewport(FILL_VIEWPORT, { width: 800, height: 600 })).toBeNull();
|
||||
});
|
||||
|
||||
test('clamps to a range a page can actually be laid out in', () => {
|
||||
expect(clampViewportSize(10)).toBe(MIN_VIEWPORT_SIZE);
|
||||
expect(clampViewportSize(99_999)).toBe(MAX_VIEWPORT_SIZE);
|
||||
expect(clampViewportSize(390.6)).toBe(391);
|
||||
expect(clampViewportSize(Number.NaN)).toBe(MIN_VIEWPORT_SIZE);
|
||||
});
|
||||
});
|
||||
|
||||
describe('presets', () => {
|
||||
test('resolves a known preset', () => {
|
||||
expect(presetViewport('iphone-14')).toEqual({ kind: 'preset', id: 'iphone-14', width: 390, height: 844 });
|
||||
});
|
||||
|
||||
test('returns nothing for an unknown id', () => {
|
||||
expect(presetViewport('nokia-3310')).toBeNull();
|
||||
});
|
||||
|
||||
test('every preset is within the layout range', () => {
|
||||
for (const preset of VIEWPORT_PRESETS) {
|
||||
expect(clampViewportSize(preset.width)).toBe(preset.width);
|
||||
expect(clampViewportSize(preset.height)).toBe(preset.height);
|
||||
}
|
||||
});
|
||||
|
||||
test('names the current preset and nothing else', () => {
|
||||
expect(describeViewport(presetViewport('ipad-mini')!)).toBe('iPad mini');
|
||||
expect(describeViewport({ kind: 'custom', width: 500, height: 500 })).toBe('');
|
||||
expect(describeViewport(FILL_VIEWPORT)).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('rotation', () => {
|
||||
test('swaps the sides', () => {
|
||||
expect(rotateViewport({ kind: 'custom', width: 390, height: 844 }))
|
||||
.toEqual({ kind: 'custom', width: 844, height: 390 });
|
||||
});
|
||||
|
||||
test('a rotated preset stops claiming to be that preset', () => {
|
||||
const rotated = rotateViewport(presetViewport('iphone-14')!);
|
||||
expect(rotated.kind).toBe('custom');
|
||||
expect(describeViewport(rotated)).toBe('');
|
||||
});
|
||||
|
||||
test('fill has no orientation', () => {
|
||||
expect(rotateViewport(FILL_VIEWPORT)).toEqual(FILL_VIEWPORT);
|
||||
});
|
||||
});
|
||||
|
||||
describe('fitting', () => {
|
||||
const viewport = { kind: 'custom', width: 400, height: 800 } as const;
|
||||
|
||||
test('keeps the chosen size and scales down to fit', () => {
|
||||
const layout = fitViewport(viewport, { width: 200, height: 800 });
|
||||
expect(layout).toEqual({ width: 400, height: 800, scale: 0.5 });
|
||||
});
|
||||
|
||||
test('fits by whichever side runs out first', () => {
|
||||
expect(fitViewport(viewport, { width: 800, height: 400 })?.scale).toBe(0.5);
|
||||
});
|
||||
|
||||
test('never enlarges, which would misrepresent the size asked for', () => {
|
||||
expect(fitViewport(viewport, { width: 4000, height: 4000 })?.scale).toBe(1);
|
||||
});
|
||||
|
||||
test('survives a container measured at zero mid-layout', () => {
|
||||
const layout = fitViewport(viewport, { width: 0, height: 0 });
|
||||
expect(layout?.width).toBe(400);
|
||||
expect(layout && layout.scale > 0).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('agent viewport vocabulary', () => {
|
||||
test('each named mode resolves to a real size', () => {
|
||||
for (const mode of ['mobile', 'tablet', 'desktop'] as const) {
|
||||
const size = viewportSize(viewportForMode(mode));
|
||||
expect(size !== null).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
test('fill has no size, as the agent should expect', () => {
|
||||
expect(viewportSize(viewportForMode('fill'))).toBeNull();
|
||||
});
|
||||
|
||||
test('accepts only the vocabulary it published', () => {
|
||||
expect(isViewportMode('mobile')).toBe(true);
|
||||
expect(isViewportMode('phone')).toBe(false);
|
||||
expect(isViewportMode(390)).toBe(false);
|
||||
});
|
||||
|
||||
test('reports back in the same words it accepts', () => {
|
||||
expect(viewportSummary(viewportForMode('mobile')).mode).toBe('mobile');
|
||||
expect(viewportSummary(FILL_VIEWPORT).mode).toBe('fill');
|
||||
});
|
||||
|
||||
test('calls a hand-typed size custom rather than the nearest name', () => {
|
||||
const summary = viewportSummary({ kind: 'custom', width: 500, height: 900 });
|
||||
expect(summary.mode).toBe('custom');
|
||||
expect(summary.width).toBe(500);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,147 @@
|
||||
/**
|
||||
* Viewport sizing for the browser panel.
|
||||
*
|
||||
* The page is rendered at a chosen size and scaled down to fit the panel when
|
||||
* it does not. Scaling is visual only: the view still lays out at the chosen
|
||||
* width, which is the whole point — a 390px layout has to be measured at 390px,
|
||||
* not at whatever the panel happens to be.
|
||||
*/
|
||||
|
||||
export type BrowserViewport =
|
||||
| { readonly kind: 'fill' }
|
||||
| { readonly kind: 'preset'; readonly id: string; readonly width: number; readonly height: number }
|
||||
| { readonly kind: 'custom'; readonly width: number; readonly height: number };
|
||||
|
||||
export const FILL_VIEWPORT: BrowserViewport = { kind: 'fill' };
|
||||
|
||||
export const MIN_VIEWPORT_SIZE = 240;
|
||||
export const MAX_VIEWPORT_SIZE = 3840;
|
||||
|
||||
export type ViewportPreset = {
|
||||
readonly id: string;
|
||||
readonly label: string;
|
||||
readonly width: number;
|
||||
readonly height: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* Sizes worth having, not every device ever made. A long list is harder to pick
|
||||
* from than it is useful, and anything missing can be typed in directly.
|
||||
*/
|
||||
export const VIEWPORT_PRESETS: readonly ViewportPreset[] = [
|
||||
{ id: 'iphone-se', label: 'iPhone SE', width: 375, height: 667 },
|
||||
{ id: 'iphone-14', label: 'iPhone 14', width: 390, height: 844 },
|
||||
{ id: 'iphone-14-pro-max', label: 'iPhone 14 Pro Max', width: 430, height: 932 },
|
||||
{ id: 'pixel-7', label: 'Pixel 7', width: 412, height: 915 },
|
||||
{ id: 'ipad-mini', label: 'iPad mini', width: 768, height: 1024 },
|
||||
{ id: 'ipad-pro', label: 'iPad Pro', width: 1024, height: 1366 },
|
||||
{ id: 'laptop', label: 'Laptop', width: 1280, height: 800 },
|
||||
{ id: 'desktop', label: 'Desktop', width: 1440, height: 900 },
|
||||
];
|
||||
|
||||
export const clampViewportSize = (value: number): number => {
|
||||
if (!Number.isFinite(value)) return MIN_VIEWPORT_SIZE;
|
||||
return Math.round(Math.min(MAX_VIEWPORT_SIZE, Math.max(MIN_VIEWPORT_SIZE, value)));
|
||||
};
|
||||
|
||||
export const viewportSize = (
|
||||
viewport: BrowserViewport,
|
||||
): { width: number; height: number } | null => (
|
||||
viewport.kind === 'fill' ? null : { width: viewport.width, height: viewport.height }
|
||||
);
|
||||
|
||||
/** Turns a preset id into a viewport, or null when the id is unknown. */
|
||||
export const presetViewport = (id: string): BrowserViewport | null => {
|
||||
const preset = VIEWPORT_PRESETS.find((entry) => entry.id === id);
|
||||
if (!preset) return null;
|
||||
return { kind: 'preset', id: preset.id, width: preset.width, height: preset.height };
|
||||
};
|
||||
|
||||
export const rotateViewport = (viewport: BrowserViewport): BrowserViewport => {
|
||||
if (viewport.kind === 'fill') return viewport;
|
||||
// Rotating a preset stops it being that preset: an iPhone on its side is no
|
||||
// longer the entry in the list, and pretending otherwise makes the picker lie.
|
||||
return { kind: 'custom', width: viewport.height, height: viewport.width };
|
||||
};
|
||||
|
||||
export type ViewportLayout = {
|
||||
/** Size to lay the page out at, in CSS pixels. */
|
||||
readonly width: number;
|
||||
readonly height: number;
|
||||
/** Visual scale, ≤ 1. Applied with a transform; the page never learns of it. */
|
||||
readonly scale: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* Fits a chosen viewport into the space available.
|
||||
*
|
||||
* Only ever scales down. Enlarging a small viewport to fill a big panel would
|
||||
* misrepresent the very thing the user asked to see.
|
||||
*/
|
||||
export const fitViewport = (
|
||||
viewport: BrowserViewport,
|
||||
available: { width: number; height: number },
|
||||
): ViewportLayout | null => {
|
||||
const size = viewportSize(viewport);
|
||||
if (!size) return null;
|
||||
|
||||
const usableWidth = Math.max(1, available.width);
|
||||
const usableHeight = Math.max(1, available.height);
|
||||
const scale = Math.min(1, usableWidth / size.width, usableHeight / size.height);
|
||||
return { width: size.width, height: size.height, scale };
|
||||
};
|
||||
|
||||
/** Label for the current viewport, for the size control. */
|
||||
export const describeViewport = (viewport: BrowserViewport): string => {
|
||||
if (viewport.kind === 'fill') return '';
|
||||
if (viewport.kind === 'preset') {
|
||||
return VIEWPORT_PRESETS.find((entry) => entry.id === viewport.id)?.label ?? '';
|
||||
}
|
||||
return '';
|
||||
};
|
||||
|
||||
/**
|
||||
* The vocabulary the agent gets.
|
||||
*
|
||||
* Named sizes rather than pixel dimensions: an agent asked to "check the mobile
|
||||
* layout" should not have to invent a width, and a number it invented tells the
|
||||
* user nothing about what was actually checked.
|
||||
*/
|
||||
const VIEWPORT_MODES = ['mobile', 'tablet', 'desktop', 'fill'] as const;
|
||||
export type BrowserViewportMode = (typeof VIEWPORT_MODES)[number];
|
||||
|
||||
const MODE_PRESETS: Record<Exclude<BrowserViewportMode, 'fill'>, string> = {
|
||||
mobile: 'iphone-14',
|
||||
tablet: 'ipad-mini',
|
||||
desktop: 'desktop',
|
||||
};
|
||||
|
||||
export const isViewportMode = (value: unknown): value is BrowserViewportMode => (
|
||||
typeof value === 'string' && (VIEWPORT_MODES as readonly string[]).includes(value)
|
||||
);
|
||||
|
||||
export const viewportForMode = (mode: BrowserViewportMode): BrowserViewport => (
|
||||
mode === 'fill' ? FILL_VIEWPORT : presetViewport(MODE_PRESETS[mode]) ?? FILL_VIEWPORT
|
||||
);
|
||||
|
||||
/**
|
||||
* Reports the current viewport in the agent's own vocabulary, so a snapshot
|
||||
* states which layout it describes.
|
||||
*/
|
||||
export const viewportSummary = (viewport: BrowserViewport): {
|
||||
mode: BrowserViewportMode | 'custom';
|
||||
width: number | null;
|
||||
height: number | null;
|
||||
} => {
|
||||
const size = viewportSize(viewport);
|
||||
if (!size) return { mode: 'fill', width: null, height: null };
|
||||
|
||||
for (const mode of ['mobile', 'tablet', 'desktop'] as const) {
|
||||
const preset = viewportForMode(mode);
|
||||
const presetSize = viewportSize(preset);
|
||||
if (presetSize && presetSize.width === size.width && presetSize.height === size.height) {
|
||||
return { mode, width: size.width, height: size.height };
|
||||
}
|
||||
}
|
||||
return { mode: 'custom', width: size.width, height: size.height };
|
||||
};
|
||||
Reference in New Issue
Block a user