fix(ui): cap extracted document context

This commit is contained in:
Bohdan Triapitsyn
2026-08-18 19:15:10 +03:00
parent 3326ad0ea8
commit a29aaa7660
3 changed files with 4 additions and 4 deletions
+1 -1
View File
@@ -57,7 +57,7 @@ So:
Local chat attachments are normalized by `attachment-files.ts` before entering `input-store.ts`. PNG, JPEG, GIF, WebP, and PDF retain their media type; HEIC/HEIF is converted to JPEG; recognized text/code formats and unknown files whose first 4 KB are text are sent as `text/plain`; binary files outside the supported media types are rejected. Jupyter notebooks become readable markdown with non-text outputs omitted. HAR credentials, cookies, and sensitive URL parameters are redacted, while request/response body text is omitted. SVG and Draw.io files are attached as source text, not executable/rendered content. Browser and VS Code pickers expose the same allowlist, while drag-and-drop may still accept an unknown extension after content inspection.
Office and OpenDocument packages are metadata-validated before asynchronous extraction, with limits of 20 MB compressed input, 5,000 archive entries, 25 MB per entry, 8 MB per XML part, and 100 MB total uncompressed content. Unsafe or non-canonical archive paths reject the whole attachment, and only XML, relationship, and supported image entries are decompressed and retained. Extracted text, including its explicit truncation notice, is bounded to 2,000,000 characters. At most 50 signature-validated PNG, JPEG, GIF, or WebP images and 40 MB of image bytes are retained, with a 20 MB per-image limit; unsupported, invalid, omitted, and truncated content remains explicit in the extracted text. Images whose citations fall beyond text truncation are not attached. Extracted document content remains a `text/plain` file attachment with the original document filename, rather than becoming visible user-message text. Supported embedded images become separate image file parts; the extracted text contains `[filename]` citations at the source paragraph, slide object, spreadsheet cell anchor, or OpenDocument text position. Generated image filenames are re-evaluated if the composer changes during asynchronous preparation, avoiding collisions. The store publishes all generated parts atomically only after every data URL is ready.
Office and OpenDocument packages are metadata-validated before asynchronous extraction, with limits of 20 MB compressed input, 5,000 archive entries, 25 MB per entry, 8 MB per XML part, and 100 MB total uncompressed content. Unsafe or non-canonical archive paths reject the whole attachment, and only XML, relationship, and supported image entries are decompressed and retained. Extracted text, including its explicit truncation notice, is bounded to 250,000 characters so compact but dense Office files cannot consume an entire model context window. At most 50 signature-validated PNG, JPEG, GIF, or WebP images and 40 MB of image bytes are retained, with a 20 MB per-image limit; unsupported, invalid, omitted, and truncated content remains explicit in the extracted text. Images whose citations fall beyond text truncation are not attached. Extracted document content remains a `text/plain` file attachment with the original document filename, rather than becoming visible user-message text. Supported embedded images become separate image file parts; the extracted text contains `[filename]` citations at the source paragraph, slide object, spreadsheet cell anchor, or OpenDocument text position. Generated image filenames are re-evaluated if the composer changes during asynchronous preparation, avoiding collisions. The store publishes all generated parts atomically only after every data URL is ready.
The composer compares normalized attachment MIME types with the selected model's declared input modalities. It warns when a newly attached file or an existing attachment after a model change requires an unsupported modality, but does not block sending. Missing modality metadata remains unknown and does not produce a warning.
@@ -196,7 +196,7 @@ describe("document attachment extraction", () => {
test("does not retain images whose citations fall beyond the text limit", async () => {
const file = zippedFile("long.docx", {
"word/document.xml": `<w:document xmlns:w="w" xmlns:a="a" xmlns:r="r"><w:body><w:p><w:t>${"x".repeat(2_000_100)}</w:t></w:p><w:p><a:blip r:embed="image"/></w:p></w:body></w:document>`,
"word/document.xml": `<w:document xmlns:w="w" xmlns:a="a" xmlns:r="r"><w:body><w:p><w:t>${"x".repeat(250_100)}</w:t></w:p><w:p><a:blip r:embed="image"/></w:p></w:body></w:document>`,
"word/_rels/document.xml.rels": relationships([{ id: "image", target: "media/image.png" }]),
"word/media/image.png": pngBytes(),
})
@@ -204,7 +204,7 @@ describe("document attachment extraction", () => {
const result = await extractDocumentAttachments(file)
const text = await result?.textFile.text() ?? ""
expect(text.length <= 2_000_000).toBe(true)
expect(text.length <= 250_000).toBe(true)
expect(text.endsWith("[Document text truncated by OpenChamber]\n")).toBe(true)
expect(text.includes("[long-image-1.png]")).toBe(false)
expect(result?.images).toEqual([])
+1 -1
View File
@@ -8,7 +8,7 @@ const MAX_ARCHIVE_ENTRIES = 5_000
const MAX_EMBEDDED_IMAGES = 50
const MAX_EMBEDDED_IMAGE_BYTES = 20 * 1024 * 1024
const MAX_EMBEDDED_IMAGES_BYTES = 40 * 1024 * 1024
const MAX_EXTRACTED_TEXT_CHARS = 2_000_000
const MAX_EXTRACTED_TEXT_CHARS = 250_000
const MAX_ODF_SPACES_PER_ELEMENT = 100
const TEXT_TRUNCATION_NOTICE = "\n\n[Document text truncated by OpenChamber]\n"