ProjectE/docs/ARCHITECTURE.md

# Architecture Documentation

Project E is a Next.js monorepo that stores application data in PostgreSQL 16 through Drizzle ORM. NextAuth credentials authentication manages user sessions. This document describes the implemented system.

## System overview

```
Browser
  │
  ├── REST requests and Server-Sent Events
  ▼
Next.js application (apps/web)
  ├── App Router pages and API routes
  ├── NextAuth credentials provider and JWT sessions
  ├── Services, validation, and collection adapter
  ├── MCP server
  └── Drizzle ORM
        │
        ▼
PostgreSQL 16
  ├── users
  ├── records (JSONB application data)
  └── project_e_events notifications
        │
        ▼
Background worker
  └── Polls queue_jobs records and processes asynchronous work
```

## Application layers

### Presentation

`apps/web/app/` contains App Router pages and API routes. `apps/web/components/` contains UI components. Zustand stores and hooks manage client-side state and consume API and SSE updates.

### Business logic

`apps/web/lib/services/` contains entity behavior such as task, habit, note, report, project, webhook, and agent-mention services. `packages/shared/` provides Zod schemas and TypeScript types for validation across the app and worker.

### Persistence

`packages/db/src/schema.ts` defines Drizzle's PostgreSQL schema. `packages/db/src/index.ts` creates a Drizzle client from `DATABASE_URL`.

The `users` table stores user identity, email addresses, and bcrypt password hashes. The `records` table stores collection data in JSONB with a collection name and timestamps. `apps/web/lib/database.ts` exposes collection operations over those records and emits PostgreSQL notifications after creates, updates, and deletes.

## Authentication and authorization

NextAuth uses the credentials provider and JWT sessions. A user signs in with an email address and password. The authorization flow normalizes the email address, reads the user from PostgreSQL, and compares the password against its bcrypt hash.

On an empty `users` table, a sign-in matching `INITIAL_ADMIN_EMAIL` and `INITIAL_ADMIN_PASSWORD` creates the first account. Set `NEXTAUTH_SECRET` for each environment and keep it stable to preserve active sessions.

API routes require an authenticated user through the application's authentication middleware. MCP requests use agent API keys and permission tiers. The `read_only` tier limits agents to read operations.

## Request and data flow

### Create a task

```
Browser → POST /api/tasks → authenticated API route
        → Zod validation → service and collection adapter
        → Drizzle write to PostgreSQL
        → pg_notify('project_e_events')
        → SSE clients receive the update
```

### Realtime updates

The `/api/realtime` endpoint authenticates the request, opens a PostgreSQL listener for `project_e_events`, and streams matching collection events through Server-Sent Events. Clients can specify a comma-separated `collections` query parameter. The endpoint sends a keepalive comment every 30 seconds and closes its PostgreSQL listener when the client disconnects.

The reverse proxy must not buffer SSE responses.

## Drizzle migrations and Docker

Drizzle Kit reads `drizzle.config.ts`, which points to `packages/db/src/schema.ts` and writes generated SQL to `drizzle/`. `DATABASE_URL` supplies the PostgreSQL connection string.

Docker Compose runs PostgreSQL 16 and mounts `drizzle/0000_first_mauler.sql` as an initialization script. PostgreSQL runs initialization scripts only for a new data volume. Apply later reviewed migrations during deployment; restarting an existing database container does not run them.

## Background worker

The worker polls `queue_jobs` records for due work. It marks a job `in_progress`, runs the handler, then records `completed` or schedules a retry after an error. Its polling interval starts at five seconds, grows to 60 seconds when no work is available, and resets when it finds jobs.

Supported job types include webhook delivery, agent mentions, report generation, recurring tasks, and cleanup. Failed jobs retry with exponential backoff up to their configured retry limit.

## MCP server

The MCP server runs at `/api/mcp` over Streamable HTTP. It establishes sessions with `GET`, accepts JSON-RPC requests with `POST`, and ends sessions with `DELETE`. The app keeps active transports in memory.

Tools live in `apps/web/lib/mcp/tools/` and register by entity. Tool handlers use the database-backed collection adapter. Agent API keys identify callers, permission tiers restrict access, and the app records agent activity.

## Security

- PostgreSQL credentials come from `DATABASE_URL` and `POSTGRES_PASSWORD`.
- NextAuth signs JWT sessions with `NEXTAUTH_SECRET`.
- The app stores password hashes with bcrypt, never plaintext passwords.
- Cookies use `httpOnly`, `sameSite: lax`, and `secure` settings appropriate to the deployment.
- Zod schemas validate API request bodies.
- Reverse proxies should enforce TLS, host restrictions, and rate limits.

## Performance and operations

PostgreSQL stores indexed collection and creation-time metadata while retaining flexible collection fields in JSONB. API routes paginate list responses. The Drizzle client uses a connection pool with a maximum of 10 connections.

Each realtime client holds a PostgreSQL notification listener. Monitor long-lived SSE connections and proxy timeouts as concurrency grows. The worker uses database-backed job records, so multiple workers can process queued work when the job-claiming behavior supports the workload.

## Accessibility

Project E targets WCAG 2.1 AA. Interactive elements support keyboard navigation, icon-only controls include accessible names, form fields use labels, dialogs manage focus, and dynamic content uses live regions. The UI respects `prefers-reduced-motion` and pairs status colors with text or icons.