fix(scheduled-tasks): prevent dual-server double dispatch of daily tasks (#2713)

* fix(scheduled-tasks): claim schedule occurrences across server instances

Two OpenChamber servers sharing project config each armed timers and both
dispatched the same daily/weekly/cron/once slot (#2710). Claim the occurrence
in shared config under a cross-process write lock before creating a session.

Co-authored-by: Serhii Dziupin <makeittech@users.noreply.github.com>

* fix(scheduled-tasks): harden occurrence claim failure and lock ownership

Address PR review blockers: release running-slot bookkeeping when claim
throws, avoid silently dropping an armed occurrence after a due-slack sync,
verify lock-file ownership on release, and cover real on-disk lock behavior.

Co-authored-by: Serhii Dziupin <makeittech@users.noreply.github.com>

* fix(scheduled-tasks): always release running slot on state-write failures

Wrap runTask bookkeeping in finally so claim, manual-start, and completion
lock timeouts cannot stuck-run a task; drop the diskNext claim guard that
suppressed later occurrences; recover unparseable locks via mtime age.

Co-authored-by: Serhii Dziupin <makeittech@users.noreply.github.com>

* fix(scheduled-tasks): stop re-arming past nextRunAt and clear stuck running

Only schedule future nextRunAt values so once-task losers and claim-failed
paths cannot spin delay-0 retries. Clear past once nextRunAt on claim, and on
completion-write failure retry terminal status so manual runNow still returns
the session instead of a hard 500 with lastStatus stuck running.

Co-authored-by: Serhii Dziupin <makeittech@users.noreply.github.com>

* fix(scheduled-tasks): release write chain on lock acquire timeout

withProjectWriteLock left the in-process promise chain pending when
acquireProjectFileLock timed out, wedging every later project write and
stranding runTask before finally. Always release the chain; surface
persistError on run; record once claim failures in task state.

Co-authored-by: Serhii Dziupin <makeittech@users.noreply.github.com>

---------

Co-authored-by: Serhii Dziupin <makeittech@users.noreply.github.com>
This commit is contained in:
Serhii Dziupin
2026-08-13 16:14:42 +03:00
committed by GitHub
co-authored by Serhii Dziupin
parent 86e6a2ae76
commit e99f6560be
20 changed files with 1436 additions and 117 deletions
@@ -262,3 +262,23 @@ describe('scheduled-task service remove', () => {
expect(Array.isArray(tasks)).toBe(true);
});
});
describe('scheduled-task service run', () => {
it('forwards persistError when the runtime reports a completion persist failure', async () => {
const { service } = createService({
scheduledTasksRuntime: {
runNow: vi.fn(async () => ({
ok: true,
sessionID: 'sess-1',
task: { id: 'task-1', state: { lastStatus: 'success' } },
persistError: 'timeout acquiring project config lock for project-test',
reason: 'completion-state-failed',
})),
},
});
const result = await service.run('project-test', 'task-1');
expect(result.sessionId).toBe('sess-1');
expect(result.persistError).toMatch(/timeout acquiring project config lock/);
});
});