Architecture
Semantask, as engineering notes.
Semantask is the open-source work-coordination platform at github.com/harshlocham/semantask. These notes walk through each component: why it exists, what it owns, the decision behind it, and what that decision costs.
Scaling notes describe the intended shape, not traffic I have measured in production. This website is a different, much smaller system — see the appendix.
System overview
Three paths: request, real-time, and worker.
Writes go over HTTP and are persisted before anything is broadcast. The socket server only transports. Work for the AI worker is enqueued in the same transaction as the write that caused it.
HTTP path
Client → Next.js API → MongoDB
Real-time path
Client ↔ Socket.IO ↔ Redis adapter
Worker path
Outbox → task worker → LLM / tools
01HTTP
HTTP — persistence and business logic
Validation, persistence and authorization. The only writer.
- Why it exists
- In Semantask, all durable writes go through the Next.js API layer. Messages, conversations, tasks, and user data are validated and persisted to MongoDB before any real-time event is emitted.
- What it owns
- Validation, persistence, authorization, and business logic — including internal HTTP endpoints the socket layer uses for ban status, conversation membership, and token validity.
- Does not own
- Fan-out, presence, or transport. The socket server stays schema-free and does not need to know about Mongoose models or database transactions.
- Decision
- Authorization goes over an internal HTTP bridge instead of giving the socket server database models (ADR-003). The socket process calls private endpoints — authorize-identity, authorize-conversation-access, authorize-message-action — authenticated by a shared internal secret compared with crypto.timingSafeEqual. Identity is re-validated against MongoDB on connect, never from JWT claims alone, so a banned user holding a valid token cannot keep operating until expiry.
- Tradeoffs
- API routes can be deployed separately from realtime infrastructure. The tradeoff is an extra HTTP hop for authorization on hot-path operations.
- Any privileged socket action depends on the web app being up.
- One shared secret with no per-service scope or rotation primitive, and a 45s cache lag on identity revocation. mTLS or short-lived service JWTs would replace the secret first.
02WebSocket
WebSocket — transport and fan-out
Transport, rooms and fan-out. Schema-free.
- Why it exists
- Socket.IO runs as a dedicated transport server. It authenticates connections, manages rooms, and fans out events to connected clients.
- What it owns
- Transport, room management, and event fan-out. Once data is safely stored, events are emitted through Socket.IO.
- Does not own
- Durable writes and Mongo schema. Persistence stays in the HTTP layer so the socket process can stay stateless.
- Decision
- The socket server never writes a message to MongoDB. The write happens in the Next.js API, which is also the only producer of the outbox. Persisting in the socket handler was rejected because it makes the process stateful and couples it to the Mongo schema; dual-write was rejected because two writers means two truths.
- Tradeoffs
- The design is a stateless transport plane behind least_conn, with the Redis adapter coordinating rooms, so a socket process can restart without losing durable state. That is the intended shape, not traffic I have measured in production.
- Persist-then-broadcast is a client contract, not a server guarantee. A misbehaving client can emit without persisting, and receivers see a ghost message until they refresh.
- Messages are emitted to the conversation room and to each online participant's user room, so every client has to dedupe on message._id.
03Redis
Redis — fan-out and ephemeral state
Adapter fan-out and ephemeral state, treated as two systems.
- Why it exists
- The Redis adapter synchronizes room state across multiple socket instances. Ephemeral state — presence, typing indicators, delivery receipts — lives in Redis on the hot path, not in MongoDB.
- What it owns
- Pub/sub fan-out for the adapter, plus hot-path ephemeral state.
- Does not own
- The system of record or the job queue. Durable conversations, tasks, and user data stay in MongoDB, and worker triggers go through the Mongo outbox.
- Decision
- Redis plays two unrelated roles — the Socket.IO adapter, and application state such as presence and dedupe — and they are treated as two systems because they fail differently. Presence is TTL-based and self-healing: a user is online only while their socket set is non-empty and a 60s presence key has not expired, and a 5s sweeper reaps stale users. An ungraceful disconnect self-corrects in about 65s without relying on a disconnect handler that never runs when a process is killed.
- Tradeoffs
- An emit to a conversation room reaches clients regardless of which pod they are connected to. Adapter fan-out and application Redis state fail differently — treat them as two systems.
- If the adapter is down, cross-pod emits silently degrade while per-pod emits still work. If application Redis is down, presence loses persistence and isUserOnline returns false.
- A dev-mode in-memory Redis shim lets the socket server run without Redis. It is not horizontally safe and is guarded only by a startup warning; a hard fail in the production profile would be better.
Related decisions: D3
04Outbox
Outbox — durable enqueue
Durable enqueue in the same transaction as the domain write.
- Why it exists
- The web layer never calls the worker directly. Domain writes and OutboxEvents commit in the same Mongo transaction, so work is durable before anything executes.
- What it owns
- Durable enqueue of work alongside the domain write. No lost events, no phantom events.
- Does not own
- LLM inference or direct RPC to the worker.
- Decision
- createMessage inserts the message, updates lastMessage and enqueues the outbox event atomically, with a no-transaction fallback for standalone Mongo. The worker claims events with an atomic findOneAndUpdate. Direct web-to-worker RPC was rejected because it couples request latency to worker availability; a dedicated queue (SQS, Kafka, BullMQ) was rejected as more operational surface for a stack that already runs Mongo and Redis.
- Tradeoffs
- At-least-once delivery with crash recovery: stale processing rows older than 5 min are re-claimable. Layered idempotency at event, action, and tool levels makes re-delivery safe.
- The retry scanner's withTransaction requires a replica-set MongoDB; a single node fails the scanner with no current guard. That deployment assumption is under-documented.
05Worker
Worker — leased, idempotent execution
Leased, heartbeated, idempotent execution.
- Why it exists
- Workers claim events atomically, hold a DB lease with heartbeat renewal, and run a bounded agent loop behind a multi-provider LLM abstraction.
- What it owns
- Task execution, LLM/tools, and lifecycle events streamed over Socket.IO for dashboards.
- Does not own
- HTTP API persistence and user-facing writes. The web layer stays the writer of record.
- Decision
- A task is executed by exactly one worker at a time via a DB-held lease (Task.leaseOwner, leaseExpiresAt), renewed by a heartbeat every leaseMs/3. Tool side effects are keyed by a run-independent idempotency key — SHA-256(taskId | stepId | toolName | params) — so a lease steal cannot mint a new key and re-run an email or webhook. Lease-busy defers the outbox event instead of completing it.
- Tradeoffs
- If the heartbeat fails, an AbortController aborts the in-flight LLM call; expired leases can be stolen for crash recovery, and the original owner's release becomes a no-op.
- Remaining cost is operational: heartbeat tuning, steal windows, and replica-set assumptions for transactional paths.
- The runner does not stream LLM output even though the provider flag exists, so worst-case latency is the full response time. Acceptable for batch-style work, not for interactive use.
06Decisions
The decision log
Each entry: the choice, the alternatives rejected, and what it actually cost.
Why separate HTTP and WebSocket layers
Socket.IO remains a transport layer. HTTP handles validation, persistence, and business logic. Once data is safely stored, events are emitted through Socket.IO.
The split is so each layer can be deployed on its own. A socket process can restart without losing durable state, because it does not own Mongo writes. I have not run this at measured production traffic.
The tradeoff is an extra HTTP hop for authorization on hot-path operations. The socket server stays schema-free, which is what makes a second socket process possible.
- D1
Socket server is transport-only
apps/socket authenticates, authorizes, and fans out events. It never writes a Message to MongoDB; the Next.js API is the only writer and the only outbox producer.
- Rejected
- Persisting in the socket handler (stateful, schema-coupled) and dual-write (two writers, guaranteed drift).
- Cost
- Persist-then-broadcast ordering is a client contract, not a server guarantee. Clients reconcile the live event against the persisted REST view.
- D2
Authorization over an internal HTTP bridge (ADR-003)
The socket server makes trust decisions by calling private web endpoints guarded by a shared internal secret. Identity is re-validated against Mongo on connect, never from claims alone.
- Rejected
- Giving the socket server Mongoose models (re-couples the schema) and trusting JWT claims (a banned user keeps operating until token expiry).
- Cost
- A network hop per authorized operation, a dependency on web uptime, one unscoped shared secret, and a 45s revocation lag.
- D3
Redis as adapter and application state — treated as two systems
One Redis serves the Socket.IO adapter and application state (presence, delivery state, idempotency keys, rate limiting), with TTL-based, self-healing presence.
- Cost
- The in-memory dev shim is not horizontally safe and is guarded only by a startup warning.
- D4
Transactional outbox as the integration boundary (ADR-002)
The domain change and an OutboxEvent commit in one Mongo transaction; the worker claims events with an atomic findOneAndUpdate.
- Rejected
- Direct RPC from web to worker (a worker outage drops work) and a dedicated queue (more operational surface).
- Cost
- At-least-once delivery, and the retry scanner requires a replica-set Mongo.
- D5
Layered idempotency instead of chasing exactly-once
Accept at-least-once and enforce idempotency at the event, action, and tool layers. The email tool forwards its key as the provider's Idempotency-Key header so the provider dedupes too.
- Cost
- The original tool key included runId, so a lease steal could re-execute a side effect. The shipped key is run-independent: SHA-256(taskId | stepId | toolName | params).
- D6
Distributed lease and heartbeat for one writer per task
A DB-held lease on the Task document, acquired atomically and renewed every leaseMs/3. A failed heartbeat aborts the in-flight LLM call and tool fetch.
- Rejected
- Optimistic concurrency alone (does not reliably guard between fetch and save) and a Redis-only lock (lease state would live apart from task state).
- Cost
- Heartbeat tuning, steal windows, and replica-set assumptions for transactional paths.
- D7
Execution state machine is authoritative (ADR-001)
Fine-grained executionState is the enforcement boundary. Legacy lifecycle fields are projected from it at write time; divergence is logged, not auto-repaired.
- Cost
- Two fields still exist on the document.
- D8
LLM provider abstraction behind capability flags
A BaseLLMProvider with flags for structured outputs, tool calling, streaming and JSON mode; OpenAI, OpenAI-compatible and Hugging Face providers are selected by LLM_PROVIDER. A test seam injects deterministic responses so the agent loop is testable without a live model.
- Rejected
- Hardcoding OpenAI, which becomes a rewrite the first time a self-hosted endpoint is needed.
- Cost
- The runner does not use streaming, so worst-case latency is the full response time.
- D9
Five coarse shared packages, workspace:* everywhere
types → db / auth → services, plus observability. Types stay free of mongoose and ioredis so the React Native client can consume them.
- Cost
- Dependency direction is convention, not enforcement, and workspace:* makes changeset versions documentary.
- D10
Redundant fan-out, client-side dedupe
Emit to the conversation room and directly to each online participant's user room, so clients that have not finished joining still receive the message.
- Cost
- Extra bandwidth and a hard dependency on every client deduping on message._id.
Appendix
This website
harshlocham.com is a Next.js UI plus a separate Socket.IO process. On each assistant message the server ranks heading chunks from markdown with a lexical scorer, then streams an answer. There is no MongoDB, no transactional outbox, and no task worker on this site.
Redis here is optional. It can attach a Socket.IO adapter and it stores the per-IP chat rate limit, with an in-memory fallback when Redis is down. Production runs one socket process, so the adapter is not what makes the chat correct.
The assistant speaks about Harshdeep in the third person, answers only from the retrieved notes, and links back to the page that covers the topic.