# Operating eorganize This file is the source of truth for how the app is used day-to-day. If you change the workflow, update this file. ## The capture → identify loop The app is designed around a two-phase workflow so capturing in the field is fast and identification can happen at a desk later. ### Phase 1 — capture (on phone, in the field) Open on your phone. 1. Pin a location once at the top of the screen. Type freeform, e.g. `Garage - middle shelf - 4th level`. Separators `>`, `/`, and ` - ` all work; missing levels in the hierarchy are auto-created. The pinned location persists in `localStorage`, so you can leave and come back without re-typing. 2. Tap "Tap to capture" → take a photo. 3. (Optional) Add context — two ways, use whichever fits the moment: - **Voice note** (tap "Record voice note"): captured as audio. Stored on the VPS. Transcribed locally on the dev Mac during the identify pass via whisper.cpp (free, ~5s per clip on a modern Mac). The cleaned transcript lands in the caption field after identify; the audio file stays on the item for replay. - **Caption text**: type, or tap the iOS keyboard mic for free on-device dictation. Lands as text immediately. **Ramble freely.** Both paths produce a transcript that the identify pass cleans up. Filler ("uh", "just a"), repetition, and false starts are expected — capture is meant to be fast and low-friction in the field. 4. Tap "Save & next". The camera reopens immediately. Keep going. The item lands in Postgres with `name = "Untitled item"`, the pinned location (creating the hierarchy if needed), the photo, and the caption text. It is fully searchable — the caption is part of the search haystack. ### Phase 2 — identify (at a desk, via Claude Code) This is the part that gives every item a real name, brand, category, and tags. It runs **on the home server via Claude Code**, not on the VPS. There is no production-side AI pipeline. The pass can be kicked off three ways: - **Automatically, by queue depth**: whenever an item enters the pending queue (a fresh capture, a `pendingNote`, or a new voice note) and the queue has reached **7** items, the app auto-launches a pass — no tap needed. This uses the same SSH launch as the button and the same in-flight guard, so it never starts a second pass while one is running. The launch is fire-and-forget so it never slows down "Save & next". Tune the threshold with `IDENTIFY_AUTO_THRESHOLD` (set to `0` to disable auto-triggering and go back to manual-only). Auto-started sessions are tagged `triggeredById = 'auto'`. - **From the app**: open `/identify` and tap **Start AI pass**. The button shows the current queue depth and the result of the last run. The VPS SSHes the home server to launch the wrapper script in the background; see "Triggering AI pass from the app" below for the wiring. - **Manually in Claude Code on the dev Mac**, with a prompt like: > "go do your thing w/ AI on the latest items" Claude Code will: 1. Query the VPS DB for items that need a look — either `name = 'Untitled item'` (fresh captures that have never been identified) **or** `needsAIReview = true` (already-identified items the user has dictated/recorded more info about, or re-recorded the voice note for — see "Adding info to an already-identified item" below). **Pull every photo for each item** (an item can have 1-N photos — multi-angle shots, labels, serial plates), along with its current name/category/brand/tags, caption, voiceNotePath, voiceNoteMime, and pendingNote. A working query shape: ```sql SELECT i.id, i.name, i.category, i.brand, i.caption, i."pendingNote", i."voiceNotePath", i."voiceNoteMime", json_agg(json_build_object('path', p.path, 'thumbPath', p."thumbPath", 'isPrimary', p."isPrimary") ORDER BY p."isPrimary" DESC, p."createdAt" ASC) AS photos FROM "Item" i LEFT JOIN "Photo" p ON p."itemId" = i.id WHERE i.name = 'Untitled item' OR i."needsAIReview" = true GROUP BY i.id; ``` 2. Download the thumbnails from `https://eorganize.etpics.com/api/uploads/...` (works because `BYPASS_AUTH=true` on the VPS gates). **Download every photo, not just the primary.** 3. **Look at every photo for the item, not just one.** Different angles often show the serial plate, brand label, or distinctive feature that the primary frame misses — using only the first photo will misidentify items the user deliberately shot from multiple sides. Then read the caption. **Transcribe `voiceNotePath` whenever the item is in the queue** — it's either the original capture audio (fresh untitled items) or a re-recorded correction (items with `needsAIReview = true`). Either way the voice may carry info that needs to land in the caption: ```bash curl -sL "https://eorganize.etpics.com/api/uploads/" -o /tmp/voice.webm ffmpeg -y -i /tmp/voice.webm -ar 16000 -ac 1 -f wav /tmp/voice.wav 2>/dev/null whisper-cli -m /tmp/whisper-models/ggml-base.en.bin -f /tmp/voice.wav --no-timestamps ``` The model lives at `/tmp/whisper-models/ggml-base.en.bin`; if `/tmp` got cleared, re-download it from (~150MB). Treat the transcript(s) + photo + existing caption + pendingNote together as ground truth for the user's intent. 4. Write a SQL batch that: - Sets `name`, `brand`, `category`, `description`, and any `qty` adjustment extracted from caption/pendingNote + photo (e.g. caption says "a couple of …" → qty = 2). - **Keeps `description` tight and search-focused.** It should describe the *item itself* — material, function, distinctive features — in one to two sentences. **Do not** add: (a) location restatement (location is already joined into `searchText`), (b) cross-references to neighboring items ("companion to the pole saw"), (c) transient state ("shown collapsed"), (d) meta-commentary about previous passes ("was misidentified as…"), or (e) brand/category restatement (those are already in their own columns and concatenated into `searchText`). Every word in `description` should help someone *find* or *recognize* the item — otherwise drop it. - Upserts tags into `Tag` and rows into `_ItemTags`. - **Rewrites `caption` in place** to a tight, readable version that preserves the user's meaning but drops filler, repetition, and false starts. When a `pendingNote` and/or fresh `voiceNote` transcript is present, *merge* its content into the new caption — caption is the one canonical short note for the item, so it should reflect everything known. If the cleanup would change the meaning, prefer to keep the ambiguity rather than guess. - **Clears `pendingNote` to NULL and sets `needsAIReview = false`** for every item it processed, so the same correction isn't picked up again next pass. The audio file at `voiceNotePath` is *kept* on the item for replay/verification. - Rebuilds `searchText` for each touched item so fuzzy search finds them. 5. Reports back with a table of identified items. #### The canonical batch-update shape When applying identifications, use one transaction per session. The pattern that works (verified against this schema): ```sql BEGIN; -- Tags (idempotent) INSERT INTO "Tag" (id, name) VALUES (gen_random_uuid()::text, 'coffee'), (gen_random_uuid()::text, 'pantry') ON CONFLICT (name) DO NOTHING; -- For each item … UPDATE "Item" SET name = '…', brand = '…', category = '…', description = '…', caption = '…', -- merged from old caption + pendingNote + voice transcript "pendingNote" = NULL, "needsAIReview" = false WHERE id = ''; INSERT INTO "_ItemTags" ("A","B") SELECT '', id FROM "Tag" WHERE name IN ('coffee','pantry') ON CONFLICT DO NOTHING; -- Rebuild searchText for everything touched UPDATE "Item" i SET "searchText" = LOWER( COALESCE(i.name,'') || ' ' || COALESCE(i.description,'') || ' ' || COALESCE(i.category,'') || ' ' || COALESCE(i.brand,'') || ' ' || COALESCE(i.caption,'') || ' ' || COALESCE(l.path,'') || ' ' || COALESCE((SELECT string_agg(t.name,' ') FROM "_ItemTags" it JOIN "Tag" t ON t.id = it."B" WHERE it."A" = i.id),'') ) FROM "Location" l WHERE l.id = i."locationId" AND i.id IN ('', '', ...); COMMIT; ``` Run it via: ```bash ssh root@etpics.com 'docker exec -i eorganize-db psql -U eorg -d eorgdb' < batch.sql ``` ### Adding info to an already-identified item On the item detail page (`/items/`) there are two affordances: - **Voice note block (top of the item)** — plays the current voice note (if any). Tap **Re-record** to replace it with a fresh recording; the old file is deleted and `needsAIReview` is flipped to `true`. If the item has no voice note yet, the block shows a **Record voice note** button. - **"Add a note for the next AI pass" panel (below)** — type or dictate via iOS keyboard mic. Saves to `Item.pendingNote` and flips `needsAIReview` to `true`. Either action puts the item back in the Phase 2 queue. After Claude Code processes it, `pendingNote` is cleared and `needsAIReview` goes back to `false`. The voice file at `voiceNotePath` stays on the item — it's canonical, not intermediate. This is the way to say things like "actually this is the 2014 model" or "the matching battery lives in the bin under it" without recapturing the item. ## Phase 3 — AI search (interactive) The `/search/ai` page lets the user describe something in natural language (typed and/or voice) and get back a ranked list of inventory matches. Each submission creates an `AISearchSession` row; the VPS SSHes the home Mac to launch `scripts/run-ai-search-local.sh`, which posts back to `/api/ai-search-session//status`. When invoked with a session id, Claude Code should: 1. **Read the session** from the production DB: ```sql SELECT id, query, "audioPath", "audioMime" FROM "AISearchSession" WHERE id = ''; ``` 2. **Transcribe `audioPath` if set** — same whisper.cpp flow as Phase 2: ```bash curl -sL "https://eorganize.etpics.com/api/uploads/" -o /tmp/q.webm ffmpeg -y -i /tmp/q.webm -ar 16000 -ac 1 -f wav /tmp/q.wav 2>/dev/null whisper-cli -m /tmp/whisper-models/ggml-base.en.bin -f /tmp/q.wav --no-timestamps ``` POST the cleaned transcript back immediately so the page shows it while the search runs: ```bash curl -X POST "$CALLBACK_BASE/api/ai-search-session//status" \ -H "authorization: Bearer $(cat ~/.config/eorganize/callback.secret)" \ -H "content-type: application/json" \ -d '{"status":"RUNNING","transcript":"…"}' ``` 3. **Combine `query` + transcript** into a single description of what the user is looking for. Strip filler the same way Phase 2 does. 4. **Pull the inventory** the search can match against. Cast a wide net — active items only, but include all the columns + tags + location.path that contribute to recognition: ```sql SELECT i.id, i.name, i.category, i.brand, i.description, i.caption, l.path AS location, (SELECT string_agg(t.name, ',') FROM "_ItemTags" it JOIN "Tag" t ON t.id = it."B" WHERE it."A" = i.id) AS tags FROM "Item" i JOIN "Location" l ON l.id = i."locationId" WHERE i.status = 'ACTIVE'; ``` 5. **Pick the best matches** — up to ~10, ranked best-first. Weight the user's specific wording over generic adjectives ("hedge trimmer" should not match every garden tool). When no item is a plausible match, return an empty list and explain in `reasoning`. 6. **POST the final result** as SUCCEEDED with the ranked ids: ```bash curl -X POST "$CALLBACK_BASE/api/ai-search-session//status" \ -H "authorization: Bearer $(cat ~/.config/eorganize/callback.secret)" \ -H "content-type: application/json" \ -d '{"status":"SUCCEEDED", "reasoning":"matched on hedge + cordless", "resultItemIds":["",""]}' ``` The page polls every 3s and renders the matching items via `ItemRow` once SUCCEEDED lands. Order matters — ids are rendered top-down in the order Claude returned them. If anything blocks progress (audio unreachable, whisper missing, no items match plausibly), POST FAILED with a one-line `errorMessage` rather than hanging. ## Triggering AI pass from the app The **Start AI pass** button on `/identify` SSHes from the VPS app container to the home server and runs `scripts/run-identify-local.sh`. The wrapper detaches, runs `claude -p` headless against the Phase 2 instructions, and posts status back to the VPS at `/api/ai-review-session//status`. ### One-time setup **On the VPS host** (`root@etpics.com`): 1. Generate a dedicated SSH key for the app: ```bash mkdir -p /home/eorganize/identify_keys ssh-keygen -t ed25519 -N '' -C 'eorganize-identify@vps' \ -f /home/eorganize/identify_keys/identify_to_home ssh-keyscan -p 2222 -H 67.182.44.118 > /home/eorganize/identify_keys/known_hosts # The container runs as uid 1001 (`nextjs`). Grant it read access: chown -R 1001:1001 /home/eorganize/identify_keys chmod 700 /home/eorganize/identify_keys chmod 400 /home/eorganize/identify_keys/identify_to_home \ /home/eorganize/identify_keys/identify_to_home.pub chmod 444 /home/eorganize/identify_keys/known_hosts ``` 2. Confirm `IDENTIFY_CALLBACK_SECRET` is set in `/home/eorganize/public_html/.env` (the deploy script writes one on first run). Copy that value — the home server needs it. **On the home server** (`erictran@67.182.44.118:2222`): 1. Clone the repo: ```bash mkdir -p /Users/erictran/Developer git clone https://github.com//eorganize.git \ /Users/erictran/Developer/eorganize ``` 2. Install prerequisites if missing: ```bash brew install ffmpeg whisper-cpp libpq # libpq gives you psql echo 'export PATH="/opt/homebrew/opt/libpq/bin:$PATH"' >> ~/.zshrc ``` 3. Log into Claude once interactively so `claude -p` works headlessly later: ```bash claude # sign in, then exit ``` 4. Drop the callback secret somewhere the wrapper can read: ```bash mkdir -p ~/.config/eorganize printf '%s' '' \ > ~/.config/eorganize/callback.secret chmod 600 ~/.config/eorganize/callback.secret ``` 5. Authorize the VPS's app key. Paste the contents of `/home/eorganize/identify_keys/identify_to_home.pub` (from the VPS step above) into `~/.ssh/authorized_keys` with a `command=` lock so the key can only run the wrapper: ``` command="/Users/erictran/Developer/eorganize/scripts/run-identify-local.sh",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty ssh-ed25519 AAAA... eorganize-identify@vps ``` The forced command runs the wrapper unconditionally; the wrapper reads the session id from `$SSH_ORIGINAL_COMMAND` (which is whatever the app sent as the SSH command — a bare UUID). ### Sanity check From the VPS, exec into the app container and try the round-trip: ```bash docker exec -it eorganize-app sh -c \ 'ssh -i /etc/eorganize/identify_to_home -p 2222 \ -o UserKnownHostsFile=/etc/eorganize/known_hosts \ erictran@67.182.44.118 not-a-real-uuid' ``` You should see `usage: …run-identify-local.sh ` on stderr and ssh exits 2. That confirms (a) the key auth works, (b) the `command=` lock is engaged and runs the wrapper, and (c) the wrapper is rejecting bogus input. Now click **Start AI pass** on `/identify` to do the real round-trip. ### Where things land on the home server - Wrapper script: `/Users/erictran/Developer/eorganize/scripts/run-identify-local.sh` - Per-run log: `~/Library/Logs/eorganize-identify-.log` - The last 4KB of the log gets POSTed back as `logTail` so it shows up under the run history on `/identify`. ### AI search wrapper (parallel setup) `/search/ai` follows the exact same pattern as the identify pass, but with a separate key + forced command so the two flows can't accidentally cross wires. **On the VPS host**, repeat the key gen with a different filename: ```bash ssh-keygen -t ed25519 -N '' -C 'eorganize-ai-search@vps' \ -f /home/eorganize/identify_keys/ai_search_to_home chown 1001:1001 /home/eorganize/identify_keys/ai_search_to_home* chmod 400 /home/eorganize/identify_keys/ai_search_to_home \ /home/eorganize/identify_keys/ai_search_to_home.pub ``` The Docker compose mounts the whole `identify_keys` directory into the container at `/etc/eorganize/`, so the new key shows up at `/etc/eorganize/ai_search_to_home` automatically — no compose changes needed. **On the home server**, add a second `authorized_keys` entry pointing at the AI search wrapper: ``` command="/Users/erictran/Developer/eorganize/scripts/run-ai-search-local.sh",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty ssh-ed25519 AAAA... eorganize-ai-search@vps ``` Per-run log lands at `~/Library/Logs/eorganize-ai-search-.log`. Same callback secret (`IDENTIFY_CALLBACK_SECRET`) authenticates both endpoints — single-user infra, single token is fine. ## Auth `BYPASS_AUTH=true` is set in the VPS `.env`, which makes every request resolve to the first seeded user. Useful for single-user mode; flip it to `false` and restart the app container to re-enable real login. ```bash ssh root@etpics.com 'sed -i "s/^BYPASS_AUTH=.*/BYPASS_AUTH=false/" /home/eorganize/public_html/.env \ && docker restart eorganize-app' ``` ## Deploy `./deploy.sh` from the repo root. It rsyncs to `root@etpics.com:/home/eorganize/public_html`, rebuilds the image, force-recreates the containers, runs `prisma migrate deploy`, then `node prisma/seed.mjs`. The script is idempotent. The nginx container publishes on host port `9543`. Apache (cPanel) terminates TLS for `eorganize.etpics.com` and proxies to `127.0.0.1:9543`. The Apache include is at: ``` /etc/apache2/conf.d/userdata/ssl/2_4/eorganize/eorganize.etpics.com/proxy.conf ``` If you add a sibling app, mirror that file and pick a new host port. ## Backups Nightly backups run at **03:50** from root's crontab on the VPS: ``` 50 3 * * * /home/eorganize/public_html/scripts/backup-production.sh >> /home/eorganize/backup.log 2>&1 ``` The script (`scripts/backup-production.sh` in this repo) `pg_dump`s the DB, tars the uploads volume, rsyncs both to the home Mac (`erictran@67.182.44.118:2222`) using `/root/.ssh/backup_to_home`, deletes the local staging copies, and prunes old remote files. Layout matches the other apps: - DB → `/Volumes/ExtraSSD/webapp_backup_dbs/eorganize_prod_.sql.gz` - Uploads → `/Volumes/ExtraSSD/webapp_backup_attachments/eorganize/eorganize_uploads_.tar.gz` Retention is **14 most recent dailies + 8 weekly snapshots** beyond that — roughly two weeks of fine granularity plus two months of weekly checkpoints. Pruning happens from the VPS over ssh after each successful upload. To run on demand: `ssh root@etpics.com /home/eorganize/public_html/scripts/backup-production.sh`. ## Known non-features - **No production AI.** Identification is a manual Claude-Code session on the dev Mac. Photos are reviewed visually; voice notes are transcribed locally via whisper.cpp. Nothing AI-shaped runs on the VPS. - **No QR/NFC tag scanning.** Locations are typed freeform with the smart-autocomplete (substring + segment-aware). Add scanning only if the typing UX actually proves slow at scale. - **No multi-tenant.** Multi-user works (via `BYPASS_AUTH=false`), but items are global — anyone signed in can see everything.