Screen awareness & visual guidance

How a screen question works

1. You open Screen mode and type. Opening it captures nothing — you get an input bar under Amethyst, and the window stays put.
2. Send captures one screenshot. Exactly at that moment, of the display your cursor is on (with a fallback to your primary display). Every Send takes a new shot; nothing is ever reused from a cache.
3. The answer streams. The screenshot and your question travel through the secure gateway to a vision model; the text reply arrives in the same conversation as everything else.
4. Guidance is grounded. For questions that ask to point or click, a grounding pass returns boxes in the screenshot’s own 0–1000 coordinate space — validated box by box, with area and aspect gates.
5. The overlay draws for eight seconds. One transparent, always-on-top, fully click-through window covers the captured display exactly, then clears itself.

Capture rules that are never crossed

• No capture when Screen mode opens — only on Send.
• No timers, watchers, periodic screenshots, or screenshot history. The code path that captures is the Send handler.
• No files on disk, no database columns, no storage buckets for images — the JPEG lives in memory until the answer completes, then it is released.
• The screenshot feeds the answer and, for grounded questions, the grounding pass (one or two providers see the same bytes). Nothing else.

Known limitations

• One screenshot per Send — the view is not live and does not update while you work.
• Capture targets the display under your cursor; there is no monitor picker yet.
• Amethyst itself is always-on-top, so it may appear in its own screenshot.
• Grounding accuracy is uncalibrated: tiny or dense targets will be missed, and the text-only fallback carries the answer.
• Overlay windows can sit beneath exclusive-fullscreen applications.
• Screen questions require a signed-in session and the vision gateway to be reachable.

DOT404 never clicks, types, or moves anything for you. What the AI can and cannot see