Ghostclick

Repetitive web tasks are an obvious thing to automate, but assistants that automate them usually work by sending what's on your screen to a server you don't control.

Ghostclick takes the other route. It's a privacy-first Chrome extension: you perform a task once while it records, and it replays that task when you ask. The model behind it runs on the device through WebLLM, so nothing is sent to a server.

These are the questions I'd ask of any record-and-replay tool, and the reason this project sits next to my LLM security research.

• Local inference removes a data flow. With no server in the loop there's no remote copy of page content to retain or leak. That is the privacy argument for running the model on the device.
• A recorder sees what the user sees. Anything on the page or typed during a recording is in scope, so what gets kept, and for how long, is the central design question.
• A replay acts as the user. It runs inside whatever sessions the browser is signed in to, so a replay should only run when the user asks for it, which is how Ghostclick works.
• Page content is untrusted input. Any model that reads a web page can be steered by text planted on it. That's the same vulnerability class the WebRedTeam work tests for.

Ghostclick is a team project. The code isn't in one of my public repositories, so there's no repository link here, and there are no published measurements to report.

I'm glad to walk through the design and my part in it in a conversation.

Get in touch