ChatGPT can now remember what you did on your Mac — without screenshots

OpenAI is launching a new feature for ChatGPT Work and Codex on macOS that sounds quite useful but may also make you feel a bit uneasy.

Computer History is a new optional feature for Pro, Business, and Enterprise users that, with your permission, can watch your day-to-day activity on your computer across apps and websites and turn that into memories and a timeline.

This, OpenAI says, will then allow ChatGPT to answer questions like “What was I debugging yesterday?” or “Where did I leave off on that PR?” — all without having to re-explain the full context of your query. Based on your history, ChatGPT will then be able to find the files, documents, and conversations relevant to your query, no matter whether they were in Google Docs, Slack, or an open browser tab.

Of course, this will also be useful for when you know you read something but have no idea where.

Neither Recall nor Chronicle

It’s important to note that OpenAI says it is doing all of this without constantly taking screenshots of your computer. The company clearly learned from Microsoft’s Recall disaster. Instead, as OpenAI’s Dominik Kundel notes in the announcement video, “Computer History doesn’t rely on screen or audio capture. Instead, it captures interaction events like clicking, typing, app switches, and more, so it can capture memories faster and more efficiently.”

“Computer History doesn’t rely on screen or audio capture.” — Dominik Kundel, OpenAI.

Computer History also different from OpenAI’s previous Chronicle research experiment, which had the same goal but used screen captures and OCR to generate its timeline, as well as Record & Replay, a feature for ChatGPT and Codex that lets users demonstrate a workflow and turn it into a reusable skill.

Keeping the user in control

With Computer History, users will get full control over which apps and websites Computer History will have access to. On macOS, there will be a new menu bar entry that allows you to exclude any app from this feature, as well as a new setting in the ChatGPT desktop app that lets you exclude apps and websites.

The feature is off by default, and you can always pause it from the menu bar, too.

All of this data is stored locally, so you can always see what the system captures and delete it if you don’t want it to be part of the timeline.

“Once your agents understand what you do and how you work, it changes how you can engage with them. You can ask it to create skills based on what you just did, run an automation every morning to draft a standup update for your team, or simply tailor its use of tools to how you work,” Kundel says.

The fact that the actual user data stays local and doesn’t involve screenshots will surely assuage some privacy concerns. But the model will still get access to the summaries and that may be enough of a concern to users — and their employers — to make this a feature that many will prefer to leave in the ‘off’ setting.

Availability

Computer History is now rolling out to all Pro, Business, and Enterprise users. For the most part, this is a global rollout, but OpenAI notes that users in the EEA, UK, and Switzerland will only get access “in the coming weeks.”

Before joining The New Stack as its senior editor for AI, Frederic was the enterprise editor at TechCrunch, where he covered everything from the rise of the cloud and the earliest days of Kubernetes to the advent of quantum computing….

Read more from Frederic Lardinois