This week on the a16z show, Greg Brockman, president and co-founder of OpenAI, made the point that developers have been “retooling the world” to make software more accessible to AI agents. But computer use could offer a simpler path forward.
And for Brockman, simplicity is really the North Star for AI: “We really should be one AI that’s unified, that makes it so easy and smooth for you to be less engaging with the computer and less wrapping yourself around the computer.”
Meanwhile, other AI companies continue to invest in connector infrastructure.
MCP servers, CLIs, APIs, and other agent integrations help AI agents reach more tools, data, and software. But that wide access doesn’t come without some burden — at least that’s what it sounds like in Brockman’s conversation with Ben Horowitz and Erik Torenberg, hosts of the a16z show. As OpenAI’s president explains:
“What if it’s more behaving like a human? Can it just use a computer?”
“People have been building these MCP servers and these CLIs and just really sort of taking the world of software and making it accessible in this almost stilted way that is not really meant for humans,” he said. “It’s like we’re kind of retooling the world.”
How did we get here? “For agentic use cases, it really comes down to the tools,” said Brockman, indicating why MCP servers and other connectors have become so common. But if models can learn to use computers the same way that people do, then perhaps developers no longer need to keep building purpose-built integrations for every application.
As Brockman wonders: “What if it’s more behaving like a human? Can it just use a computer?”
OpenAI has been thinking about computer use since the beginning
As Brockman details on the podcast, the idea traces back to the early days of the AI company when the team laid out a three-step plan during an offsite meeting in November 2015 that the president says the team largely stuck to for the following 10 years.
Around this time, Brockman says the OpenAI team also broached the idea of using reinforcement learning directly against a computer interface:
“We also talked about, what if we could do reinforcement learning where the environment is screen pixels, keyboard, mouse, right? Same interface as a human.”
From where he sits, this could open up the computer for nearly any sort of task. More importantly, Brockman muses that computer use for agents could help the broader industry advance AI without requiring purpose-built integrations every step of the way. Instead of building and maintaining scores of specific connectors, an agent could simply use the same computer interface people do.
“There’s so much software that you don’t even think about,” he told Horowitz and Torenberg. “How much of your life is clicking around menus and typing things into a spreadsheet and things like that? None of that is what we should be doing.”
Instead of building and maintaining scores of specific connectors, an agent could simply use the same computer interface people do.
In fact, Brockman says computer use capabilities are part of why he considers GPT-6 Astra, OpenAI’s newest flagship model, “pretty reasonable to call” AGI.
But connectors aren’t disappearing yet
Though Brockman is bullish on computer use, the rest of the industry isn’t giving up on structured integrations. In fact, some are plowing ahead.
AWS, for example, recently launched a managed consent portal, a managed web experience and session binding endpoint for AgentCore Gateway, a capability of Amazon Bedrock AgentCore that connects agents to external tools and services. On Monday, the cloud company published a detailed walkthrough of the feature.
For example, with the Consent portal, an administrator could configure GitHub and Slack as gateway targets and send the portal URL to developers who can then open the URL, sign in with their corporate IdP, and connect GitHub and Slack as needed, independently.
Meanwhile, even at OpenAI, plugins aren’t going away. Though OpenAI Codex arrived in the browser with a new Chrome extension back in May that allowed agents to operate within live browser sessions and authenticated workflows across multiple tabs, plugins remained the preferred route because they allowed Codex to work directly with services, such as Slack, Gmail, and GitHub, without manually navigating their interfaces.
Brockman told the a16z show hosts that simplicity should be the North Star. Reliable computer use could be one way to get there. But for now, connectors are still hanging in there.
YOUTUBE.COM/THENEWSTACK
Tech moves fast, don’t miss an episode. Subscribe to our YouTube
channel to stream all our podcasts, interviews, demos, and more.
