Tech Innovator Claims Major AI Labs Misunderstand User Needs

Tim O’Reilly’s benchmark for assessing the value of a company, individual, or society has consistently been create more value than you capture. It’s hardly surprising that O’Reilly—an influential publisher, internet pioneer, venture capitalist, conference organizer, and tech thought leader—is applying this principle to the design and usage of AI. He advocates for a future wherein open-source AI becomes accessible to everyone. His concern is that, similar to Microsoft in the ’90s, today’s major tech firms are attempting to entrap users within their ecosystems. Thus, he is championing initiatives to open-source AI technology—not just by making essential technical elements like neural-network weights available, but also by decentralizing the entire AI system architecture, empowering designers and users alike.

O’Reilly views AI as a novel creative medium that he engages with extensively—and even maintains a blog documenting his interactions with it. In our discussion, we found ourselves at odds regarding AI’s capacity to generate original content. Try to guess who sided with which perspective.

STEVEN LEVY: You’re fully committed to open-source AI. Present your argument.

TIM O’REILLY: First, let’s ensure we’re on the same page. When most people refer to open-source AI, they usually mean open-weight models. It encompasses much more than that. Back in the ’90s, while others fixated on open-source licenses, I thought, “No, it’s about the system’s architecture. Does it facilitate participation?”

Why is this necessary?

The major labs are misreading the future. They’ve constructed a narrative that insists having the largest, most advanced model is the ticket to success. Large models like Claude may optimize for specific use cases, yet they don’t always align with what people truly desire. I want to incorporate my unique contributions. The key is to maintain a clear distinction between the model, the framework, and the application. Right now, we’re lacking that clarity. They’ve designed an architecture that emphasizes control instead of fostering freedom and participation, enabling them to monitor users.

Isn’t it counterproductive for large companies to relinquish control?

Absolutely, it’s against their interests. However, that doesn’t imply they are making the right long-term choices. For a significant time, the latest high-performing models were superior across all dimensions. Now, they excel in some areas while underperforming in others. There’s much discussion surrounding why Fable and Sol are considered inferior writers compared to lower-level models. [Note: Companies like Anthropic and OpenAI might take issue with this claim.] The advancements we see in so-called frontier AI could actually steer us further from the genuine needs of everyday users. We might triumph in frontier AI within the US, yet China could outpace us due to their widely distributed lower-level models. The objective is to empower individuals to innovate freely, to create beyond conventional boundaries.

Some express concerns that open-source software poses security risks, as malicious entities might bypass the boundary protections of frontier models.

The cybersecurity incidents we’ve observed all stem from the frontier models. Thus, issues like cybersecurity and the potential development of harmful technologies actually support the argument for tempering the frontier models more than for restricting open-weight models.

So, do you believe that open source will diminish the dominance of AI powers?

I can’t predict the future, but I can say the world is evolving in a manner I had hoped for. What if the prominent frontier models turn into the modern equivalent of mainframes or supercomputers—focused on complex challenges that won’t necessarily permeate society at large? People are creating platforms like Pi, an open-source [agentic] framework. One initiative we are developing at my nonprofit, the AI Disclosures Project, is the concept of an open-memory consortium. Mark Zuckerberg’s strategy seems to be locking users in by asserting that Meta will provide the AI that understands you best. The vision for open source should reject this notion. It can offer the flexibility to switch models, change providers, and retain all necessary context.