Western open-weight models still can’t beat China

A slew of new open-weight artificial intelligence models from American and European companies aim to convert users of Chinese open models who are concerned about security, but fall short of the dominant Chinese systems on performance despite having greater resources, experts said at a Rest of World event in San Francisco.   

Last week, American startup Reflection launched its first open-weight model, and European labs Aleph Alpha and Mistral also released new open-weight models. Open models such as these offer users greater customization and control than the closed models of OpenAI, Anthropic, and Google at significantly lower costs.

For businesses and governments in lower-income countries, in particular, cost is the primary factor, as well as the expertise needed to customize open models for their specific needs, including in areas such as education, healthcare, and farming, Daniel Björkegren, an assistant professor at Columbia University, said at the Rest of World event last week.

“In the low-income context, there’s a lot of concern about being dependent on the U.S. for frontier intelligence,” he said. “American models are also very tuned for American workloads, but when you take those same models and try to deploy them in remote parts of the world, often they don’t work so well.”

Users, particularly businesses, switch models “when new capabilities and lower prices emerge,” Björkegren said. But “one enduring question is, how easy is it to switch?”

Since the launch of DeepSeek early last year, Chinese open models have rapidly closed in on their U.S. rivals. A growing number of U.S. startups and businesses, as well as the European Union’s sovereign AI project, use them. The OpenRouter leaderboard of large language models had four Chinese models among its top five last week — DeepSeek, Z.ai, XiaomiiXiaomiXiaomi is a Chinese consumer electronics company that leads the country in smartphone manufacturing and sales., and TencentiTencentBest known for its super-app WeChat, Tencent is a Chinese technology conglomerate and a major player in the video gaming industry.. LiveBench, which ranks models on various benchmarks, recently had DeepSeek in the top five for the first time, and showed that the top Chinese models lagged behind their U.S. rivals by just 3% on benchmark scores.

Critical to China’s ambition

China’s leader Xi Jinping has described open models as key to the country’s ambition to shape the development of the technology, and reach its goal of “global AI leadership” by 2030. This year, China established the World Artificial Intelligence Cooperation Organization to promote AI adoption and shape AI governance in developing nations. That includes access to Chinese models, and training for students and workers.

China’s plan for economic progress is through innovation, and open-weight models give it an opportunity to get around some of the concerns about its presence in overseas markets, Kristy Loke, a research fellow at the independent AI safety and security program MATS, said at the Rest of World event.

“China has, for a long time, talked about open innovation and contributing to global innovation, and they weren’t able to do that because in some cases, like Huawei, the U.S. pushed back,” she said. “With open models, because people can download them and use them and own them, it’s private to them — so you kind of break down the trust problem when it comes to concern over the Chinese state.”

“It’s good for soft power. It’s good domestically, as well,” Loke said.

The directive from the U.S. government to Anthropic earlier this year to suspend access to its Fable 5 and Mythos models for all foreign nationals on national security grounds also reinforced to several governments that they should not depend entirely on American systems, Björkegren said.

“If your digital infrastructure is dependent on a foreign country, and for geopolitical reasons access could be pulled, that is really problematic,” he said. “Strategically, a government is going to be in a better position if there are more providers from different countries that have diverse interests, so that you’re not going to be subject to a holdup problem.”

A growing number of instances of AI agents at OpenAI, Anthropic, and Meta hacking websites and taking other unauthorized actions has shifted the narrative around open models being less safe. Hugging Face said it used a Chinese open-weight model while  defending itself from a swarm of AI agents from OpenAI because American models refused its prompts.

Trade restrictions that limit China’s access to the most advanced chips from Nvidia have forced Chinese companies to build more efficient models that require less computing power. American open models from Meta, Nvidia, and others do not come close in capability, Graham Webster, a research scholar and editor-in-chief at the DigiChina Project at Stanford University, said at the Rest of World event.

“If you need relatively frontier capability, right now that’s Chinese open models only,” he said. “If there were U.S.-made open models that were at such high performance as the Chinese open models, you might observe whether people had a preference for a U.S.-made open model. But you don’t see that because the U.S. companies just don’t release those.”

“Competition is hard“

Last week, as Reflection announced the launch of its first open-weight model, Nathan Lambert, head of AI research lab Trillium Labs, wrote on X: “Reflection joins the list of Nvidia & Thinking Machines who have released their strongest models and come up behind Chinese counterparts. There’s a lot of ways people will overthink this, but the clearest takeaway should be that the Chinese are very very good at building LLMs.”

When the head of product at Reflection responded that it was their first product, Lambert wrote: “The point is that some of the best and most funded researchers in the U.S. may not win. Competition is hard.”

Still, Chinese AI companies are grappling with compute limits and the question of how to make money. Their open models can do some tasks 50% to 90% cheaper than the leading American frontier models. Moonshot AI is negotiating revenue-sharing deals with Microsoft, Amazon, and Google to host its Kimi K3 model — a rare deal between a Chinese AI firm and U.S. cloud providers.

The Chinese government will be keen to maintain the lead, Webster said.

“The government would prefer that China be home to the leading open-weight model,” he said. “They can throw more money at things, but you can only buy so many chips under current conditions. The Chinese companies have been doing a very good job of figuring out how to do more with less. But they may run into a wall.”

Monetization is a question that concerns every Chinese AI company, Loke said. “The hope is to get as much traction and adoption, and that once people like your model enough, hopefully, they stick with you,” she said.

For the government, enabling Chinese open models to thrive locally and globally is critical. “Strategically, it makes a lot of sense for them to win hearts and minds, win markets,” Loke said.