Chinese researchers build military AI using Meta’s open-source Llama model — ChatBIT allegedly performs at around 90% of the performance of OpenAI GPT-4 LLM

Chinese researchers with ties to China’s People’s Liberation Army (PLA) have built an AI model called ChatBIT, designed for military applications using Meta’s open-source Llama model. According to Reuters, some researchers are associated with the Academy of Military Science (AMS), the PLA’s top research group.

Three academic papers and several analysts have confirmed the information, with ChatBIT using Meta’s Llama 13B large language model (LLM). This LLM has been modified for intelligence gathering and processing, allowing military planners to use it for operational decision-making.

According to one of the papers that Reuters cited, the military AI is “optimized for dialogue and question-answering tasks in the military field.” It also claimed that ChatBIT performs at around 90% of the performance of OpenAI’s GPT-4 LLM, although the paper did not reveal how they tested its performance or say if the AI model has been used in the field. Nevertheless, its use of open-source AI models could potentially allow it to match the latest models released by American tech giants in benchmark tests.

“It’s the first time there has been substantial evidence that PLA military experts in China have been systematically researching and trying to leverage the power of open-source LLMs, especially those of Meta, for military purposes,” says Jamestown Foundation Associate Fellow Sunny Cheung, a Washington, D.C.-based think tank that looks at China’s emerging and dual-use technologies, including artificial intelligence. Meta’s license explicitly bans Llama’s use for military applications, but its open-source nature makes it nearly impossible to enforce such limits.

However, Meta said in a statement that this alleged use of the Llama 13B LLM — which it says is an “outdated version” given that it’s already training Llama 4 — is largely irrelevant, especially given that China is investing trillions of dollars to gain an edge in AI technologies. Furthermore, other researchers noted that ChatBIT only used 100,000 military dialogue records, a drop in the bucket given that the latest models are trained on trillions of data points.

Some experts question the viability of such a small data set for military AI training. But ChatBIT could also just be proof of concept, with the involved military research institutes planning to create more expansive models. Aside from that, the Chinese government might have released these research papers as a sign to the U.S. that it is not afraid of using AI to give it a technological advantage on the global stage.

No matter how big or small this development is, Washington has been afraid of this news — the use of American open-source technologies that will give its opponents a military advantage. That’s why, aside from expanding ongoing export controls in China, many U.S. lawmakers also want to block the country from accessing open-source/open-standard technologies like RISC-V. It’s also taking steps to stop American entities from investing in Chinese AI, semiconductors, and quantum computing.

This is the two-edged sword that American policymakers must contend with. They naturally don’t want to give opponents access to advanced technologies via the open-source route; however, open-source technology is also a major driver of technological advancements, and curbing it could put U.S. companies at a disadvantage.