Report identifies 'sweet spot' for enterprise LLMs

Smaller, specialized large language models are emerging as a cost-effective option for enterprise applications, with a new industry report finding that they can deliver performance close to much larger models at a fraction of the cost.

Drawing on more than 120 enterprise AI deployments and interviews with chief technology officers at 15 companies, a CCID Consulting report found that larger models do not always translate into better business results. Tests across four enterprise tasks showed that performance gains flatten as model size grows, particularly in structured tasks.

The report identified models with 7 billion to 30 billion parameters, fine-tuned with domain-specific data, as the "sweet spot" for enterprise LLMs.

"They can deliver 90 to 95 percent of the performance of models with more than 100 billion parameters at just 5 to 10 percent of the cost," it said.

In one case cited by the report, a global bank cut cloud inference spending by 63 percent after switching from a 175-billion-parameter open-source model to a 13-billion-parameter specialized model, without measurable loss of accuracy.

The consultancy describes the emerging approach as "precision intelligence" — combining domain-specific models with retrieval-augmented generation, external tools, and human review. It aims to match model capabilities to specific tasks while balancing performance, cost, and compliance, rather than relying on parameter size alone.

The economics of inference are becoming more important as AI moves from model development to large-scale deployment. In an October report, McKinsey projected that inference would account for about 60 percent of AI computing demand by 2030, compared with about 40 percent for training. As AI use expands, the cost of running models at scale is becoming a central consideration for companies.

China's LLM development has followed two main tracks — improving technical efficiency and expanding industrial applications, Bai Runxuan, an analyst at CCID Consulting's AI and big data research center, said.

"Compared with overseas approaches that place greater emphasis on scaling up model parameters, China's strengths lie in its broad range of application scenarios and mature engineering capabilities," Bai said.

By the end of August, 1,112 generative AI services had completed filing in China, while 731 generative AI applications or functions had been registered, according to the Cyberspace Administration of China.