Dear Aventine Readers, 

As AI companies compete to deliver the best models at the lowest prices, the amount it costs to answer prompts is becoming a sticking point in how profitable these companies can be — one with trillions of dollars riding on it. We look at the tools being used to bring such costs down and where further savings could come from. 

Also in the issue: 

We will not be publishing for the next two weeks, so have a wonderful end of summer and see you after Labor Day. 

Danielle Mattoon 
Executive Director, Aventine

Subscribe

Subscribe to our newsletter and be kept up to date on upcoming Aventine projects

"Every enterprise now is thinking about spend and the value they're getting in exchange for AI," said Sam Altman, CEO of OpenAI, in an interview with CNBC in July, as he announced the launch of his company's most advanced model yet, GPT 5.6 Sol. The new model, he said, was "54 percent more token efficient on agentic coding tasks."  

In more recent AI pricing news, DeepSeek, long known for providing models at low cost, raised its prices significantly this week.

The two announcements underscore perhaps the most consequential question about the economics of AI today: How cheap can it get to both produce and pay for the newest forms of artificial intelligence? If the labs that build AI models can’t cut the cost of delivering their services and are forced to compete aggressively on price, the resulting razor-thin margins could put trillions of dollars of investments at risk.

To understand the dynamics of this question and the technologies in play, Aventine spoke with AI researchers who are exploring the cost savings that could make or break the business model for frontier AI. 

How we pay for AI

Tokens have become the economic underpinning of AI, the driver of cost that makes the technology so expensive to use at scale.

Typically a few characters long, tokens are scraps of text, including spaces, punctuation and other symbols. The phrase, "Sam Altman loves AI" might be broken into five sections as “Sam” “ Alt” “man” “ loves” “AI.” They are the building blocks that large language models use to read and generate information. Processing them is much of what the GPUs in data centers are doing, which is why AI companies charge by the token.

A short back-and-forth with one of OpenAI's models might use a few thousand tokens; output from its most advanced model costs $30 per million tokens, which would mean a few cents for that level of interaction. Complex tasks rack up bills far faster. Jean-Stanislas Denain, a senior researcher at Epoch AI, a nonprofit research institute that tracks AI trends, said that when his organization compares models on tasks such as reverse engineering a piece of software, it might pay $5,000 to $10,000 for the tokens used on a single task. Here, models aren't just generating the text they output but also creating large internal chains of reasoning, all of which consumes tokens.

Over the past few years, the cost of tokens has dropped considerably. According to analysis by AI.cc, a company that aggregates AI models, the average token cost paid by large companies fell to $6.07 per million in early 2026, down 67 percent from $18.40 per million a year earlier. In the meantime, total spending on tokens has grown for two reasons: a dramatic increase in use, and the fact that AI agents and the latest reasoning models chew through far more tokens than chatbots did two years ago. The cost of running frontier-level models has been increasing by a factor of between three and eighteen a year, according to MIT FutureTech, a lab dedicated to studying the trends driving progress in AI.

To fully understand the cost of AI, however, tokens are merely a rough measure; a token from a poor model does not achieve as much as a token from a superior one. "What matters is the inference price to reach a capability level, to perform a task," said Denain. 

And overall spending on delivering inference has soared. Four years ago, when ChatGPT launched, it cost far more to train a large language model than to run it. That dynamic has since reversed. According to the research firm Gartner, spending on training and inference became roughly equal across AI cloud infrastructure in 2025; by 2029 the amount spent on inference is projected to be double what is spent on training. So it is the total cost of inference —- the process of running a large language model to generate outputs — rather than tokens, that is the true measure of AI’s cost. This is the cost AI companies are desperately trying to bring down. 

Finding efficiencies

There are, crudely, three ways to save money on AI inference: make the models more efficient; make more efficient use of the hardware they run on (mostly  GPUs); and make the hardware itself more efficient. Let’s take the technological factors in reverse order.

Historically, GPUs have become more cost-effective over time. Analysis by Epoch shows that the performance of chips per dollar increases by 30 percent each year, and that they become 40 percent more energy efficient, performing more calculations for the same cost of electricity.

Second, the way hardware is used has grown far more sophisticated, said Tim Dettmers, a computer science professor at Carnegie Mellon University. One example, he said, is that in data centers, clusters of GPUs are now dedicated to running specific calculations within an LLM, creating new efficiencies. More savings, he said, come from not moving data unnecessarily. A major source of inefficiency within an LLM is the "key-value cache," a running memory that grows longer as conversations run and tasks mount up. That forces GPUs to move large quantities of data in and out of memory — an energy-intensive process. But software can help. Open-source systems such as vLLM and techniques like FlashAttention let developers use GPU memory more efficiently and cut unnecessary data movement.

The biggest gains, though, come from making the AI model itself more efficient, said Jonny Rosenfeld, who leads the Fundamental AI group at MIT FutureTech. And there is a laundry list of techniques to achieve this, developed from multiple sources including academia, open-source projects and the labs of frontier models. Certain tweaks make it possible for a model to use only the most relevant slices of its working memory rather than all of it — an approach used in Chinese models such as those built by DeepSeek. Other adjustments, including one being developed by Dettmers, compress the memory, so the model has fewer tokens to reprocess each time it revisits what it has already done. Still others shrink the model or reduce how much of it must run for a given task, while retaining performance.

Many of these approaches are public, published in journals or in the technical reports that accompany open-weight models like China's DeepSeek and Kimi K3, said Dettmers, though some are closely guarded methods developed inside companies like OpenAI and Anthropic. "It's hard to know what's happening at the frontier labs," said Denain.

What does it add up to? According to research Epoch published in early 2025, the price of running language models at a fixed level of performance has been falling between 9- and 900-fold a year, depending on the task, with a median of around 50-fold. But that figure, Denain admits, is becoming outdated because the analysis rests on models that predate today's reasoning systems, which consume more tokens than previous ones. Analysis done in late 2025 by MIT FutureTech based on data collected between April 2024 and November 2025, which included the latest reasoning models, suggests the declines in the cost of specific capabilities may be more like a factor of five to ten.

Savings yet to be found 

The big question is whether there is a floor to how cheap inference can get. 

Rosenfeld is optimistic that if there is, we aren’t near it, arguing that we are not even close to the physical limits of hardware efficiency. Many companies are racing to build dramatically more efficient chips, though there has been no game-changing leap yet. Still, he said, "If there is room [for improvement], then we will find it."

In software, the sources of some gains are reaching their limits while others are yet to be discovered. Dettmers spent years working on quantization — reducing the precision of the numbers used to encode a model, like compressing a high-resolution image to shrink a file while barely affecting the way it looks — and estimates that most of the easy savings there are banked. Further gains might be in the order of single percentage points. Other software improvements might still deliver. Distillation — using a large “teacher” model to train a smaller “student” model that is more efficient with almost equivalent performance — remains highly effective. This approach is widely thought to lie behind some of the cheap, capable models from Chinese labs. Research keeps pushing into new areas, too, including fresh ways to compress a model's memory and smarter ways to split up parts of inference.

Even diminishing returns are worth chasing, Dettmers argues. "If you have an algorithm that's 5 percent more efficient,” he said, “you need to build 5 percent fewer data centers."

And yet spending at the frontier keeps climbing, because the hardest problems swallow tokens at a rate that dwarfs any saving per token. Every efficiency gain makes a given capability cheaper, but the ceiling on what it is possible to spend keeps rising.

Commodities and scarcities

Assume for a moment that model capabilities converge, so that using a model from OpenAI is indistinguishable from using one from Anthropic, Google or DeepSeek. That points to a future in which well-established capabilities become increasingly interchangeable. The conclusion, argued by the analyst Benedict Evans among others, is that AI models will be commodities: undifferentiated products whose price is a race to the bottom, meaning that the production of large language models will never be that profitable. 

For now, though, each new generation of a model can undertake more advanced tasks than the last, and those tasks require more tokens. Denain is skeptical that commoditization will play out fully, because there is persistent high demand for cutting-edge abilities. "Ultimately we'll get crazier capabilities, and that's just a huge market," he said. "I don't expect full commoditization."

The result may be not one market but two, moving in opposite directions. Behind the frontier, capability commoditizes quickly: A task only the best model could manage a year ago can soon be done far more cheaply by most models, and it makes sense to assign it to the cheapest provider. At the cutting edge, scarcity will remain an undeniable pull. Frontier models will continue to improve and someone will be willing to pay a premium for the capabilities only they provide.

For companies spending on AI, the important question may be not who sells the cheapest tokens, but what is the cheapest system that can reliably do a particular job. Solving the hardest problems, meanwhile, may continue to command extraordinary sums.

A landslide in California, 2021. AP Photo/Noah Berger, File

Climate change is destabilizing the ground we live on. It’s old news that global warming is changing our atmosphere and oceans. Now, evidence suggests, climate change is transforming land masses as well, leading to destabilizing, dangerous events like landslides. The Atlantic reports on three major factors accelerating this phenomenon. One is the retreat of glaciers, which expose massive and unstable cliffs to the elements in areas including Alaska, Norway, Chile and the Alps. Landslides resulting from these newly exposed and precarious rock faces have buried an entire Swiss village and generated tsunamis. If the past is prelude, geologists predict worse could come. During the planet’s last glacial thaw about 20,000 years ago, landslides are thought to have been orders of magnitude larger than anything we have experienced since. Second is thawing permafrost. As it melts, rock gives way, destabilizing the ground. Disappearing sub-surface permafrost also leaves voids into which surrounding earth collapses. In Alaska, such voids have forced communities to relocate. In Siberia, destabilized ground has led to methane explosions, creating craters that can be 100 feet deep. Third, extreme rainfall, which researchers attribute to climate change, is triggering more — and more severe — landslides. In 2024, Hurricane Helene caused more than 2,000 landslides in a week. A 2018 mudslide in Montecito, California, destroyed 100 homes, killing 23 people. There’s debate about what other effects climate change may have on the ground. There is some evidence it could be a contributor to small earthquakes in the Alps, caused by changes in the weight of ice on the Earth’s crust. Some researchers also believe it could drive volcanic eruptions. The one sure takeaway is that we’re a long way from understanding the full effects of a warming planet. 

Boosting DNA repair could help slow aging. The DNA in a typical cell can be damaged as many as 100,000 times in a single day. The vast majority of that damage is repaired through a multitude of mechanisms, but some is missed and some of the fixes are botched; the residual damage is believed to drive aging. So what if we could improve the quantity and quality of the repairs the body makes? Taking inspiration from longer-living animals and humans, researchers are now investigating which approaches to focus on, reports Nature. Research on bowhead whales, a species that lives for over 200 years and in which cancer rarely appears, shows that they have an unusually accurate mechanism for fixing double-strand breaks in DNA. A protein called CIRBP, which helps cells survive cold-related stress, seems to have a role, and when expressed in human cells, that whale protein has been shown to increase DNA repair. Meanwhile, another gene called SIRT6, known to be involved in aging, is revealing potential routes forward. Beavers carry a version of SIRT6 that differs from the mouse version in subtle ways, yet appears to be linked to extended lifespans. Beavers live for 10–12 years in the wild, whereas mice live a few years at most. Analysis of human centenarians suggests they may carry another superior variant of this gene. And in experiments in which fucoidans — SIRT6-activating compounds found in brown seaweed — were given to mice, DNA repair was improved and lifespans extended. A trial involving humans is now underway. It is, obviously, early days for this line of research, but it is perhaps reassuring to know that lurking within our own cells could be a key to longer and healthier lives. 

Burning iron could be a source of clean energy. You can't set fire to an iron ingot. But pulverize it into a fine powder and it ignites beautifully. (See: fireworks.) That idea is being embraced by university researchers and startups looking to use the metal as a means of creating heat, reports C&EN. Iron burns at about the same temperature as gas, releases roughly twice the energy per unit volume as coal, produces iron oxide rather than CO₂ as it burns, and that byproduct can be converted back into iron — in theory making it an endlessly recyclable fuel, useful as long-duration energy storage. (Other metals burn well too. Aluminum packs nearly twice the energy per volume, but ignites explosively and creates nanoparticles that are hard to capture.) There are sticking points. Today, recycling iron oxide back into iron is done with hydrogen, and since most hydrogen is derived from fossil fuels that negates the carbon savings, at least until green hydrogen becomes cheaper and more plentiful. There are other wrinkles to work out too, such as how to capture iron oxide soot, how to produce iron powder economically and how to create the best conditions for ignition and combustion safely. But startups such as Altiro Energy and FeX Energy in Canada, Fenix Energy in France and RIFT in the Netherlands, as well as the Metal Energy Hub at the Technical University of Darmstadt, are all tackling these problems. RIFT is already heating 500 households from a 1-megawatt pilot boiler in Helmond, and the University of Darmstadt has plans to retrofit a shuttered coal power plant in Berlin for district heating.

An early glimpse of allergy protection from fecal transplants. A recently conducted trial provides the first evidence that bacteria harvested from the stool samples of healthy donors can help treat food allergies. The results of the study, which involved 15 people, were published in Science Translational Medicine. They show that a body of work so far conducted only in mice appears to extend to humans, too. Participants in the trial all had a severe allergy to peanuts and were unable to tolerate more than 100 mg of peanut protein (less than half a peanut) without an immune reaction. For the trial they swallowed odorless, tasteless capsules of gut bacteria from healthy, nonallergic donors. Four months later, six had increased their tolerance: One was able to tolerate 300 mg of peanut protein and five managed 600 mg or more, or roughly two and a half peanuts. The authors speculate that the donor bacteria could have failed to establish in the nine who did not respond, for various reasons — possibly because of gender differences (five of those who responded well were men), or because the doses were given on a single day. The researchers then transplanted gut bacteria from the positive responders into mice, which ended up protecting the animals against an egg protein allergy; bacteria from those who hadn't responded left the mice susceptible. That suggests the approach could work beyond peanuts, providing broader protection for allergy sufferers. A larger study, using a bigger dose spread over a month, is now underway. But don’t hold your breath: A researcher now collaborating with the team told Nature that even with sufficient investment, it could take as long as a decade for the approach to become sufficiently validated and regulated for widespread use.

Inside The Lab Growing Mutant Bacteria To Terraform Earth, from Noema
6,000 words, or about 23 minutes

A popular sci-fi trope involves making the surface of other planets habitable. But what if we need to do that on this planet? This story explores how the Complex Systems Lab at Universitat Pompeu Fabra in Barcelona is exploring just that. The world's drylands, which cover more than 40 percent of our landmass, are home to more than two billion people and up to 44 percent of agriculture. Due to their dry conditions, these areas are at ongoing risk of becoming too arid to support plant life, and it is often not evident this transition is happening until their ecosystems abruptly collapse, causing changes that don't reverse even if rains return. One idea to prevent that is introducing specific soil microbes into dryland to help the ground retain moisture. Developing such microbes starts with a process called hardening, in which samples are subjected to extreme heat, drought and UV light, so survivors are tougher — and possibly genetically mutated. It’s also possible to edit the genomes of the surviving microbes so they secrete hyaluronic acid, a substance that traps moisture and holds it in the soil. In the eyes of Ricard Solé, who leads the Complex Systems Lab, such work is just a case of turbocharging what nature has already provided. That said, releasing engineered microbes into the wild is highly controversial: At the International Union for Conservation of Nature congress last October, a proposed blanket ban on performing such experiments was defeated by a single vote. Yet as many of the scientists featured in this piece argue, commitment to a stable, restorable state of nature may not be realistic. The real question may be how far we go when confronted by that reality.

How rogue officers turned a nationwide camera network into a tool for stalking, from The Washington Post
6,000 words, or about 24 minutes

A company called Flock has built a network of more than 120,000 AI-enabled roadside cameras across the US that collect 20 billion licence plate scans every month. It seems to be a powerful crime-fighting tool: The company claims it was used to catch a person who had abducted a child in Tennessee, and claims that evidence from its system supported more than one million criminal investigations last year. But it’s also ripe for abuse. According to this story, at least 50 law enforcement officers have been accused of or charged with using licence-plate reading technology for unauthorized purposes, with Flock's system involved in 46 of those cases. In at least 26 of the 50 cases, the tool was used to spy on wives, girlfriends, exes, exes' new partners, or women the officers wanted to meet. In one particularly traumatizing case, the police chief of a small town near Atlanta looked up the licence plates of his ex-girlfriend and her teenage daughter roughly 600 times over 13 months, texting the ex-girlfriend  about where she had been — the supermarket, the doctor, a date, a trip out of town — sometimes within minutes. He was arrested on several charges, and later found dead in his home before trial in what officials said was suicide. Flock has argued that misuse is unfortunate and rare but inevitable. Yet it has now also said it is planning better tools to prevent abuse. The larger point, argued by Meredith Whittaker, president of the Signal Foundation to Aventine just last week, is that even well-intentioned surveillance can be abused — and the more pervasive and capable surveillance becomes, the more opportunity there is for bad actors.

Can we finally use bacteria from crime scenes to solve cases? From New Scientist
2,700 words, or about 11 minutes

Every crime scene carries microbial traces — including those of perpetrators and victims. For decades, forensic scientists have been trying to turn them into worthwhile evidence. In theory, we all leave a trail of our microbiome behind wherever we go, and its structure — the relative abundances of one microbe versus another — could be used to place a known suspect at a crime scene. New results suggest that modern techniques based around this idea work better than earlier attempts did, though they are far from being able to match DNA as evidence that can be used in a criminal case. Instead, this approach is being pitched as something closer to fiber evidence: an extra data point that can narrow the field of suspects. More compelling might be the use of so-called microbial clocks to predict time of death. Most time-of-death predictors work well for the first 24 hours, and relatively well for a few more days. But research shows that microbes in a decomposing body increase and decrease in predictable sequence. Microbes could also reveal whether a body has experienced unusual conditions, such as being frozen. One researcher believes microbial clocks are almost ready for use in court, though it will ultimately be a judge who decides whether or not to allow this new evidence.