AnswersPublished20 sources
Cheap AI Tokens, Expensive Compute: Why the AI Boom Is Becoming a Margin Squeeze
AI inference prices have fallen roughly 95% for comparable capability, while semiconductor and financing costs are rising. Nvidia’s fiscal Q2 2027 revenue reached $96.2 billion, up 106%, with roughly 70% fiscal 2028 growth expected—but higher memory costs are already weighing on future margins.
Published byEdited with GPT-5.6 LunaImages generated with GPT Image 1.5
Research answer
Key takeaways
- AI inference prices have fallen roughly 95% for comparable capability, while semiconductor and financing costs are rising.
- Nvidia’s fiscal Q2 2027 revenue reached $96.2 billion, up 106%, with roughly 70% fiscal 2028 growth expected—but higher memory costs are already weighing on future margins.
- Broadcom’s proposed AI chip financing is still under negotiation, not a completed transaction, but its potential $60 billion plus debt structure shows how falling token prices can become a credit risk.
AI Prompt
openai.comCreate a landscape editorial hero image for this Studio Global article: How is the AI market being squeezed as token prices fall roughly 95% since early 2024—illustrated by the free near-frontier Ox Alpha model a. Article summary: The AI economy is experiencing a margin-and-financing squeeze: model capability is becoming commoditized faster than the physical infrastructure needed to produce it. That favors scarce-hardware suppliers in the near ter. Topic tags: general, news, general web, user generated. Style: premium digital editorial illustration, source-backed research mood, clean composition, high detail, modern web publication hero. Use reference image context only for broad subject, composition, and topical grounding; do not copy the exact image. Avoid: logos, brand marks, copyrighted characters, real person likenesses, fake screenshots, UI text, readable text, watermarks, charts w
The AI market is being pulled in opposite directions. The price of using advanced models is collapsing: a free model called Ox Alpha reportedly appeared near the frontier, while OpenAI cut prices on two GPT-5.6 tiers by 80% and 20% in July. 17
16 Broader estimates put the decline in inference costs at roughly 95% for comparable capability over the past two years.
22
At the same time, the physical systems required to deliver those tokens are becoming more expensive. Samsung raised prices for some advanced chipmaking orders by as much as 15%, and it expects the global memory shortage to intensify and last into 2028. 32
33 That creates an unusual economic pattern: output prices are deflating while critical input costs remain inflationary.
The AI value chain is splitting in two
The most important distinction is between the commoditizing intelligence layer and the supply-constrained infrastructure layer.
General-purpose model access is becoming easier to substitute. If one provider cuts its API prices, releases a smaller model, or offers a capable model for free, competitors face pressure to respond. OpenAI’s July pricing changes reduced GPT-5.6 Luna’s input price from $1 to $0.20 per million tokens and cut Terra’s price from $2.50 to $2. 16
Infrastructure is different. Advanced fabrication capacity, high-bandwidth memory, networking, power, and complete AI server systems cannot be expanded as quickly as software prices can be reduced. Samsung has said that customers are seeking multiyear supply agreements and that shortages could become more acute through 2028. 33 Reported plans for Nvidia-based server price increases of more than 15% show how suppliers may try to pass those costs downstream, although the report is not an official Nvidia pricing announcement.
36
For AI-service providers, the commercial challenge is straightforward: revenue per token can fall even when usage rises, while the cost of capacity, energy, hardware, and financing may not fall with it.
What Nvidia’s results reveal
Nvidia’s fiscal second-quarter 2027 results show that the infrastructure boom is still exceptionally strong. The company reported $96.2 billion in quarterly revenue, up 106% year over year, and adjusted earnings per diluted share of $2.22. 60 Management also indicated that fiscal-2028 revenue could grow by roughly 70%.
51
Those figures are not evidence that the entire AI economy has solved its profitability problem. They show that demand for scarce accelerator systems remains powerful. Nvidia is positioned at a bottleneck where customers are still competing for supply.
But rising memory costs are beginning to affect the quality of that growth. Nvidia guided to gross margins of about 74% for the following quarter, down from 75% in the reported quarter, and reporting on the outlook linked pressure partly to memory prices. 51 The implication is not necessarily a collapse in Nvidia’s economics. Rather, future performance may depend more on price increases, system-level integration, networking, software, and supply-chain execution—not only on selling scarce chips at high margins.
Broadcom turns infrastructure demand into a financing test
Broadcom illustrates a different part of the squeeze. Reuters and Bloomberg reported that the company was in talks to raise more than $60 billion in debt for an AI-chip financing arrangement benefiting Anthropic and other customers. The structure could include roughly $30 billion of junior debt alongside a senior-secured tranche, with terms still being negotiated. 1
2
That distinction matters: this was a proposed financing, not a completed borrowing. The reported figures should therefore be treated as a range under discussion rather than a finalized transaction.
The economic question is whether the AI revenue generated by those chips will be durable enough to support debt service. A chip can be technically valuable while still producing a poor leveraged investment if model prices fall, customer concentration is high, equipment values decline, or refinancing becomes more expensive.
This is why credit markets matter. Once infrastructure is funded with substantial debt, falling token prices are no longer just a pricing issue for software companies. They can affect the expected cash flows supporting leases, special-purpose vehicles, suppliers, and lenders.
Hyperscalers have advantages—but not immunity
Large cloud companies are better positioned than standalone model providers because they can bundle AI with existing cloud services, enterprise software, advertising, data, and distribution. They can also subsidize AI features while monetizing customers elsewhere.
That does not automatically establish an attractive return on every AI investment. Cheap models may increase usage while reducing revenue per unit. Excess capacity could turn AI into a lower-margin feature of cloud computing rather than a separate profit pool. The key metric is not how many tokens are processed, but whether customers pay enough for the resulting productivity, revenue, or cost savings.
What cheaper tokens mean for businesses
For businesses, falling inference prices are genuinely positive. Lower costs make it easier to test AI applications, automate narrow workflows, and use stronger models for tasks that previously failed an economic hurdle.
The caveat is that token costs are only one component of deployment. Businesses still face integration work, data preparation, security controls, governance, evaluation, and change-management costs. A lower API bill does not guarantee a positive return on investment.
The market is therefore moving toward a more demanding question: what measurable business outcome does each unit of AI compute produce? Companies that can connect model usage to revenue or verifiable cost savings should benefit more than those accumulating capacity without a clear use case.
The investment and credit-market read-through
Investors should separate exposure to scarce inputs from exposure to commoditizing model access.
The supply-constrained side includes memory, advanced fabrication, networking, power, and integrated accelerator systems. These businesses can retain bargaining power while customers compete for capacity. Samsung’s pricing actions and shortage outlook are evidence of that current leverage. 32
33
The model-access side faces a harder test. Free or sharply discounted models can expand the market, but they also make it difficult to preserve high prices for undifferentiated inference. Sustainable economics may instead depend on proprietary distribution, data, specialized workflows, contractual demand, or software that captures value beyond the token itself.
Credit investors should focus on contracted cash flows, customer concentration, residual hardware value, debt structure, and refinancing dates. The proposed Broadcom transaction is a useful warning because it shows how rapidly AI infrastructure can become a financing question as well as a technology question. 1
2
The bottom line
The AI boom is not facing one uniform cost curve. Intelligence is becoming cheaper and more abundant, while the physical capacity needed to produce it remains scarce and expensive.
That mismatch helps explain why Nvidia can report extraordinary growth while warning about margin pressure, and why a proposed multibillion-dollar chip-financing structure can attract attention from credit markets even as demand for AI remains strong. The next phase of the market will be decided less by model capability alone than by whether customers can turn cheap intelligence into durable cash flow before expensive, debt-funded compute capacity must be refinanced.
Studio Global AI
Continue your research
This page includes a source-backed answer you can continue inside Studio Global.
People also ask
What is the short answer to "Cheap AI Tokens, Expensive Compute: Why the AI Boom Is Becoming a Margin Squeeze"?
AI inference prices have fallen roughly 95% for comparable capability, while semiconductor and financing costs are rising.
What are the key points to validate first?
AI inference prices have fallen roughly 95% for comparable capability, while semiconductor and financing costs are rising. Nvidia’s fiscal Q2 2027 revenue reached $96.2 billion, up 106%, with roughly 70% fiscal 2028 growth expected—but higher memory costs are already weighing on future margins.
What should I do next in practice?
Broadcom’s proposed AI chip financing is still under negotiation, not a completed transaction, but its potential $60 billion plus debt structure shows how falling token prices can become a credit risk.