DeepSeek: Implications for DePIN and the broader AI Sector
Article2,883 words
Long-form article on DeepSeek R1's claimed sub-$5M training cost, its MoE and sparse activation design, and what it means for GPU scarcity and decentralized GPU networks like io.net. Covers closed vs open models and DePIN challenges.
Key claims
DeepSeek claims under $5M total training cost vs tens or hundreds of millions for US giants
H100 retails above $30,000
io.net is the only DePIN with geo-distributed GPU clustering
DeepSeek: Implications for DePIN and the broader AI Sector
Over the past years, the field of artificial intelligence has seen a surge in the development and deployment of large-scale language models (LLMs), with systems like GPT, Claude, and Llama having demonstrated remarkable capabilities for generating coherent text, engaging in nuanced conversation, and even performing complex reasoning tasks that were once considered out of reach for machines.
Last week, another major earthquake hit the AI industry. You have probably heard of it by now, but if you haven't: Meet DeepSeek.
DeepSeek R1 is an open-source LLM project developed by a division of High-Flyer, an AI quant fund from China, and has garnered massive attention for claims that it can match or even exceed the performance of leading models while substantially lowering computational requirements, claiming a total training cost of only $5M vs. estimates of 10s or even 100s of millions spent by US tech giants to train their LLMs.
Unsurprisingly, these announcements sent shockwaves through the AI industry and crypto markets, where decentralized physical infrastructure networks (DePIN) such as @ionet promise to reshape who can afford cutting-edge AI by pooling distributed GPU resources.
But what does DeepSeek actually mean for @ionet?
This article examines how DeepSeek’s technology intersect with economic models in the DePIN space, and explores the broader implications for AI development costs, performance benchmarks, and open-source models.
Let's dive in.
1. The Ever-Growing Appetite of Large Language Models
1.1 Scale, Complexity, and Costs
Modern LLMs can contain tens to hundreds of billions of parameters, enabling them to capture vast linguistic and real world knowledge. This size, however, translates into substantial hardware and operational costs.
Training a model in the range of GPT-3.5 or Claude requires:
Hardware Investment: Dozens to hundreds of NVIDIA A100 or H100 GPUs, each costing tens of thousands of dollars. A single state-of-the-art GPU (e.g., H100) can retail well above USD 30,000. Securing hundreds of these, plus the associated data-center infrastructure, can push capital expenditure into the tens or hundreds of millions.
Electricity and Cooling: Running GPUs for weeks or months incurs hefty power bills and cooling expenses, easily tipping total training costs higher by millions more.
Engineering and Operations: Orchestrating large clusters requires specialized expertise, networking hardware, and robust software frameworks like PyTorch or TensorFlow in distributed configurations.
OpenAI’s original GPT-3 training was often cited as costing millions of dollars, and later versions (like GPT-4) are rumored to have pushed that figure well above USD 50 million when factoring in iteration and experimentation cycles.
Some sources suggest GPT-4 training may have exceeded $100 million. This figure accounts not only for straightforward single-run training but also for multiple experimental runs, infrastructure overhead, and the iterative RLHF (Reinforcement Learning from Human Feedback) process though.
Similarly, there is no public info on the training cost of Anthropic's Claude, but various estimates peg the total expenses in the tens of millions of dollars, factoring in large-scale cluster usage and extended development cycles.
The demand for AI compute will only continue to rise as researchers and organizations push to build ever-larger models and more specialized architectures. With each new breakthrough, scaling laws suggest that bigger models trained on richer datasets tend to outperform smaller counterparts, driving a near-constant push to expand both model size and training data volume.
Additionally, novel application areas, such as multi-modal understanding, complex reasoning, and real-time interaction, require even more parameters, more sophisticated training routines, and higher inference throughput.
Compounded by the highly competitive AI market, in which organizations vie to demonstrate state-of-the-art performance, the trend toward increasingly massive compute requirements is poised to accelerate, likely outpacing even the already significant hardware and infrastructure investments being made today.
1.2 Closed-Source vs. Open-Source Models
Closed-Source Models
Proprietary model offerings like GPT and Claude often restrict access to their model weights, offering only an API endpoint. This constrains users who may want to fine-tune these models on proprietary data or analyze the underlying architecture for research. The recurring API costs can become significant for projects requiring large-scale inference.
Open-Source Models
In contrast, Meta’s Llama, despite its initial restrictive release, has spawned a host of open-source derivatives (e.g., Alpaca, Vicuna, etc.). These projects enable developers to run and fine-tune models locally, bypassing per-token or per-hour API fees. Yet training them from scratch, or even performing large-scale fine-tuning, can still be economically out of reach for many smaller institutions due to high GPU costs.
1.3 GPU Scarcity and Price Surges
The global shortage of more recent, advanced GPUs, fueled partly by a monopolization of these resources by big tech companies, has led to inflated costs and competition for top-tier hardware.
Many research labs and AI startups have been effectively priced out of acquiring the latest NVIDIA A100 or H100 GPUs, which creates a hardware crunch that has also contributed to dramatic stock price fluctuations for GPU manufacturers (see massive increase in $NVDA valuation in recent years).
Even large organizations with substantial budgets find themselves in bidding wars for limited supply, exacerbating an already challenging situation.
2. DePIN: Decentralizing AI Infra
2.1 Concept and Rationale
DePIN, a.k.a. decentralized physical infrastructure networks, offer a novel solution to the above-mentioned GPU shortage: pool GPU resources from a distributed community of node operators. Instead of renting an entire data-center cluster from a single cloud provider, users can tap into a global, decentralized network, paying only for actual GPU time consumed.
This model can lower costs by leveraging otherwise idle hardware. Enthusiast gamers, small businesses, or independent data centers with underutilized GPU rigs, can collectively form a dynamic supply pool. By disintermediating large cloud vendors, DePIN networks strive to democratize high-performance computing, ushering in a new age of AI innovation that is not dominated by big tech companies with unlimited financial resources to acquire GPUs and centralize supply.
2.2 IOnet: The GPU Aggregation Layer
One prominent example of a large-scale DePIN for AI compute is IOnet, which connects AI builders directly with GPU owners. Through smart orchestration algorithms, IOnet parcels out workloads to nodes that meet user-defined performance or price parameters. Key benefits of this approach include:
Cost Savings: GPU capacity can be offered at competitive rates, undercutting traditional cloud services.
Scalability and Flexibility: As more node operators join, the total capacity grows, and the network can scale to support larger or parallelized training jobs.
Geo-Distributed Clustering: IOnet's Ray-based solution is the only DePIN in the market that allows for geo-distributed clustering of GPU resources, also allowing for location-based optimizations.
However, altough very promising, some early-stage challenges persist, such as ensuring consistent data transfer speeds, maintaining reliability across heterogeneous hardware, and creating robust frameworks for scheduling and fault tolerance.
3. DeepSeek: Game-Changer or Marketing Coup?
3.1 Touted Efficiency Gains
DeepSeek's launch sent shock waves through the tech industry when it burst onto the scene with claims that it can match or even surpass GPT-3.5–level performance on standard benchmarks such as SQuAD (Stanford Question Answering Dataset), GLUE (General Language Understanding Evaluation), and MMLU (Measuring Massive Multitask Language Understanding), all at a fraction of the usual training cost.
Public statements suggest they spent under USD 5 million to achieve these results, an amount that sparks both intrigue and skepticism in an industry where USD 20 million to USD 50 million is not unusual for large-scale training runs (or even hundreds of millions as seen with other closed-source models).
3.2 Technical Differentiators
So how does DeepSeek claim to have done this?
Mixture-of-Experts (MoE)
DeepSeek employs a Mixture-of-Experts design, where the network is divided into specialized “experts.” At inference, only those experts most relevant to the input are activated. This architecture drastically reduces redundant computations per query.
Sparse Activation
Supplementing MoE, DeepSeek’s architecture also relies heavily on sparse activation, ensuring that only the most pertinent neurons fire when processing text. Together with dynamic sampling, model parallelism optimizations, and low-precision arithmetic (e.g., FP16 or FP8), these design choices streamline training. Early reports suggest that DeepSeek can match or surpass certain benchmarks held by GPT-3.5-scale models while requiring less advanced hardware to train.
Custom Training Pipeline
To optimize performance, the team employs techniques like:
Dynamic Sampling: Prioritizing the most informative training samples at critical points.
Lower-Precision Arithmetic: Using FP16 or FP8 to decrease memory usage and speed up calculations.
Advanced Model Parallelism: Splitting the model across multiple GPUs in ways that minimize communication overhead.
So what if this is true?
DeepSeek claims that even on mid-tier GPUs, such as the RTX 3090 or NVIDIA T4, it can achieve training efficiency that edges closer to top-tier results.
While some caution that training the largest DeepSeek models would still benefit from more powerful hardware, the central message is that cutting-edge performance no longer mandates HPC-grade clusters of A100s or H100s.
Should these claims hold up under independent testing, it may reduce the market premium on the latest GPU hardware.
4. DeepSeek's DePIN Implications
4.1 Shift in GPU Demand Ahead?
If DeepSeek-level efficiency truly gains traction, overall demand for the newest GPU generations may soften.
Over the past year, major cloud providers and AI labs have scrambled to secure high-end hardware, pushing NVIDIA H100 prices over the USD 30,000 mark and sometimes reaching even higher on secondary markets.
A credible alternative that performs well on mid-range GPUs could significantly cool this segment of the market:
That can pose a risk to DePIN networks that have high GPU exposure.
Some organizations, including DePIN projects that have purchased large fleets of expensive GPUs (e.g., A100s or H100s). They might find their assets less valuable or profitable if more AI builders pivot to cheaper equipment.
That means a possible rebalancing of GPU prices could occur, alleviating some of the bottlenecks but also impacting the margins of hardware-heavy DePIN providers and potentially even the viability of their business models.
4.2 Meet the Winner: IOnet
Marketplaces like @ionet that act as blockchain-based orchestration layers to coordinate GPU resources in a decentralized setting, but that don’t directly own the GPUs face much less risk.
They act as a middleware layer matching supply and demand and will likely even see an uptick in demand, precisely because they cater to scenarios where top-of-the-line GPUs are no longer mandatory for respectable results and where open-source models run on decentralized infrastructure can compete with their closed-source counterparts.
More specifically, with costs lowered, more AI innovators may attempt large-scale experimentation and specialized fine-tuning, driving usage across the network, as @ionet seamlessly allows smaller organizations or independent researchers to easily spin up training clusters through its marketplace, making models like DeepSeek accessible throughout the industry, and also opening up interesting (agentic) use cases in crypto.
5. Evaluating DeepSeek
5.1 Publicly Reported Benchmarks
Though still in the early days, DeepSeek’s developers have published partial results that claim near or slightly better performance than open-source LLMs in various tasks.
SQuAD (Stanford Question Answering Dataset)
DeepSeek demonstrates strong performance on SQuAD, coming within approximately 1–2 percentage points of GPT-3.5’s F1 score for extractive question answering. Concretely, if GPT-3.5 hovers around a reported 90–92% F1 on SQuAD v1.1, DeepSeek’s performance trails slightly but remains well above the 88% F1 mark. In practice, this gap often narrows or even disappears on certain passages if the model is carefully fine-tuned. Qualitatively, DeepSeek’s answers are rarely off-topic; rather, it occasionally falls short in exact phrasing or misses minor keywords—thus lowering the F1 metric slightly even though the content of the answer is semantically correct.
GLUE (General Language Understanding Evaluation)
The GLUE benchmark comprises a suite of tasks—from natural language inference (MNLI) to paraphrase detection (QQP) to sentiment analysis (SST-2). In preliminary trials, DeepSeek’s aggregate GLUE score rivals that of Llama 2 with a comparable parameter count, sometimes surpassing it by up to 1–2 percentage points on tasks requiring nuanced reasoning (e.g., MNLI) or careful paraphrase detection (QQP). For instance, if Llama 2 at the same scale achieves a combined GLUE score in the mid-80s, DeepSeek consistently lands at the higher end of that range, occasionally reaching the upper 80s. The model’s performance gains are especially evident when domain-specific fine-tuning is applied, as it effectively aligns its embeddings toward specialized vocabulary and context.
MMLU (Massive Multitask Language Understanding)
DeepSeek’s results on the MMLU benchmark—covering a variety of academic and professional topics (math, history, physics, law, etc.)—track closely with other large LLMs of similar size, including GPT-3.5 and Claude. The model reliably attains results within a few percentage points of these established baselines. For example, if GPT-3.5 is reported to achieve an average accuracy in the high 60s to low 70s across MMLU’s diverse tasks, DeepSeek’s scores tend to cluster similarly, usually within 2–3 points. In subject areas requiring advanced reasoning (e.g., graduate-level mathematics), DeepSeek sometimes lags behind specialized instruction-tuned models, but it still demonstrates remarkable generality across the majority of domains. This breadth is attributed to its extensive pre-training corpus, which includes academic textbooks, scientific papers, and domain-specific articles.
AIME (American Invitational Mathematics Examination)
DeepSeek’s ability to tackle AIME-like problems highlights its aptitude for multi-step logical and numerical reasoning, matching or closely trailing top-tier models like GPT-3.5 on many complex questions. Since AIME challenges often involve creative problem-solving across algebra, combinatorics, and geometry, DeepSeek’s performance underscores its grasp of advanced mathematical concepts and systematic solution paths. Although integer-based answers can penalize small slips in reasoning, DeepSeek’s success suggests that its chain-of-thought approach allows it to carefully work through most steps, making it a strong candidate for mathematically intensive tasks.
MATH-500
On the curated MATH-500 benchmark—which assesses mathematical problem-solving across pre-algebra, calculus, and beyond—DeepSeek achieves results comparable to other leading LLMs, including GPT-3.5 and Claude. Notably, it demonstrates a high success rate on multi-part problems, maintaining logical coherence across multiple steps and formula transformations. Occasional errors tend to stem from minor oversight in intermediate steps rather than a lack of conceptual understanding, indicating that further fine-tuning on step-by-step solutions could help DeepSeek reach or exceed state-of-the-art performance in math-focused domains.
GPQA (General Purpose Question Answering)
DeepSeek’s top-tier ranking on GPQA underscores its broad ability to handle both factual queries and more involved, multi-hop reasoning tasks across disparate subject areas such as history, science, and current events. Its extensive pre-training corpus allows DeepSeek to retrieve relevant facts accurately, while its reasoning mechanisms enable it to integrate information from multiple sources or context segments. In most head-to-head comparisons, DeepSeek’s performance closely matches or even surpasses GPT-3.5 and Claude, demonstrating its capability as a versatile question-answering model suitable for real-world applications like customer support, research assistance, and knowledge management.Independent validation and peer-reviewed results are still pending, and many in the AI community await more comprehensive studies to fully assess DeepSeek’s consistency, reliability, and generalizability.
5.2 The USD 5 Million Training Cost Claim
DeepSeek’s claim of training a cutting-edge LLM for under USD 5 million has stirred debate. Some experts suggest that cost might exclude certain overhead or rely on an uncommonly efficient pipeline that may not translate to all environments. Standard reference points in the industry place training budgets for large LLMs at USD 10 to 50 million or more. It’s plausible DeepSeek’s figure is a best-case scenario or a marketing-friendly snapshot of marginal expenses, rather than a fully loaded cost. Even if partially inflated, the focus on cost efficiency underscores potential breakthroughs in hardware utilization and training algorithm optimization though.
6. Conclusion
DeepSeek exemplifies a continued drive toward democratizing AI capabilities, and is a strong showcase of the potential that open-source AI innovation bears.
Even if the $5M training cost claim might not be entirely accurate, DeepSeek is a valuable contribution to the overall space, aiming to dissolve the link between cutting-edge performance and exorbitant hardware costs, and it highlights critical shifts in the AI industry:
Emerging Cost-Effective Models: The notion that advanced NLP systems can be trained and run on mid-tier hardware represents a step toward wider AI accessibility, potentially expanding the field of innovators and research labs that can participate.
DePIN’s Value Proposition: Decentralized GPU marketplaces like @ionet could see increased usage as developers look for flexible, low-cost compute compatible with new generation models like DeepSeek. This synergy may help alleviate supply chain bottlenecks and reduce reliance on single-vendor solutions.
Moderated Impact: While DeepSeek may not dramatically reduce overall GPU demand, given continued growth in AI applications and models, its reported breakthroughs can at least temper the market’s reliance on premium hardware. This equilibrium may partially lower costs and entry barriers.
A Net Positive, with Realistic Caution: Even if the $5 million training figure is an optimistic headline, DeepSeek’s open-source push and hardware savings represent a net positive for both AI and the broader tech community. At the same time, one should remain cautious about adopting a model still in early testing. The gulf between proof-of-concept efficiency and scalable, production-level deployment can be wide.
DeepSeek won't change the world. It's a pretty good model though and it can be run at a great price. It definitely adds momentum to the shift toward open-source, more resource-efficient AI development and could turn out to be a strong catalyst for well designed DePINs such as @ionet.
Though the overall impact on the AI industry might not be as revolutionary as everyone says, DeepSeek’s approach underscores an important message: the future of AI may not be exclusively dictated by who can purchase the most expensive GPUs, but rather by how innovators optimize and distribute the compute they already have at their disposal.
What came next · since 30 Jan 2025
Show how the 2 covered assets moved
This article covers the subject without making a call on the token, so price moves since publication are shown as market context.
Covered, no call
Mentioned for context
Since then
Zero has written 17 more pieces on io.net and $IO since this one.
- io.net IDE tokenomics: #1 revenue DePIN at 5.3x MC/ARRBullish call on $IO
- DeAI as crypto's endgame: compute, agents and autonomous economiesOn io.net
- Why decentralized compute solves the startup GPU cost squeezeOn io.net
- Big compute is cracking: summary of the DeAI case against centralized cloudOn io.net
Daily closes in USD from DefiLlama and Yahoo Finance. Only a bullish or bearish call in a thesis or market take is tracked; explainers, lists, maps and mentions are neutral context. Not financial advice.