Co-Packaged Optics: The Hidden Fuel Line Behind Your AI

When my AI feels slow or my usage gets throttled, my first instinct is to blame the model. I’m usually wrong. A big share of what I pay for is not “thinking” at all. It’s moving bits between chips. Co-packaged optics is the quiet rebuild of that plumbing, and it explains more about your AI bill than the next model release will.

Here’s the promise: by the end of this, you’ll understand the AI data center interconnect bottleneck without a hardware degree. I’ll map the terms, show you one comparison table, and admit where my own mental model was broken.

The frame I keep coming back to is simple. Compute is the engine. The interconnect is the fuel line. A faster engine does nothing if the fuel line is too thin.

The mental model: compute is the engine, interconnect is the fuel line

Picture a rack of GPUs as one big machine. Each GPU is strong. But no single GPU runs a frontier model alone. They work as a team, passing huge amounts of data back and forth, thousands of times per second.

That passing-back-and-forth is the interconnect. It’s the wiring between chips, between boards, and between racks. When people say “AI factory,” they mean a building full of these connections, not just chips.

Here’s the part nobody told me. Inside that factory, a large slice of the power and cost goes into links, not logic. The wires that carry data sip electricity and throw off heat the whole time the cluster runs.

So when your AI request travels through a cluster, it isn’t just waiting on a model to think. It’s waiting on data to move. If the fuel line is narrow or expensive, you feel it as latency, as caps, and as cost passed down the chain.

That’s the angle I want you to hold. The interconnect, not the model, is the hidden tax on the AI you use.

What is co-packaged optics, in plain terms

Let me define it cleanly, the way I’d want it explained to me.

Co-packaged optics (CPO) means putting the optical parts that turn electrical signals into light right next to the switch or accelerator chip, inside the same package. Only a thin fiber leaves the package. The electrical path from logic to light shrinks from tens of centimeters down to a few millimeters. That short path is the whole point.

To get there, you need to know two more words. The first is the optical engine: the little block that converts electrons into photons and back. The second is silicon photonics.

Silicon photonics is the trick that lets us build those light-handling parts on the same kind of silicon wafer we use for chips. Instead of bulky, separate optical modules, the light path gets etched into silicon, close to the logic. The Wikipedia entry on silicon photonics is a fair starting map if you want the textbook version.

Why does “in the same package” matter so much? Because moving an electrical signal across a board is expensive in power and noisy over distance. The longer the electrical run before it becomes light, the more energy you burn and the more the signal degrades.

CPO cuts that run to almost nothing. Shorter electrical path, less power, cleaner signal. That’s the trade the whole industry is chasing.

Why copper is hitting a wall in the AI data center interconnect

For short hops, copper wire is still the cheapest way to move bits. It needs no laser. It barely sips power. So why replace it?

Reach. Copper runs out of road fast. At today’s per-lane speeds, a passive copper cable carries data only about one meter before the signal falls apart. As speeds climb past 200 gigabits per lane toward 400, that reach shrinks further.

A rack is bigger than one meter. A cluster of racks is much bigger. So copper works inside a tight neighborhood, then stops. That short leash is what people mean by the copper wall.

You can push copper harder with more powerful chips driving the signal. But that costs power and throws off heat, and it hits physical limits in the silicon. The faster you go, the worse the trade gets.

This is the same scaling pressure I wrote about in the move to 2nm chips and backside power delivery. We squeeze the transistors, then the wires connecting them become the new ceiling. Compute got cheaper faster than communication did.

Light doesn’t have copper’s reach problem. A photon in fiber travels far with little loss. So the industry’s answer is to move the electrical-to-light conversion as close to the chip as possible, then let light do the long carrying. That’s the optical interconnect story in one breath.

CPO vs pluggable transceivers: the table that frames the trade-off

Today, most optical links use pluggable transceivers. These are modules you snap into the front panel of a switch, like a thick USB stick. They convert electricity to light at the panel, then send it down fiber.

Pluggables have one giant advantage: if one dies, you pull it and plug in a new one. A technician fixes it in seconds. No special tools, no downtime for the whole switch.

But pluggables sit far from the switch chip, on the front panel. That long electrical run is exactly what burns power. By industry accounts, pluggable optics can eat around half of a traditional switch system’s power and more than half of its cost. That’s the tax CPO is trying to cut.

Here’s how the two stack up, using sourced figures. I’ve hedged the energy numbers as “roughly,” because sources vary.

Factor	Pluggable transceivers	Co-packaged optics (CPO)
Electrical path to light	Tens of centimeters (front panel)	A few millimeters (in package)
Energy per bit	Roughly ~20 pJ/bit	Pushing toward roughly ~5 pJ/bit and below
System power share	Optics can be ~50% of switch power	Reported ~70% lower interconnect power (Broadcom)
Cost share	Can exceed 50% of switch cost	~8x better silicon-area efficiency (Broadcom)
Serviceability	Swap a module in seconds	Hard — optics bonded near the ASIC
Status	Mature, everywhere today	Shipping from 2025, broader in 2026

Read the table as a trade, not a winner. CPO wins on power and efficiency. Pluggables win on repair. The honest tension is efficiency versus repairability, and the industry is still negotiating it.

That “pJ/bit” column deserves a plain reading. A picojoule per bit is just energy spent to move one bit. Lower is better. Front-panel pluggable optics sit around 20. CPO aims for roughly 5 or under. Passive copper is also around 5, but only over that one-meter leash. So copper is cheap and short; CPO is efficient and long. That gap is the reason photonics moved into the package.

Where I was wrong

I’ll own my broken mental model, because correcting it is the whole point of this post.

For a long time I assumed the AI race was mostly about bigger, smarter models. Faster AI, I figured, just meant more parameters and more training. So when my AI felt slow or expensive, I blamed the model and waited for the next version.

I missed the real lever. A huge part of the 2025-2026 story isn’t the model at all. It’s photons replacing electrons inside the rack. The bottleneck moved from “how smart is the model” to “how fast and how cheaply can we move bits between the chips running it.”

I also assumed the data center was mostly compute, with networking as a side detail. Wrong. Once I saw that optics can eat half the power and cost of a switch, the picture flipped. The fuel line isn’t a footnote. In an AI factory, it’s a main character.

The third thing I got wrong: I thought a new switch technology would just replace the old one cleanly. It doesn’t. Every CPO switch shipping today still pairs with pluggable optics somewhere in the system. The two will coexist for years. New plumbing gets laid alongside the old, not on top of it overnight.

Fixing that mental model changed how I read AI cost news. Now when I see a headline about an AI factory’s power draw, I don’t just think “training is expensive.” I think “a lot of that is the fuel line, and the fuel line is being rebuilt.”

What’s actually shipping: silicon photonics moving from slides to racks

This isn’t a someday technology. It started shipping.

NVIDIA used its GTC stage in March 2025 to announce CPO switches: Quantum-X for InfiniBand and Spectrum-X for Ethernet. Quantum-X runs 115.2 terabits per second across 144 ports, with 24 optical engines built on a TSMC silicon-photonics process. NVIDIA’s claims include 3.5x better power efficiency and 4x fewer lasers, with an estimated 40 megawatts of data-center power saved at scale. Their technical blog on scaling AI factories with co-packaged optics lays out the power-efficiency case in detail.

Broadcom got there earlier on the Ethernet side. Its Bailly switch was billed as the industry’s first 51.2-terabit CPO Ethernet switch, pairing a Tomahawk 5 chip with eight silicon-photonics optical engines. Broadcom’s investor relations release on the 51.2-Tbps co-packaged optics switch reports roughly 70% lower interconnect power and 8x better silicon-area efficiency versus pluggables. The company also reported shipping more than 50,000 such switches during 2025.

If you want the neutral, standards-body view rather than a vendor’s, the IEEE Electronics Packaging Society paper on CPO for hyperscale networking is the calmest authority I found. It treats CPO as a real industry direction, not a marketing slide.

Read all those numbers as engineering examples, not stock tips. I’m not telling you any company is a buy. I’m showing you that the fuel-line rebuild is real, funded, and already in racks. The names matter only as the teams shipping it.

How a non-developer should think about this

You won’t buy a CPO switch. So why hold any of this in your head? Because it changes how you read the AI you pay for.

Three takeaways I keep.

First, when AI cost or speed news lands, separate two questions: is this about the model, or about moving data? A lot of the cost story is the fuel line. Headlines rarely split them, but you now can.

Second, “more compute” has a hidden partner. Every time a lab adds GPUs, it also has to connect them. Past a point, connection cost grows faster than you’d guess. The interconnect is why scaling AI is not just “add more chips.” I touched the same money thread in the Anthropic and OpenAI IPO outlook — who pays for all this infrastructure is downstream of how efficient the plumbing gets.

Third, efficiency in the fuel line is a long game, not a launch. CPO won’t trend on your feed like a new chatbot. But cheaper, cooler links are part of why running large models can get less wasteful over time. That’s the kind of progress you feel in your bill, not in a demo.

This is also a clean example of frontier hardware translated for normal users, the same way I tried with humanoid robots going mainstream. The goal isn’t to make you an engineer. It’s to make the engineering legible.

FAQ

What is co-packaged optics (CPO)? Co-packaged optics puts the parts that convert electrical signals into light right next to the switch or accelerator chip, inside the same package. Only fiber leaves the package. This shrinks the electrical path from tens of centimeters to a few millimeters, which cuts power and cleans up the signal.

What is silicon photonics? Silicon photonics is the technique of building light-handling parts on the same kind of silicon used for chips. Instead of bulky separate optical modules, the light path is etched into silicon close to the logic. It’s what makes the in-package approach practical at data-center scale.

Why does AI need optical interconnects instead of copper? Copper is cheap and low-power for short hops, but it runs out of reach. A passive copper cable carries data only about one meter at today’s per-lane speeds, and that shrinks as speeds rise. AI clusters are far bigger than one meter, so light, which travels far in fiber with little loss, takes over the longer links.

When will co-packaged optics ship? Are CPO switches available yet? They already started. Deployment-ready CPO products arrived in 2025, with broader shipping expected through 2026. Broadcom reported shipping more than 50,000 CPO switches in 2025, and NVIDIA announced CPO switches at GTC in March 2025. Pluggable optics still ship alongside them.

Is CPO better than pluggable transceivers? It depends on what you weigh. CPO wins on power and efficiency, with reported figures like roughly 70% lower interconnect power. Pluggables win on serviceability — you can swap a failed module in seconds, while CPO optics sit bonded near the chip. The two coexist rather than one fully replacing the other.

Why is copper hitting a wall in AI data centers? Per-lane speeds keep climbing, but pushing copper faster hits thermal and signal-integrity limits in the silicon driving it. Reach collapses as speed rises. That short, hot leash is the copper wall, and it’s the gap optical interconnects are built to close.

The reframe

So the real question isn’t “when does my AI get a smarter model.” It’s “how cheaply can the rack move a bit.” Once you see the interconnect as the fuel line, half the AI cost story stops being mysterious. The wires were always the wall — co-packaged optics is the industry quietly moving that wall back.

Next in this Framework Deep Dive series, I’ll map the other half of the plumbing: how scale-up and scale-out networking decide whether your model runs on one rack or a thousand, and why that choice shapes the AI you can actually rent.

---

seonjae — Korean office worker documenting his transition into AI systems, agents, and vibe coding — without a CS background. Shipping in public.