# How On-Chip Photonic Processors Are Accelerating AI Inference

An episode of [The Hardware Podcast with Fexingo: Chips, Devices, and Electronics Engineering Conversations](https://invisiblenetwork.ai/podcast/1896799982), 26 Jul 2026, 7:54.

Audio: https://audio.fexingo.com/business/the-hardware-podcast/episode-0133.mp3

Episode 133 dives into the emerging technology of on-chip photonic computing for artificial intelligence. Lucas and Luna explore how startups like Lightmatter and established players like IBM are integrating lasers, modulators, and detectors onto a single silicon chip to perform matrix multiplications using light. They discuss a specific chip that achieves over 10 peta-operations per second at just 30 watts, dramatically reducing the energy cost of AI inference. The hosts examine the key engineering challenges—from co-packaging lasers to managing thermal stability—and why this approach could reshape data center hardware by 2030. This episode is essential for anyone following the future of chip design and AI hardware. #Photonics #AIAccelerator #Lightmatter #IBM #SiliconPhotonics #OnChip #Inference #MatrixMultiplication #PowerEfficiency #CMOS #LaserIntegration #DataCenter #Hardware #TechPodcast #FexingoBusiness #BusinessPodcast #Engineering #FutureOfChips Keep every episode free: buymeacoffee.com/fexingo

## Transcript

Lucas: You hear a lot about the compute bottleneck in AI — but what gets less attention is the data movement bottleneck. The energy cost of shoving data from memory to compute units and back again can be ten to a hundred times higher than the actual arithmetic.
Luna: And that's where photonics comes in? Moving data with light instead of electrons?
Lucas: Exactly. Light doesn't generate the same RC delays or resistive losses. But today we're talking about a step beyond just optical interconnects — actually doing the computation itself with photons. On-chip photonic processors for AI inference.
Luna: So not just sending data over fiber, but doing matrix multiplies in the optical domain. How does that work at the chip level?
Lucas: The core idea is to encode information on the intensity or phase of light beams, then use passive optical components like directional couplers and micro-ring resonators to perform multiplication and addition. A single photonic circuit can execute a matrix-vector product in femtoseconds, with almost no power consumption in the optical domain. The energy mostly goes into the lasers and the photodetectors.
Luna: That sounds almost too good to be true. What's the catch?
Lucas: The big one is integration. You need a laser source — ideally on-chip — plus modulators to encode data, a mesh of interferometers for the math, and then detectors to convert back to electrical. Getting all that onto a single die in a standard CMOS process is brutally hard. But companies like Lightmatter have been shipping products like their Envise processor that claim 10.6 peta-operations per second at 30 watts for inference.
Luna: 10.6 peta-ops — that's over 350 tera-ops per watt. Compare that to an NVIDIA H100, which is maybe 1 peta-op at 700 watts -- that's a fraction of the efficiency.
Lucas: Right. The H100's tensor cores are incredible, but they're still fighting electrical impedance and heat. Photonic chips operate at room temperature with much lower energy per operation, and the data stays in the optical domain through the entire matrix multiply, so there's no analog to digital conversion until the output.
Luna: But aren't there precision issues? Analog optical computation is inherently noisy compared to digital electronics.
Lucas: That's a fair concern. Most photonic AI chips today use low-precision formats — 4-bit or 8-bit — and rely on the fact that inference is often robust to some noise. For training, high precision is still needed, so photonics is mostly aimed at inference workloads.
Luna: So where does the on-chip engineering get tricky? I assume the laser integration is the nightmare.
Lucas: It is. Indium phosphide lasers don't grow well on silicon. Companies use either heterogeneous integration — bonding a iii v laser die onto a silicon photonic chip — or they use hybrid approaches with external lasers and fiber coupling. Both add cost. But IBM has demonstrated a fully integrated photonic AI accelerator using a 45-nm CMOS silicon photonics process that includes an on-chip laser array.
Luna: IBM's work from last year — they published a paper showing 94 gigaMACs per watt? That's still well below what Lightmatter claims, but it's fully integrated on a single chip.
Lucas: Exactly. And the thermal stability is another challenge. Micro-ring resonators have a temperature sensitivity of about 10 picometers per degree Celsius. You need active thermal tuning, which eats into the power budget. But clever design can mitigate that.
Luna: So the technology is real, but it's not yet mainstream. What would it take for photonic AI chips to compete with, say, NVIDIA's next-generation Blackwell architecture?
Lucas: Scale. The current chips have limited compute density because photonic devices are larger than transistors. You can't pack a billion modulators on a die yet. But as manufacturing matures — and we see moves like the AIM Photonics initiative and foundries adding photonic process modules — the density will improve.
Luna: And the market need is there. Data centers are starving for power-efficient inference.
Lucas: Speaking of which — if today's tech conversation gave you something usable, a couple of dollars a month is genuinely what keeps these episodes ad-free. Buy me a coffee dot com slash fexingo, if you've gotten something out of them.
Luna: Yeah, every contribution makes a real difference. And we're back to the tech.
Lucas: So where are we on the roadmap? Lightmatter recently announced a second-generation chip that can handle both inference and training, using a hybrid analog-digital architecture. They claim to have achieved 40 peta-ops per watt for certain workloads.
Luna: Wow. Forty peta-ops per watt. That would change the economics of large-scale AI deployments. But are there any fundamental physical limits? Can photons do everything electrons can?
Lucas: Not everything. Logic operations with photons are much harder — the nonlinear interactions needed for transistors are weak in optics. So photonics is primarily for linear algebra, which is 90% of neural network workloads. But for the remaining 10% — activation functions, pooling, etc. — you still need electronics. That's why all photonic processors are hybrid chips.
Luna: So it's a co-integration story. Which brings us back to advanced packaging and chiplets. Are we seeing photonic chiplets?
Lucas: Absolutely. Several startups are offering photonic engine chiplets that can be assembled with electronic compute dies on an interposer. This allows data center operators to mix and match. For example, a photonic matrix multiply chiplet next to a conventional GPU for the nonlinear parts.
Luna: And what about the software stack? You can't just drop in a photonic chip and expect it to work with PyTorch.
Lucas: That's a critical point. Lightmatter provides a compiler that maps neural network graphs onto their photonic mesh. It handles pipelining, precision scaling, and calibration for thermal drift. IBM has their own software stack for their photonic accelerator. The ecosystem is still immature compared to NVIDIA's CUDA, but it's evolving fast.
Luna: So by 2027 or 2028, we could see a significant share of inference workloads running on photonic hardware. Any particular industry that would adopt first?
Lucas: Edge inference is low volume, but the real opportunity is in large-scale cloud inference — think recommendation engines, large language model serving. Those hyperscale data centers have the budget and the need for energy savings.
Luna: Makes sense. And with the power constraints we saw during the AI boom of 2024-2026, any improvement is welcome.
Lucas: Exactly. To me, photonic AI chips represent one of the most promising ways to keep the scaling trend alive. We're not hitting the physical limits of 'what can a chip do' — we're hitting the limits of 'what can a chip do while staying cool and not burning the grid'. Photonics offers a way around that wall.
Luna: And it's a classic hardware story: the physics was always there, the engineering took decades. Now we're finally seeing products.
Lucas: Well said. That's our episode. Thanks for listening to The Hardware Podcast with Fexingo.

Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them.