# Why Chip Designers Are Embracing In-Memory Computing

An episode of [The Hardware Podcast with Fexingo: Chips, Devices, and Electronics Engineering Conversations](https://invisiblenetwork.ai/podcast/1896799982), 19 Jun 2026, 9:28.

Audio: https://audio.fexingo.com/business/the-hardware-podcast/episode-0061.mp3

Episode 61 of The Hardware Podcast with Fexingo dives into in-memory computing, a paradigm shift that runs computations directly inside memory cells to break the von Neumann bottleneck. Lucas and Luna explore the physics behind analog compute-in-memory using resistive RAM (RRAM) crossbar arrays, and contrast it with digital near-memory processing. They discuss why this matters for AI inference on edge devices, citing a concrete example: a startup called Mythic that shipped analog compute-in-memory chips in 2022, achieving 10x energy efficiency over traditional digital accelerators. The hosts also touch on Samsung's 2021 paper demonstrating in-memory computing on an MRAM bank, and the current challenges around precision and noise. No prior chip design knowledge required. #InMemoryComputing #ComputeInMemory #AnalogComputing #RRAM #MRAM #Mythic #Samsung #VonNeumannBottleneck #AIInference #EdgeAI #ChipDesign #Hardware #Semiconductors #Technology #FexingoBusiness #BusinessPodcast #TheHardwarePodcast #ChipRevolution Keep every episode free: buymeacoffee.com/fexingo

## Transcript

Lucas: Most of the energy inside an AI chip is spent moving data, not computing. Data travels from memory to processor and back — over and over. That round-trip cost is called the von Neumann bottleneck, and it's been the dominant constraint in computing for decades.
Luna: And in-memory computing flips that — you do the math right where the data lives, inside the memory cell itself. No data movement.
Lucas: Exactly. It's not a new idea — researchers have been chasing it since the 1960s. But the practical versions we're seeing today are built on a specific kind of memory: resistive RAM, or RRAM. These are two-terminal devices whose resistance can be programmed, and they can be laid out in a crossbar array — a grid where each intersection stores a weight.
Luna: And the beautiful thing is that when you apply a voltage to a row and read the current from a column, Kirchhoff's laws perform a multiplication and accumulation in a single step. It's analog, in-memory matrix multiplication.
Lucas: That's the physics. And for a neural network, most of the work is matrix multiplies. So if you can do them directly in memory, you save enormous amounts of energy and time. Let's talk about a real implementation.
Luna: You're thinking of Mythic, right? The startup that shipped an analog compute-in-memory chip for edge AI?
Lucas: Mythic is the most prominent example. They announced their M1076 chip in 2022. It's a processor that uses RRAM crossbars to run neural network inference. The claimed energy efficiency is about 10 tera-operations per second per watt — roughly 10 times better than a leading digital accelerator at the time.
Luna: That's a huge leap. But it comes with trade-offs. Analog computing is inherently noisy. The precision is limited — you're dealing with roughly 4 to 8 effective bits, not the 16 or 32 bits you get in digital.
Lucas: And that has real consequences. For some models — like small convolutional networks for wake-word detection or gesture recognition — 4-bit precision is fine. But for large language models with hundreds of billions of parameters, you'd need more precision than analog can currently deliver. So in-memory computing is finding its niche at the edge, not the data center.
Luna: That's where Samsung's 2021 paper comes to mind. They demonstrated in-memory computing on a commercial MRAM bank — not RRAM, but MRAM. They performed a handwritten digit recognition task directly on a 64-by-64 array of MRAM cells. The accuracy was about 95 percent, comparable to a software baseline.
Lucas: MRAM is already used in some embedded systems as a non-volatile memory. Samsung's paper showed it could double as a compute engine. The concept is the same — crossbar array, analog multiply-accumulate — but with a different memory technology. The key metric is energy per operation: they reported 1.3 picojoules per multiply-accumulate, which is orders of magnitude better than a digital core.
Luna: Let's step back — what makes a chip designer choose in-memory computing for a product? Is it purely efficiency, or are there other factors?
Lucas: Efficiency is the headline, but latency is a close second. If you're doing real-time inference on a drone or a hearing aid, you can't wait for data to shuttle between memory and processor. In-memory computing gives you deterministic, low-latency compute because the data stays put. There's also a density advantage: because you're eliminating the memory to logic interface, you can pack more compute into the same area.
Luna: But there's a reason it hasn't taken over everything yet. The biggest hurdle is probably the accuracy versus energy trade-off. Designers need to verify that their application can tolerate the noise.
Lucas: Right. And that's where the engineering work is happening. People are developing algorithmic techniques — like training neural networks with noise injection during training, so they become robust to the analog imperfections at inference time. It's a co-design problem, where the hardware and software are optimized together.
Luna: One company that's been quiet but interesting is International Business Machines — IBM. They demonstrated a 1-million-cell phase-change memory crossbar in 2018 and showed it could classify images at 74 percent accuracy on the CIFAR-10 dataset. And they've been publishing steadily on in-memory computing ever since.
Lucas: IBM's work is foundational. They've been researching phase-change memory for years. And what's interesting is that they're not only looking at analog compute-in-memory, but also at digital near-memory compute — where you place small compute units very close to memory, but not inside the cell itself. It's a spectrum, not a binary choice.
Luna: Near-memory is less radical but easier to integrate with existing digital flows. Companies like Samsung and SK Hynix are producing high-bandwidth memory with logic dies stacked directly on top of the memory dies. That's near-memory computing at scale.
Lucas: Exactly. And it's already shipping in products like the NVIDIA H100, which uses HBM3 with a logic die. That's a form of near-memory compute. The in-memory approach is further out, but the payoff is potentially larger. The question is about timing and risk.
Luna: Speaking of timing — where do you see in-memory computing in, say, five years? Will it be mainstream for edge devices?
Lucas: I think by 2031, analog compute-in-memory will be a standard option in the edge AI chip designer's toolkit. It won't replace digital accelerators, but it will be a well-understood, characterized building block — like an SRAM macro or a DSP core. We're already seeing foundries develop memory design kits that include RRAM models for analog compute. That's a strong signal.
Luna: And the data center? Any chance it moves beyond the edge?
Lucas: Maybe for specific workloads like recommendation systems or inference on smaller models. But for training, digital will dominate for the foreseeable future. The precision requirements are too high, and the throughput expectations too large. In-memory computing at the data center would require a major breakthrough in device uniformity and error correction.
Luna: So it's not a revolution overnight — it's an evolution. But the physics is compelling, and the first commercial products are already out there.
Lucas: Absolutely. And the research pipeline is deep. Just last year, a team at Stanford published a paper showing in-memory computing on ferroelectric field-effect transistors, which are compatible with CMOS logic. The field is moving fast.
Luna: I want to go back to something you mentioned earlier — the idea that most of the energy in a chip goes to data movement. I think that's one of those facts that, once you hear it, changes how you look at every computer.
Lucas: Yeah, it's a shift in perspective. You realize that the processor itself is almost free, energetically. It's the wires that cost. And in-memory computing cuts the longest wires in the system — the ones between memory and processor. That's why it's so compelling.
Luna: Before we wrap, I want to ask you about something I've seen in some papers — the concept of 'computational RAM' or CRAM. Is that the same as in-memory computing?
Lucas: CRAM is a specific implementation, yes. The term was coined by a group at the University of Minnesota. They propose a reconfigurable logic layer embedded within the memory array, so you can do bit-level operations directly in the memory. It's more digital than analog, but it shares the same philosophy: reduce data movement by integrating computation into memory.
Luna: So there's a whole spectrum: analog compute-in-memory, digital compute-in-memory, near-memory compute. And each has its own sweet spot.
Lucas: Exactly right. And the choice depends on the application, the memory technology, and the required precision. It's not one-size-fits-all, but the direction is clear: we are moving computation closer to data.
Luna: Final thought — if a chip design student were to focus on one area today, would you say in-memory computing is the right bet?
Lucas: I would. It touches device physics, circuit design, architecture, and algorithms. It's a rich, interdisciplinary field with real impact. And the industry is hungry for people who understand both the hardware and the software side of the co-design problem.
Luna: Good advice. And on that note, I think we have a solid picture of where in-memory computing stands in mid-2026.
Lucas: We do. The hardware is real, the applications are emerging, and the community is growing. It's an exciting time to be watching this space.

Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them.