Samsung’s LPDDR5X-PIM Brings 614 GB/s In-Memory AI Inference to Low-Power DRAM

Samsung used Hot Chips 2026 to explain how LPDDR5X-PIM adds processing directly to low-power DRAM for AI inference. The design combines 16 in-memory processing blocks, parallel multiply-accumulate logic, and floating-point and integer arithmetic units inside a 16 GB LPDDR5X package. Samsung reported a 2.28× reduction in inference time and 3.01× higher token throughput on an edge-AI SoC running Llama 3.1 8B, although those figures are company-reported results rather than a broad independent evaluation. 1 9

What LPDDR5X-PIM is designed to solve

Large-language-model inference often spends substantial time moving model weights and intermediate data between DRAM and a processor. Processing-in-memory, or PIM, addresses that bottleneck by performing selected operations beside the DRAM arrays instead of sending every operand across the external memory interface. Samsung describes LPDDR5X-PIM as its first LPDDR-based PIM solution for AI inference. 2 9 13

That makes the technology particularly relevant to workloads dominated by repeated matrix-vector operations, where reducing data movement can matter as much as adding more conventional compute capacity.

The architecture: 16 PIM blocks inside a four-die package

Samsung’s announced configuration is a 16 GB LPDDR5X package built from four dies in a JEDEC-style 561-ball package, operating at 9,600 Mb/s per pin. The four dies share command and address signals while using separate data lines, consistent with the parallel organization used by LPDDR5 systems. 9 17

The package contains 16 PIM blocks distributed across the DRAM banks. Each block combines two types of hardware:

• MAC trees perform the reduction-heavy multiply-accumulate work used in GEMV and related matrix operations.
• Floating-point and integer ALUs handle elementwise and other supporting calculations.

This heterogeneous arrangement lets the memory handle more than simple arithmetic on a single data type. Samsung’s earlier PIM material also describes a SIMD floating-point unit and bank-level placement of PIM units, illustrating the broader design approach of putting specialized compute close to the data it needs. 9 11

Why the 614 GB/s figure is easy to misunderstand

Samsung quoted up to 614 GB/s of internal aggregate PIM bandwidth. That is not the ordinary off-package bandwidth available to a host processor through a conventional LPDDR5X interface. It reflects the parallel work performed across multiple banks and in-memory processing blocks. 1

In Samsung’s comparison, the internal PIM figure was roughly eight times the conventional LPDDR5X bandwidth used for the baseline. The important distinction is that PIM exposes bandwidth inside the memory for supported operations; it does not turn the external LPDDR interface into a 614 GB/s link.

Address alignment and bank-level execution

A key part of the design is Address Align Mode. Corresponding operands are laid out at matching addresses across participating banks, so a shared PIM command can trigger the same operation in parallel. Because the dies in an LPDDR rank receive common command and address signals while maintaining distinct data paths, this layout enables parallel execution without treating the package as one conventional wide external channel. 17

Samsung described two operating styles:

• Single-bank operation provides finer-grained control and leaves more opportunity for ordinary DRAM traffic to proceed alongside PIM work.
• Multi-bank operation activates more PIM blocks at once, increasing internal parallelism and bandwidth for suitable kernels.

The trade-off is workload-dependent. More participating banks can improve throughput, but they also reduce the banks available at that moment for conventional memory accesses. PIM therefore works best when the software, tensor layout, and target kernels are mapped to the memory’s bank structure rather than treated as an invisible drop-in accelerator.

Samsung’s Llama 3.1 8B inference results

On an edge-AI SoC running Llama 3.1 8B, Samsung reported the following comparison with conventional LPDDR5X:

• 2.28× lower inference time
• 3.01× higher token throughput
• Up to 614 GB/s of internal PIM bandwidth, compared with a conventional baseline reported at about 76.8 GB/s in the presentation coverage

These numbers describe a specific Samsung test configuration, not a universal performance multiplier. Results will depend on factors such as model precision, tensor placement, batch size, memory capacity, software support, and how much of the workload consists of GEMV-like operations.

Samsung also presented lower power consumption as an architectural benefit of reducing movement between memory and the SoC. However, the material provided here does not establish a comparable independently verified watts-per-inference or joules-per-token figure. The power advantage is therefore best understood as a design objective and expected benefit for suitable workloads, not as a published percentage reduction.

LPDDR5X-PIM versus HBM

HBM remains the stronger fit when an accelerator needs maximum external memory bandwidth, especially for large-scale training and high-end data-center workloads. Its performance comes with stacked DRAM, through-silicon vias, advanced packaging, and substantial silicon-area and thermal requirements. At Hot Chips, Micron described the wafer-area penalty of HBM relative to DDR5 as widening, with the same-capacity HBM design requiring roughly three times the wafer area in its comparison.

Samsung’s approach makes a different compromise. LPDDR5X-PIM keeps the low-power DRAM form factor and adds limited compute close to the banks. It cannot offer HBM’s general-purpose accelerator bandwidth, but it may be attractive when the dominant problem is moving data for inference rather than supplying peak floating-point throughput.

That gives the technology a differentiated target:

• Mobile and client devices: on-device AI under tight power and thermal limits.
• Edge systems: local inference without a discrete HBM-based accelerator.
• Servers: memory-movement-bound inference and data-processing tasks where HBM’s cost and packaging overhead are difficult to justify.

The most accurate description is a complement or cost-sensitive alternative to HBM for selected inference workloads—not a universal HBM replacement.

How the Hot Chips announcement fits Samsung’s roadmap

Samsung had already demonstrated LPDDR5X-PIM at FMS 2026. The Hot Chips session added a more detailed architectural explanation and performance discussion; the conference program listed the Samsung presentation as a dedicated memory-session talk focused on LPDDR-based PIM for AI inference. 9 13

The broader direction is toward making processing-in-memory a more deployable low-power memory feature rather than limiting it to specialized HBM experiments. Samsung’s longer-term LPDDR6-PIM direction is also tied to standardization ambitions, although the available evidence does not establish a finalized JEDEC specification or ratification date. DeepX has said it plans to use Samsung LPDDR5X-PIM in its second-generation DX-M2 chip and is targeting LPDDR6-PIM for a later DX-M3 design. 15

LPDDR5X-PIM and XCENA MX1 are related—but different

Samsung’s presentation appeared in the same Hot Chips memory session as XCENA’s MX1, but the two technologies operate at different levels of the system.

XCENA MX1 is a CXL Type 3 computational-memory device aimed at rack-scale infrastructure. Its design combines up to 2 TB of DDR5 capacity, SSD-backed capacity, and more than 1,000 RISC-V cores for near-memory processing. 4 10

LPDDR5X-PIM, by contrast, integrates a smaller set of specialized compute units directly into LPDDR DRAM. Both approaches attempt to reduce host-memory data movement, but MX1 is an attached CXL server device while Samsung’s design is a low-power memory package intended to sit close to an SoC or other LPDDR-style system.

The practical takeaway

Samsung’s Hot Chips 2026 disclosure frames LPDDR5X-PIM as a targeted answer to the memory bottleneck in AI inference. Its 16 processing blocks, bank-level parallelism, MAC trees, and mixed-precision ALUs can keep selected operations near the data, while the package retains an LPDDR5X-compatible form factor and a 9,600 Mb/s-per-pin operating point. 9

The reported Llama 3.1 8B results are promising, but they should be read as workload-specific vendor benchmarks. The technology’s real significance is strategic: it extends PIM into low-power memory that could serve mobile, edge, client, and selected server systems, while leaving HBM to workloads that genuinely need its much higher general-purpose bandwidth.