GoKawiilSK hynix, TetraMem, and researchers from the University of Southern California have developed a memristor-based in-memory computing (IMC) system-on-chip (SoC) for AI edge devices. The device is designed to accelerate neural network inference in lightweight AI models while consuming a fraction of the power that higher-end GPUs or NPUs would. To a large degree, the SoC is a proof-of-concept chip, as its performance would peak at around 2.54 TOPS in a theoretical best-case scenario, which is 16X below Microsoft's Copilot+ requirements.
A DWC-optimized IMC architecture
Memristor-based in-memory computing (IMC) accelerates neural networks by performing analog computations directly inside memory arrays, which reduces data movement and power consumption. However, depthwise convolution (DWC) — a core operation in lightweight networks such as MobileNet — performs independent per-channel filtering with limited data reuse and therefore maps poorly onto conventional crossbar arrays. To address this limitation, researchers from SK hynix, TetraMem, and USC developed an SoC that features both conventional IMC crossbars and a memristor-based IMC architecture specifically optimized for DWC.
(Image credit: SK Hynix)
The jointly developed SoC is based on an embedded RISC-V processor that schedules workloads and features 10 neural processing units (NPUs). One NPU out of 10 is dedicated to depthwise convolution, while the remaining nine execute pointwise and dense operations. Nine out of 10 NPU include a 256 × 256 memristor crossbar that performs the analog vector-matrix multiplication (VMM), 256 8-bit DACs that convert digital activations into analog voltages, 256 8-bit ADCs that convert the analog outputs back into digital values, and additional peripheral circuitry for reading, writing, programming, and controlling the crossbar.
Latest Videos From Watch full video here:
The DWC-optimized NPU replaces its conventional array with eight specialized 252 × 28 zig-zag crossbar blocks, but retains DACs and ADCs. SK hynix developed and fabricated the memristor devices and integrated the resistive switching cells on top of the 65 nm CMOS circuitry using its back-end process.
That DWC-optimized NPU is the key feature of the whole SoC. To accelerate depthwise convolution, TetraMem replaced the straight selection lines used in conventional 1T1R crossbars with a zig-zag topology. As a result, the NPU contains eight 252 × 28 crossbar blocks whose diagonal selection lines activate 252 memory cells across 28 columns, which enables 28 independent 3 × 3 convolutions to run in parallel while using 100% of the array for weight storage. The remaining nine NPUs retain conventional 1T1R crossbars for 1×1 pointwise and dense layers and preserve the throughput and energy efficiency of traditional in-memory computing.
Great efficiency, low performance overall
To demonstrate the architecture, the researchers deployed a customized MobileNetV1Small neural network for the Visual Wake Words benchmark. The network contains approximately 36,000 parameters; all depthwise layers were mapped to the dedicated NPU, and pointwise layers were mapped to the remaining NPUs.
... continue reading