RoMe: Row Granularity Access Memory System for Large Language Models
By Hwayong Nam †, Seungmin Baek †, Jumin Kim †, Michael Jaemin Kim ‡, and Jung Ho Ahn †
Seoul National University †, Meta ‡

Abstract
Modern HBM-based memory systems have evolved over generations while retaining cache line granularity accesses. Preserving this fine granularity necessitated the introduction of bank groups and pseudo channels. These structures expand timing parameters and control overhead, significantly increasing memory controller scheduling complexity. Large language models (LLMs) now dominate deep learning workloads, streaming contiguous data blocks ranging from several kilobytes to megabytes per operation. In a conventional HBM-based memory system, these transfers are fragmented into hundreds of 32B cache line transactions. This forces the memory controller to employ unnecessarily intricate scheduling, leading to growing inefficiency.
To address this problem, we propose RoMe. RoMe accesses DRAM at row granularity and removes columns, bank groups, and pseudo channels from the memory interface. This design simplifies memory scheduling, thereby requiring fewer pins per channel. The freed pins are aggregated to form additional channels, increasing overall bandwidth by 12.5% with minimal extra pins. RoMe demonstrates how memory scheduling logic can be significantly simplified for representative LLM workloads, and presents an alternative approach for next-generation HBM-based memory systems achieving increased bandwidth with minimal hardware overhead.
To read the full article, click here
Related Semiconductor IP
- HBM Memory Controller
- HBM Memory Model
- HBM DFI Verification IP
- HBM 4 Verification IP
- HBM Synthesizable Transactor
Related Articles
- SkipOPU: An FPGA-based Overlay Processor for Large Language Models with Dynamically Allocated Computation
- SV-LLM: An Agentic Approach for SoC Security Verification using Large Language Models
- Assertain: Automated Security Assertion Generation Using Large Language Models
- LACE: Large Language Model Aided Multi-Agent Framework for Agile RISC-V Instruction Extension
Latest Articles
- An Agentic Python Framework for LLM-Driven UVM Testbench Synthesis with Closed-Loop Compiler Feedback
- Reconfiguring System-Level Protection Through Bounded Delegation in RISC-V MCUs
- SkipLane: Write-Once, Conflict-Free Accumulation for Zero-Skipping CNN Engines on FPGAs
- Priority-Aware Deficit Round Robin for AXI4 Bandwidth Fairness in ASAP-7nm Process
- Atik: A Shared-Mesh Transformer Accelerator for RISC-V