RISC-V Summit

The size of the Foundation models behind the Generative AI revolution have grown at a rate of more than 400x every 2 years, while DRAM memory capacity has been increasing only at 2x every two years, leading to what is commonly called the “memory wall”. Similarly, while the required throughput rate of LLMs making up the Foundation models has been increasing at 10x per year, the increase in computational capability of GPUs has been at a pace of only 10x in 4 years, leading to what is commonly called the “compute wall”. These trends have raised a new set of challenges in how to economically train these models, cost-effectively run them, and manage the tremendous increase in electrical power. The first contribution of this session are lessons learned in leveraging hardware and software developed for traditional AI workloads and how it was extended to support Generative AI. The session’s next main contribution is how we are applying lessons learned from our first-generation technology to our next generation. In this session’s final contribution, we will also discuss how the RISC-V ISA could be extended in ways that would make it more efficient and compelling at running Generative AI.