RISC-V Summit

The size of the Foundation models behind the Generative AI revolution have grown at a rate of more than 400x every 2 years, while DRAM memory capacity has been increasing only at 2x every two years, leading to what is commonly called the “memory wall”. Similarly, while the required throughput rate of LLMs making up the Foundation models has been increasing at 10x per year, the increase in computational capability of GPUs has been at a pace of only 10x in 4 years, leading to what is commonly called the “compute wall”. These trends have raised a new set of challenges in how to economically train these models, cost-effectively run them, and manage the tremendous increase in electrical power. The first contribution of this session are lessons learned in leveraging hardware and software developed for traditional AI workloads and how it was extended to support Generative AI. The session’s next main contribution is how we are applying lessons learned from our first-generation technology to our next generation. In this session’s final contribution, we will also discuss how the RISC-V ISA could be extended in ways that would make it more efficient and compelling at running Generative AI.

Our client is developing a massively parallel 64-bit chip for AI inference workloads. To facilitate early software development, we are bringing up an AI tool flow for this chip in a QEMU RISC-V environment. In this talk, we'll share our experience of getting three key AI frameworks working with RISC-V QEMU: Pytorch, Tensorflow and the OpenXLA compiler. Our talk will share our experience addressing two key issues. We will describe the challenges we faced, their solutions and reflect on the lessons learned for future work. The first of these is simply getting the tools to effectively run in an emulated RISC-V environment. These tools are large, fast moving pieces of software with extensive external dependencies. Our second challenge is performance. AI workloads are inherently parallel, and hence run efficiently on vector enabled hardware. However RISC-V vector (RVV) is relatively new, and we experienced difficulty getting the performance we expected out of the tool flow. At the end of this talk, we hope our audience will have a better understanding of the challenges in bringing up an AI tool flow under QEMU. We hope out experience will help them bring up their own AI tool flows.

I would like to introduce our success story of adopting RISC-V CPU in the embedded domain with the ability to customize the architecture. For this success, we developed a scalable, orthogonal and transparent interrupt architecture which enabled the control of extremely timing-critical tasks. I believe this architecture can be widely adopted across multiple domains with the configurability to adopt to their own requirements.

The RISC-V Foundation's Olympia Performance Model is a great tool as the basis for designing a high-performance RISC-V CPU design. This session will provide a high-level overview of the Olympia Performance Model and then provide examples of how to use the model for tradeoff analysis on different RISC-V Out-of-Order superscalar designs.

The advancement of large language models (LLMs) has significantly enhanced natural language processing capabilities, enabling complex text understanding and generation tasks. This presentation focuses on optimizing the open-source LLaMA CPP project for the RISC-V P extension. By running the TinyLLaMA 1.1B model on the Andes Voyager development board using a quad-core CPU supporting the RISC-V P extension, performance results show that the model can achieve near real-time response. This work highlights the potential of RISC-V as an efficient platform for deploying advanced AI models in resource-constrained environments, contributing to the growing field of edge computing and embedded AI applications.

Berberis is an open source userspace dynamic binary translator facilitating cross-architecture development and testing of RISC-V Android applications. It translates native riscv64 code inside of an Android APK to x86_64 at runtime, enabling developers to test RISC-V builds of their apps on their workstations when target device hardware is unavailable. This presentation will cover the motivation and benefits of userspace translation versus whole-system emulation, the challenges of translating RISC-V code to x86_64, and use cases and future directions for the project.

Version 1.0.0 of the RISC-V Vector Cryptography extensions specification was ratified in late 2023 and adds high-performance cryptography operations to the comprehensive list of ISA features that RISC-V officially supports. We present the first open-source implementation of the RISC-V Vector Cryptography specification, using the PULP Project's Ara vector processor as a baseline and targeting a 28nm technology node. We present the design/verification opportunities and challenges that were encountered. Furthermore, a detailed review of the implementation and benchmarking results will be included in the presentation.

QEMU is an emulator that developers can developer and debug their software on it before getting the real RISC-V hardware. We observed that vectorized executables run much slower than non-vectorized ones on QEMU. From benchmarks (e.g. SPEC CPU2k6 h264), we can see that most of the execution time is occupied by RVV load/store instructions. The same observation has been reported in the QEMU community. For example, the glibc memcpy benchmark runs 2x to 60x slower than its scalar equivalent on QEMU. We aim to improve the performance of RVV instructions in QEMU, thereby reducing the execution time required for tasks such as Android bootup. In this talk, we will provide an overview of how we enhanced the performance of QEMU RVV load/store instructions and discuss future work.