RISC-V Summit

Today’s Artificial Intelligence (AI) companies and products are at the forefront of innovation, unlocking new markets and tackling the toughest technological challenges of the future. Innovation isn’t just a buzzword; it’s the gateway to new revenue streams and higher profits. At the heart of this innovation lies the need for new architectures that push the limits of performance while slashing costs and power consumption. This is where Custom Compute comes in – transforming these groundbreaking ideas into reality. But even the most advanced tech isn’t enough if it's not the right fit. To launch game-changing products that drive growth and maximize profits, they must be developed quickly and with confidence. That’s where Codasip Studio Fusion comes in – making Custom Compute the ultimate choice by eliminating Fear, Uncertainty, and Doubt, so you can innovate boldly and lead the market.

There has been huge interest in building accelerators for AI in the decade since AlexNet ushered in the current deep learning revolution. Billions of dollars in capital have been committed, and many ambitious projects have been launched, across established manufacturers, hyperscalers, and startups. In this talk, we will reflect on our experiences at Google designing and deploying successful accelerators and the different ways that subtle challenges make effective acceleration hard. RISC-V potentially helps with these challenges, while lowering barriers to entry, reducing risks, and sharing the benefit of expertise and experience. We will make connections between our experiences and how RISC-V accelerates accelerator development itself, highlighting how the shared work on a RISC-V ecosystem for deep learning acceleration can be positive-sum, benefiting all who participate.

The size of the Foundation models behind the Generative AI revolution have grown at a rate of more than 400x every 2 years, while DRAM memory capacity has been increasing only at 2x every two years, leading to what is commonly called the “memory wall”. Similarly, while the required throughput rate of LLMs making up the Foundation models has been increasing at 10x per year, the increase in computational capability of GPUs has been at a pace of only 10x in 4 years, leading to what is commonly called the “compute wall”. These trends have raised a new set of challenges in how to economically train these models, cost-effectively run them, and manage the tremendous increase in electrical power. The first contribution of this session are lessons learned in leveraging hardware and software developed for traditional AI workloads and how it was extended to support Generative AI. The session’s next main contribution is how we are applying lessons learned from our first-generation technology to our next generation. In this session’s final contribution, we will also discuss how the RISC-V ISA could be extended in ways that would make it more efficient and compelling at running Generative AI.

Our client is developing a massively parallel 64-bit chip for AI inference workloads. To facilitate early software development, we are bringing up an AI tool flow for this chip in a QEMU RISC-V environment. In this talk, we'll share our experience of getting three key AI frameworks working with RISC-V QEMU: Pytorch, Tensorflow and the OpenXLA compiler. Our talk will share our experience addressing two key issues. We will describe the challenges we faced, their solutions and reflect on the lessons learned for future work. The first of these is simply getting the tools to effectively run in an emulated RISC-V environment. These tools are large, fast moving pieces of software with extensive external dependencies. Our second challenge is performance. AI workloads are inherently parallel, and hence run efficiently on vector enabled hardware. However RISC-V vector (RVV) is relatively new, and we experienced difficulty getting the performance we expected out of the tool flow. At the end of this talk, we hope our audience will have a better understanding of the challenges in bringing up an AI tool flow under QEMU. We hope out experience will help them bring up their own AI tool flows.

The advancement of large language models (LLMs) has significantly enhanced natural language processing capabilities, enabling complex text understanding and generation tasks. This presentation focuses on optimizing the open-source LLaMA CPP project for the RISC-V P extension. By running the TinyLLaMA 1.1B model on the Andes Voyager development board using a quad-core CPU supporting the RISC-V P extension, performance results show that the model can achieve near real-time response. This work highlights the potential of RISC-V as an efficient platform for deploying advanced AI models in resource-constrained environments, contributing to the growing field of edge computing and embedded AI applications.

This deep dive will explain an end to end software stack solution to RISC-V based AI chips, including an innovation way to write AI kernels with new programming languages such as Triton (and Mojo later), using MLIR/LLVM based AI compiler infra to lower Triton kernels and neural networks from frameworks such as Pytorch, ONNX, Tensorflow and JAX into a range of high/middle/low level of MLIR dialects to do coarse grained high level optimizations such as loop tiling, kernel fusion, auto-vectorization etc. This paves the way of sharing common open source Triton kernels libraries provided in PyTorch and other frameworks, and greatly reduces the adoption time for AI software stack to RISC-V based AI chip. This talk will also explore the limitation of Triton language, and how can we extend the Triton language, and also the MLIR conversion and optimization passes to better support non GPU architecture target such as RISC-V.