How much "Brain Damage" can an LLM Tolerate? (2024)
Resistive RAM (RRAM) is a non-volatile memory technology that shows promise for deploying large language models on edge devices due to its high density and low power consumption, despite challenges like write endurance and inherent noise during read/write operations. The noise characteristics of RRAM can introduce errors in neural network weights, analogous to 'brain damage' in biological systems, which this article explores through simulated experiments on LLMs. The study draws on prior research about how such noise affects deep neural networks, particularly in image classification, and investigates how much degradation an LLM can tolerate before performance significantly deteriorates.
Opening excerpt (first ~120 words) tap to expand
How much “Brain Damage” can an LLM Tolerate? Figure 1: A boxer in a situation that may cause brain damage [1]. Resistive Memory or Resistive RAM (RRAM), a type of random access memory based on memristors, is an area of research that is experiencing ever increasing interest because of its unique combination of properties: It offers high density, low power consumption (when reading from it, we will get to that later), but is also persistent [2]. As a machine learning engineer, this makes it very attractive, as it could potentially open the door to deploy large models, including LLMs, to many more devices than is possible today, such as edge devices. Such a deployment scenario has many benefits, among them better privacy and security.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at HAWAII Lab.
Discussion
0 commentsMore from HAWAII Lab
