[AI 가속기] DianNao: A Small-Footprint High-Throughput Accelerator for Ubiquitous Machine-learning 논문 리뷰 (1/2)
J92022. 2. 18. 23:40
개인적인 공부 차원에서 정리합니다.
2014년에 발표된 AI 가속기의 시발점, 최초의 AI 가속기 논문으로 유명한 " DianNao: A Small-Footprint High-Throughput Accelerator for Ubiquitous Machine-Learning " (DianNao는 컴퓨터, 전자계산기라는 뜻)
Diannao, Eyeriess등으로 이어지는 중국 Chen (Tianshi Chen , Yunji Chen, Yu-Hsin Chen) 시리즈가 있고 AI 가속기를 연구하는 분들에게 필수 논문이라고함
Abstract
연구배경을 설명하고 논문 제목과 같이 memory footprint를 줄여 efficiency와 performance 를 높이는 large scale AI 가속기를 제안함.
결론적으로 논문은 tiling을 통해 off chip memory 대신 on chip memory 사용하는 방법, 그리고 이를 검증하기 위한 AI HW design에 대해 설명하고 있음.
Background - Machine learning has proven a feasibility and becoming pervasive - Traditional ML accelerator only focused on implementing the computational part of AL - Trend in large scale CNNs and DNNs
Goal - An accelerator for large-scale CNNS, DNNS with emphasis on the impact of memory considering performance and energy - Layout at 65nm - High throughput : 452GOP/s - small footprint : 3.02mm2 and 485mW - 117.87x faster , reduce 21.08x energy compared to 128bit 2GHz SIMD
1. Introduction
구체적인 motivation과 논문의 contribution 을 설명
Nerual network 기반 Maching learing이 대중화되고 Hetrogeneous systrem (cores and acclerators)로 아키텍쳐가 발전함에 따라 가속기를 통한 flexibility와 efficiency 최적화가 필요해짐.
현재까지의 CNN, MLP를 검증하고 연구했던 많은것들은 memory transfer를 무시하고 computational part에만 집중되어 왔음.
computational part가 결과를 확인하기 위한 첫번째 중요한 순서이나 결과적으로 memory transfer를 고려하지 않으면 성능, 효율, 비용 모든 면에서 문제가 될 수 있음. 이를 개선하기 위해 본 논문에서는 memory trasfer minimizing에 중점을 두고 연구하였으며 이에 따른 contrubution을 공유함
- Trend in heterogeneous system (multi-cores and acclerators), designing accelerator which realize the best possible tradeoff between flexibility and effiiciency - Previous research and workfocused on implementing the computational primitives , ignore memory transfers for simplicity -> need to care about Amdahl's law - Suggest an acceleratorfocus on memory usage , minimize memory transfer and perform efficiently
※ Amdahl's law (암달의 법칙)이란?
"the overall performance improvement gained by optimizing a single part of a system is limited by the fraction of time that the improved part is actually used" From 위키디피아 정보  하나의 성능을 무한대까지 끌어 올려도 결국 limitation이 존재, 결국 다른곳도 함께 개선을 해야함
※ Contribution
- A synthesized (place & route) accelerator design for large-scale CNNs and DNNs - high throughput in a small area, power and energy footprint - focus on memory behavior and measurements are not circumscribed to computational tasks they factor in the performance and energy impact of memory transfers
2. Primer on Recent Machine-Learning Techniques
NPU가 inferencing, training 모두 할 수 있지만 더 많은 end user가 있는 inferencing 이 주안점임
각 layer별 개선 아이디어를 설명하기 전 CNN의 general structure 대해 먼저 설명을 함
※ Processing vs training - Focus on foward propagation (inferencing NPU) because back propagation for training is small markets
※ General structure
Convolution layer - To apply one or several local filters to data from the input (previous) layer - 2D image (Kx X Ky) -> input feature map -> kernel (Ni) -> output feature map (Kx X Ky X Ni) - A non-linear function is applied to CONV layer output (ex : tanh(x) , Relu, sigmoid etc)
Pooling layer - To aggregate information among a set of neighbor input data - preferred techniques are the average and max operation
Classifier layers - The top of the hierarchies is usually a classifier - Classifiers aggregate(flatten) all feature maps, so there is no notion of feature maps in classifier layers Note) kernel value : synaptic weight
3. Processor-Based Implementation of (Large) Neural Networks
알고리즘 측면 main idea로 Classifier , Convolution, Pooling 주요 3개 layer에 대해 memory bandwidth 측면 개선점과 분석 결과를 보여줌
Tiling 기법을 통해 main memory access를 최소화하고 내부 on chip L1, L2 cache를 사용함 (Memory bandwidth 측정을 위해 virtual computational structure에 cache simulator를 추가)
Tiling은 input, output, synapses(weight), kernel 같은 모든 data들이 off chip memory대신 data travel distance가 짧은 L1, L2에 store될 만큼 잘라서 data reuse 하도록 하는것
CONV layer에서 remarkable improvement 확인됨
Classifier Layers - Input/Output - Input neuron are reused for each output neuron - Tiling input as Tii * Tnn to put it in L1 cache - No need to access off chip memory
- Synapses - In case of CNN and DNN with shared kernel, L2 cache can cover synapses data (synapses : weights)
Convolutional layers - Input/Output - sliding window (Kx * Ky / sx * sy, s=stride) used to scan 2D input layer - No need to tile for Nn because Kx * Ky * Ni fits in L1 cache - If not the case, tile input feature maps(ii)
- Synapses - in case of shared kernel, synaptic weights are reused across all output feature maps -> memory bandwidth is already low - If shared kernel capacity is bigger than L1 cache then tile again output feature maps -> Kx * Ky * Ni * No -> Kx * Ky * Ni * Tnn (depth can be tiling)
synapses가 모두 다르고 재사용이 불가능한 private kernel 을 사용한 CONV5는 synapses tiling이 불가능하여 main memory bandwidth 개선 안됨
논문에서도 private kernel 필요성에 대한 논쟁이 있다고 말하고 있고 현재는 shared kernel 사용하는것으로 보임
Pooling layers - The number of input and output feature is the same and no kernel (no synaptic weight to store) - sources of reuse comes from the sliding window (2D window, not like CONV 3D) - Compared to CONV layer , less memory bandwidth improvements
