Macdown integring figure software#
Macdown integring figure code#

The energy efficiency on GAP-8 is 14.1 × higher than STM32L4 and 39.5 × higher than STM32H7, at the maximum efficiency operating point. The proposed library, when running on a GAP-8 processor, outperforms by 36.8 × and by 7.45 × the execution on energy efficient MCUs such as STM32L4 and high-end MCUs such as STM32H7 respectively, when operating at the maximum frequency. Using PULP-NN, a CIFAR-10 network on an octa-core cluster runs in 30 × and 19.6 × less clock cycles than the current state-of-the-art ARM CMSIS-NN library, running on STM32L4 and STM32H7 MCUs, respectively. The proposed library exploits both the digital signal processing extensions available in the PULP RISC-V processors and the cluster’s parallelism, achieving up to 15.5 MACs/cycle on INT-8 and improving performance by up to 63 × with respect to a sequential implementation on a single RISC-V core implementing the baseline RV32IMC ISA. The key innovation in PULP-NN is a set of kernels for quantized neural network inference, targeting byte and sub-byte data types, down to INT-1, tuned for the recent trend toward aggressive quantization in deep neural network inference. We present PULP-NN, an optimized computing library for a parallel ultra-low-power tightly coupled cluster of RISC-V processors. We release all our developments as open-source software. Using our tool, GAP-8 can perform end-to-end inference of a 1.0-MobileNet-128 network consuming just 63 pJ/MAC on average 4.3 fps - 15.4x better than an STM32-F746.

Macdown integring figure software#

On this device, DORY achieves up to 2.5x better MAC/cycle than the GWT proprietary software and 18.1x better than the state-of-the-art result on an STM32-F746 MCU on single layers. As a case study, we target GWT GAP8, an advanced parallel ultra-low power MCU.

Furthermore, DORY augments the CP formulation with heuristics promoting performance-effective tile sizes.

Macdown integring figure code#

Then, it generates C code to orchestrate off- and on-chip transfers and computation. DORY abstracts tiling as a Constraint Programming (CP) problem: it maximizes L1 memory utilization under the DNN's topological constraints. We propose DORY (Deployment Oriented to memoRY) - an automatic tool to deploy DNNs on MCUs with less than 1MB of on-chip SRAM memory. Mapping DNNs on these systems requires aggressive topology-dependent tiling and double-buffering. Low-Cost MCU-based end-nodes have limited on-chip memory and often replace caches with scratchpads, requiring explicit memory transfers. The deployment of Deep Neural Networks (DNNs) on end-nodes at the extreme edge of the Internet-of-Things is critical to support pervasive Deep Learning-enhanced applications. With a peak of 2.22 TOPs/s/W, the proposed solution achieves efficiency levels comparable with dedicated DNN inference accelerators, and up to three orders of magnitude better than state-of-the-art ARM Cortex-M based microcontroller systems such as the low-end STM32L4 MCU and the high-end STM32H7 MCU. QNN convolution kernels on a parallel cluster implementing the proposed extension run 6 x and 8 x faster when considering 4- and 2-bit data operands, respectively, compared to a baseline processing cluster only supporting 8-bit SIMD instructions. To evaluate the proposed extensions, we fully implement the cluster of processors in GF22FDX technology.

To further push the efficiency, we integrate the RISC-V extended core in a parallel cluster of 8 processors, with near-linear improvement with respect to a single core architecture. Also, we propose a custom execution paradigm for SIMD sum-of-dot-product operations, which consists of fusing a dot product with a load operation, with an up to 1.64x peak MAC/cycle improvement compared to a standard execution scenario. By extending the ISA with nibble (4-bit) and crumb (2-bit) SIMD instructions, we are able to show near-linear speedup with respect to higher precision integer computation on the key kernels for QNN computation. This work introduces lightweight extensions to the RISC-V ISA to boost the efficiency of heavily Quantized Neural Network (QNN) inference on microcontroller-class cores.