A Fused Inference Design for Pattern-Based Sparse CNN on Edge Devices

Jia Guo, Radu Teodorescu, Gagan Agrawal

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Weight pruning approaches for Convolution Neural Networks (CNN) has been well developed in the past years. Compared with traditional unstructured and structured pruning, the new state-of-the-art sparse convolution pattern (SCP) based pruning uses certain patterns that lead to both high pruning rate and low accuracy loss. This paper introduce a novel inference scheme to accelerate the execution of SCP-pruned models on IoT devices with limited resources. This inference scheme applies and combines ideas from direct sparse convolution and layer fusion. To fully utilize the power of modern IoT processors, the inference is also mapped to all available cores and optimized with SIMD instructions. The experimental results show good performance improvement as well as scalability of our scheme on an edge device.

    Original languageEnglish (US)
    Title of host publicationProceedings - 2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics, HiPC 2021
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages424-429
    Number of pages6
    ISBN (Electronic)9781665410168
    DOIs
    StatePublished - 2021
    Event28th IEEE International Conference on High Performance Computing, Data, and Analytics, HiPC 2021 - Virtual, Bangalore, India
    Duration: Dec 17 2021Dec 18 2021

    Publication series

    NameProceedings - 2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics, HiPC 2021

    Conference

    Conference28th IEEE International Conference on High Performance Computing, Data, and Analytics, HiPC 2021
    Country/TerritoryIndia
    CityVirtual, Bangalore
    Period12/17/2112/18/21

    Keywords

    • Deep Neural Networks
    • Edge Computing

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Computer Networks and Communications
    • Computer Science Applications
    • Hardware and Architecture
    • Information Systems

    Fingerprint

    Dive into the research topics of 'A Fused Inference Design for Pattern-Based Sparse CNN on Edge Devices'. Together they form a unique fingerprint.

    Cite this