Parallelizable Reachability Analysis Algorithms for Feed-Forward Neural Networks

Hoang Dung Tran, Patrick Musau, Diego Manzanas Lopez, Xiaodong Yang, Luan Viet Nguyen, Weiming Xiang, Taylor T. Johnson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

Artificial neural networks (ANN) have displayed considerable utility in a wide range of applications such as image processing, character and pattern recognition, self-driving cars, evolutionary robotics, and non-linear system identification and control. While ANNs are able to carry out complicated tasks efficiently, they are susceptible to unpredictable and errant behavior due to irregularities that emanate from their complex non-linear structure. As a result, there have been reservations about incorporating them into safety-critical systems. In this paper, we present a reachability analysis method for feed-forward neural networks (FNN) that employ rectified linear units (ReLUs) as activation functions. The crux of our approach relies on three reachable-set computation algorithms, namely exact schemes, lazy-approximate schemes, and mixing schemes. The exact scheme computes an exact reachable set for FNN, while the lazy-approximate and mixing schemes generate an over-approximation of the exact reachable set. All schemes are designed efficiently to run on parallel platforms to reduce the computation time and enhance the scalability. Our methods are implemented in a toolbox called, NNV, and is evaluated using a set of benchmarks that consist of realistic neural networks with sizes that range from tens to a thousand neurons. Notably, NNV successfully computes and visualizes the exact reachable sets of the real world ACAS Xu deep neural networks (DNNs), which are a variant of a family of novel airborne collision detection systems known as the ACAS System X, using a representation of tens to hundreds of polyhedra.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE/ACM 7th International Workshop on Formal Methods in Software Engineering, FormaliSE 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages51-60
Number of pages10
ISBN (Electronic)9781728133737
DOIs
StatePublished - May 2019
Event7th IEEE/ACM International Workshop on Formal Methods in Software Engineering, FormaliSE 2019 - Montreal, Canada
Duration: May 27 2019 → …

Publication series

NameProceedings - 2019 IEEE/ACM 7th International Workshop on Formal Methods in Software Engineering, FormaliSE 2019

Conference

Conference7th IEEE/ACM International Workshop on Formal Methods in Software Engineering, FormaliSE 2019
CountryCanada
CityMontreal
Period5/27/19 → …

Keywords

  • formal methods
  • machine learning
  • parallel algorithms
  • reachability analysis

ASJC Scopus subject areas

  • Software
  • Safety, Risk, Reliability and Quality

Fingerprint Dive into the research topics of 'Parallelizable Reachability Analysis Algorithms for Feed-Forward Neural Networks'. Together they form a unique fingerprint.

  • Cite this

    Tran, H. D., Musau, P., Manzanas Lopez, D., Yang, X., Nguyen, L. V., Xiang, W., & Johnson, T. T. (2019). Parallelizable Reachability Analysis Algorithms for Feed-Forward Neural Networks. In Proceedings - 2019 IEEE/ACM 7th International Workshop on Formal Methods in Software Engineering, FormaliSE 2019 (pp. 51-60). [8807491] (Proceedings - 2019 IEEE/ACM 7th International Workshop on Formal Methods in Software Engineering, FormaliSE 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/FormaliSE.2019.00012