Notre Dame

ID: N1

Title: Representation power of quantized neural networks: from the complexity bound to the
insight

Abstract: DNN compression is of great importance for implementing DNNs on
resource-constrained platforms. As a popular compression technique, quantization constrains
the number of distinct weight values and thus reducing the number of bits required to
represent and store each weight. While the representation power of conventional neural
networks is well-studied, the theoretical analysis of that for quantized neural networks is still
missing. There are many interesting questions remain unanswered. For example: 1) is quantized
neural networks more efficient than unquantized neural networks? 2) what is the optimal
bit-width for the parameters in neural networks? In order to answer these questions, we
propose to investigate the representation power of quantized neural networks especially its
comparison with unquantized neural networks. The upper bound and lower bound of the
network size for approximating any function with a given error bound for quantized neural
networks will be proved. Then the bounds derived will be used to provide early-stage resource
estimation and network design guideline.
Deliverable: 1) One submission to premium ML conference or journal; 2) Code and benchmarks
used in the experiment.

Proposed budget: $50K.

Performing site: Notre Dame (PI: Yiyu Shi).

ID: N2

Topic: Intelligent P-QRS-T peak detection for ultra-low power implantable devices

Abstract: T-wave alternans (TWA) detection is a very important clinical method and it is ideally
achieved in real-time with very limited computation power and battery capacitor for
implantable devices such as ICD. Most of the existing works either need high computation
power or are not accuracy enough. Using the dynamic time warping (DTW) method offers the
best tradeoff between accuracy and power consumption. We are proposing a method to select
the most efficient setting (for instance: the number of templates for the DTW algorithm) to
achieve the most energy efficient solution for ECG P-QRS-T peak detection.
Deliverable: 1) A submission to premium ML conference or journal; 2) Code and benchmarks
used in the experiment.

Proposed budget: $100K.

Performing site: Notre Dame (PI: Yiyu Shi).

ID: N3

Title: Guide for hybrid quantization of deconvolution-based generators

Abstract: As spin-offs from conventional convolutional neural networks (CNNs), GANs have
attracted much attention in the fields of reinforcement learning, unsupervised learning and also
semi-supervised learning, but they encounter the same heavy pressure from hardware as
conventional CNNs. Quantization is a popular, efficient and hardware-friendly compression
technique to alleviate the pressure on CNNs. However, directly adopting quantization to all
layers may fail deconvolution-based generator in generative adversarial networks based on our
observation. We propose to investigate the unique property of deconvolution-based neural
networks and apply hybrid quantization to obtain the best performance-cost trade-off. The
focus will be estimating the relative redundancy in each layer of deconvolution-based
generators. Based on the estimated redundancy, an effective guideline for the hybrid
quantization of deconvolution-based generators will be developed.
Deliverable: 1) A submission to premium ML conference or journal; 2) Code and benchmarks
used in the experiment.

Proposed budget: $50K.

Performing site: Notre Dame (PI: Yiyu Shi).

ID: N4

Topic: Hardware aware neural architecture search

Abstract: The success of DNN in a wide collection of applications mainly owes to the invention
of task-specific and well-tailored architectures. The traditional architecture engineering process
relies heavily on human effort and is unavoidably slow, error-prone, and limited of variety.
Recently, growing attention and effort have been cast to neural architecture search (NAS), a
process that automates the design of DNN architectures. The previous work on NAS, however,
only considers the accuracy at the sole objective to direct the searching but oversees the the
performance in hardware sense. We propose to incorporate hardware specifications such as
latency, power, and resources etc. into the NAS system to build a hardware-aware NAS
framework. Given the target hardware and performance for a specific task, a dedicated DNN
architecture can then be generated to meet all the requirements without any human
involvement.
Deliverable: 1) Two submissions to premium ML conference or journal; 2) Code and
benchmarks used in the experiment.

Proposed budget: $100K.

Performing site: Notre Dame (PI: Yiyu Shi).

 

ID: N5

Topic: SCNN: A general distribution based statistical convolutional neural network with
application to video object detection

Abstract: Various convolutional neural networks (CNNs) were developed recently that achieved
accuracy comparable with that of human beings in computer vision tasks such as image
recognition, object detection and tracking, etc. Most of these networks, however, process one
single frame of image at a time, and may not fully utilize the temporal and contextual
correlation typically present in multiple channels of the same image or adjacent frames from a
video, thus limiting the achievable throughput. This limitation stems from the fact that existing
CNNs operate on deterministic numbers. We propose a novel statistical convolutional neural
network (SCNN), which extends existing CNN architectures but operates directly on correlated
distributions rather than deterministic numbers. By introducing a parameterized canonical
model to model correlated data and defining corresponding operations as required for CNN
training and inference, we show that SCNN can process multiple frames of correlated images
effectively, hence achieving significant speedup over existing CNN models. We use a CNN based
video object detection as an example to illustrate the usefulness of the proposed SCNN as a
general network model.
Deliverable: 1) A submission to premium ML conference or journal; 2) Code used in the
experiment.

Proposed budget: $50K.

Performing site: Notre Dame (PI: Yiyu Shi)