Multi-core, integrated CPU-GPU embedded systems provide new capabilities for sophisticated real-time systems with size, weight, and power limitations; however, interference between shared resources remains a challenge in providing necessary performance guarantees. The shared main memory is a notable system bottleneck - causing throughput slowdowns and timing unpredictability.
In this paper, we propose a full system mechanism which can provide memory bandwidth regulation across both CPU and the GPU complexes. This system monitors the memory controller accesses directly through hardware statistics counters, performs memory regulation at the software level for real-time CPU tasks, and incorporates a feedback-based throttling mechanism for non-critical GPU kernels using hardware within the NVIDIA Tegra X1 memory controller subsystem. The system is built as a loadable Linux kernel module that extends the MemGuard tool. We show that this system can make CPU task execution more predictable against co-running, memory intensive interference on either CPU or GPU.

Unmanned aerial vehicles (UAVs) and autonomous vehicles are becoming more ubiquitous than ever before. From medical to delivery drones, to space exploration rovers and self-driving taxi services, these vehicles are starting to play a prominent role in society as well as in our day to day lives.

 Efficient computation and communication strategies are paramount to the effective functioning of these vehicles. Mobile Edge Computing (MEC) is an innovative network technology that enables resource-constrained devices - such as UAVs and autonomous vehicles - to offload computationally intensive tasks to a nearby MEC server. Moreover, vehicles such as self-driving cars must reliably and securely relay and receive latency-sensitive information to improve traffic safety. Extensive research performed on vehicle to vehicle (V2V) and vehicle to everything (V2X) communication indicates that they will both be further enhanced by the widespread usage of 5G technology.

 We consider two relevant problems in mobile edge computing for unmanned vehicles. The first problem was to satisfy resource-constrained UAV's need for a resource-efficient offloading policy. To that end, we implemented both a computation and an energy consumption model and trained a DQN agent that seeks to maximize task completion and minimize energy consumption. The second problem was establishing communication between two autonomous vehicles and between an autonomous vehicle and an MEC server. To accomplish this goal, we experimented by leveraging an autonomous vehicle's server to send and receive custom messages in real time. These experiments will serve as a stepping stone towards enabling mobile edge computing and device-to-device communication and computation.

With the increased adoption of machine learning algorithms across many different fields, powerful computing platforms have become necessary to meet their computational needs. Multicore platforms are a popular choice due to their ability to provide greater computing capabilities and still meet the different size, weight, and power (SWaP) constraints. As a result, multicore systems are also being employed at an increasing rate. However, contention for hardware resources between the multiple cores is a significant challenge as it can lead to interference and unpredictable timing behaviors. Furthermore, this contention can be intentionally induced by malicious actors with the specific goals of inhibiting system performance and increasing the execution time of safety-critical tasks. This is done by performing Denial-of-Service (DoS) attacks that target shared resources in order to prevent other cores from accessing them. When done properly, these DoS attacks can have significant impacts to performance and can threaten system safety. For example, we find that DoS attacks can cause >300X slowdown on the popular Raspberry Pi 3 embedded platform. Due to the inherent risks, it is vital that we discover and understand the mechanisms through which shared resource contention can occur and develop solutions that mitigate or prevent the potential impacts.

In this work, we investigate and evaluate shared resource contention on multicore platforms and the impacts it can have on the performance of real-time tasks. Leveraging this contention, we propose various Denial-of-Service attacks that each target different shared resources in the memory hierarchy with the goal of causing as much slowdown as possible. We show that each attack can inflict significant temporal slowdowns to victim tasks on target platforms by exploiting different hardware and software mechanisms. We then develop and analyze techniques for providing shared resource isolation and temporal performance guarantees for safety-critical tasks running on multicore platforms. In particular, we find that bandwidth throttling mechanisms are effective solutions against many DoS attacks and can protect the performance of real-time victim tasks.

Convolutional Neural Networks (CNNs) have significantly improved the performance on various computer vision tasks such as image recognition and segmentation based on their rich representation power. To enhance the performance of CNN, a self-attention module is embedded after each layer in the network. Recently proposed Transformer-based models achieve outstanding performance by employing a multi-head self-attention module as the main building block. However, several challenges still need to be addressed, such as (1) focusing only on class-specified limited channels in CNN; (2) limited respective field in the local transformer; and (3) addition of redundant features and lack of multi-scale features in U-Net type segmentation architecture.

In our work, we propose new strategies to address these issues. First, we propose a novel channel-based self-attention module to diversify the focus more on the discriminative and significant channels, and the module can be embedded at the end of any backbone network for image classification. Second, to limit the noise added by the shallow layers of an encoder in U-Net type architecture, we replaced the skip connections with the Adaptive Global Context Module (AGCM). In addition, we introduced the Semantic Feature Enhancement Module (SFEM) for multi-scale feature enhancement in polyp segmentation. Third, we propose a Multi-scaled Overlapped Attention (MOA) mechanism in the local transformer-based network for image classification to establish the long-range dependencies and initiate the neighborhood window communication.

Code that can run asynchronously is important in a wide variety of situations, from user interfaces to communication over networks, to the use of concurrency for performance gains. One widely used method of specifying asynchronous control flow is the Promise model as used in Javascript. Promises are powerful, but can be confusing and hard-to-debug. This problem is exacerbated by Javascript’s permissive type system, where erroneous code is likely to fail silently, with values being implicitly coerced into unexpected types at runtime.

The present work implements Javascript-style Promises in Haskell, translating the model to a strongly typed framework where we can use the type system to rule out some classes of bugs.

Common errors – such as failure to call one of the callbacks of an executor, which would, in Javascript, leave the Promise in an eternally-pending deadlock state – can be detected for free by the type system at compile time and corrected without even needing to run the code.

We also demonstrate that Promises form a monad, providing a monad instance that allows code using Promises to be written using Haskell’s do notation.

Since convolutional neural network (CNN) was first implemented by Yann LeCun et al. in 1989, CNN and its variants have been widely implemented to numerous topics of pattern recognition, and have been considered as the most crucial techniques in the field of artificial intelligence and computer vision. This dissertation not only demonstrates the implementation aspect of CNN, but also lays emphasis on the methodology of neural network (NN) based classifier.

As known to many, one general pipeline of NN-based classifier can be recognized as three stages: pre-processing, inference by models, and post-processing. To demonstrate the importance of pre-processing techniques, this dissertation presents how to model actual problems in medical pattern recognition and image processing by introducing conceptual abstraction and fuzzification. In particular, a transformer on the basis of self-attention mechanism, namely beat-rhythm transformer, greatly benefits from correct R-peak detection results and conceptual fuzzification.

Recently proposed self-attention mechanism has been proven to be the top performer in the fields of computer vision and natural language processing. In spite of the pleasant accuracy and precision it has gained, it usually consumes huge computational resources to perform self-attention. Therefore, realtime global attention network is proposed to make a better trade-off between efficiency and performance for the task of image segmentation. To illustrate more on the stage of inference, we also propose models to detect polyps via Faster R-CNN - one of the most popular CNN-based 2D detectors, as well as a 3D object detection pipeline for regressing 3D bounding boxes from LiDAR points and stereo image pairs powered by CNN.

The goal for post-processing stage is to refine artifacts inferred by models. For the semantic segmentation task, the dilated continuous random field is proposed to be better fitted to CNN-based models than the widely implemented fully-connected continuous random field. Proposed approaches can be further integrated into a reinforcement learning architecture for robotics.

Quantum computing is a promising technology that can potentially demonstrate supremacy over classical computing in solving specific problems. At present, two critical challenges for quantum computing are quantum state decoherence, and low scalability of current quantum devices. Decoherence places constraints on realistic applicability of quantum algorithms as real-life applications usually require complex equivalent quantum circuits to be realized. For example, encoding classical data on quantum computers for solving I/O and data-intensive applications generally requires quantum circuits that violate decoherence constraints. In addition, current quantum devices are of small-scale having low quantum bit(qubit) counts, and often producing inaccurate or noisy measurements, which also impacts the realistic applicability of real-world quantum algorithms. Consequently, benchmarking of existing quantum algorithms and investigation of new applications are heavily dependent on classical simulations that use costly, resource-intensive computing platforms. Hardware-based emulation has been alternatively proposed as a more cost-effective and power-efficient approach. This work proposes a hardware-based emulation methodology for quantum algorithms, using cost-effective Field-Programmable Gate-Array(FPGA) technology. The proposed methodology consists of three components that are required for complete emulation of quantum algorithms; the first component models classical-to-quantum(C2Q) data encoding, the second emulates the behavior of quantum algorithms, and the third models the process of measuring the quantum state and extracting classical information, i.e., quantum-to-classical(Q2C) data decoding. The proposed emulation methodology is used to investigate and optimize methods for C2Q/Q2C data encoding/decoding, as well as several important quantum algorithms such as Quantum Fourier Transform(QFT), Quantum Haar Transform(QHT), and Quantum Grover’s Search(QGS). This work delivers contributions in terms of reducing complexities of quantum circuits, extending and optimizing quantum algorithms, and developing new quantum applications. For higher emulation performance and scalability of the framework, hardware design techniques and hardware architectural optimizations are investigated and proposed. The emulation architectures are designed and implemented on a high-performance-reconfigurable-computer(HPRC), and proposed quantum circuits are implemented on a state-of-the-art quantum processor. Experimental results show that the proposed hardware architectures enable emulation of quantum algorithms with higher scalability, higher accuracy, and higher throughput, compared to existing hardware-based emulators. As a case study, quantum image processing using multi-spectral images is considered for the experimental evaluations. 

Word embedding has become a popular form of data representation that is used to train deep neural networks in many natural
language processing tasks, such as Machine Translation, Question Answer Generation, Named Entity Recognition, Next
Word/Sentence Prediction etc. With embedding, each word is represented as a dense vector which captures its semantic relationship
with other words and can better empower Machine Learning models to achieve state-of-the-art performance.
However, due to the memory and time intensive nature of learning such word embeddings, transfer learning has emerged as a
common practice to warm start the training process. As a result, an efficient way is to initialize with pretrained word vectors and then
fine-tune those on downstream domain specific smaller datasets. This study aims to find whether we can infer the contextual
distribution (i.e., how words cooccur in a sentence driven by syntactic regularities) of the downstream datasets given that we have
access to the embeddings from both pre-training and fine-tuning processes.
In this work, we propose a focused sampling method along with a novel model inversion architecture “Invernet” to invert word
embeddings into the word-to-word context information of the fine-tuned dataset. We consider the popular word2Vec models
including CBOW, SkipGram, and GloVe based algorithms with various unsupervised settings. We conduct extensive experimental
study on two real-world news datasets: Antonio Gulli’s News Dataset from Hugging Face repository and a New York Times dataset
from both quantitative and qualitative perspectives. Results show that “Invernet” has been able to achieve an average F1 score of 0.75
and an average AUC score of 0.85 in an attack scenario.
A concerning pattern from our experiments reveal that embedding models that are generally considered superior in different tasks
tend to be more vulnerable to model inversion. Our results suggest that a significant amount of context distribution information from
the downstream dataset can potentially leak if an attacker gets access to the pretrained and fine-tuned word embeddings. As a result,
attacks using “Invernet” can jeopardize the privacy of the users whose data might have been used to fine-tune the word embedding
model.

Coq's extraction plugin is used to produce code of a general purpose
  programming language from a specification written in the Calculus of Inductive
  Constructions (CIC). Currently, this mechanism is trusted, since there is no
  formal connection between the synthesized code and the CIC terms it originated
  from. This comes from a lack of formal specifications for the target
  languages: OCaml, Haskell, and Scheme. We intend to use the formally specified
  CakeML language as an extraction target, and generate a theorem in Coq that
  relates the generated CakeML abstract syntax to the CIC terms it is generated
  from. This work expands on the techniques used in the HOL4 translator from
  Higher Order Logic to CakeML. The HOL4 translator also allows for the
  generation of stateful code from the state and exception monad. We expand on
  their techniques by extracting terms with dependent types, and generating
  stateful code for other kinds of monads, like the reader monad, depending on
  what kind of computation the monad intends to represent.

Recent years have seen tremendous developments in the field of computer vision and its extensive applications. The fundamental task, image classification, benefiting from deep convolutional neural networks (CNN)'s extraordinary ability to extract deep semantic information from input data, has become the backbone for many other computer vision tasks, like object detection and segmentation. A modern detection usually has bounding-box regression and class prediction with a pre-trained classification model as the backbone. The architecture is proven to produce good results, however, improvements can be made with closer inspections. A detector takes a pre-trained CNN from the classification task and selects the final bounding boxes from multiple proposed regional candidates by a process called non-maximum suppression (NMS), which picks the best candidates by ranking their classification confidence scores. The localization evaluation is absent in the entire process. Another issue is the classification uses one-hot encoding to label the ground truth, resulting in an equal penalty for misclassifications between any two classes without considering the inherent relations between the classes.

My research aims to address the following issues. (1) We proposed the first location-aware detection framework for single-shot detectors that can be integrated into any single-shot detectors. It boosts detection performance by calibrating the ranking process in NMS with localization scores. (2) To more effectively back-propagate gradients, we designed a super-class guided architecture that consists of a superclass branch (SCB) and a finer class branch (FCB). To further increase the effectiveness, the features from SCB with high-level information are fed to FCB to guide finer class predictions. (3) Recent works have shown 3D point cloud models are extremely vulnerable under adversarial attacks, which poses a serious threat to many critical applications like autonomous driving and robotic controls. To increase the robustness of CNN models on 3D point cloud models, we propose a family of robust structured declarative classifiers for point cloud classification, where the internal constrained optimization mechanism can effectively defend adversarial attacks through implicit gradients.