Defense Notices

Crowd estimation is an integral part of crowd analysis. It plays an important role in event management of huge gatherings like Hajj, sporting, and musical events or political rallies. Automated crowd count can lead to better and effective management of such events and prevent any unwanted incident. Crowd estimation is an active research problem due to different challenges pertaining to large perspective, huge variance in scale and image resolution, severe occlusions and dense crowd-like cluttered background regions. Current approaches cannot handle huge crowd diversity well and thus perform poorly in cases ranging from extreme low to high crowd-density, thus, leading to crowd underestimation or overestimation. Also, manual crowd counting subjects to very slow and inaccurate results due to the complex issues as mentioned above. To address the major issues and challenges in the crowd counting domain, we separately investigate two different types of input data: uni-modal (Image) and multi-modal (Image and Audio).

In the uni-modal setting, we propose and analyze four novel end-to-end crowd counting networks, ranging from multi-scale fusion-based models to uniscale one-pass and two-pass multi-task models. The multi-scale networks also employ the attention mechanism to enhance the model efficacy. On the other hand, the uni-scale models are equipped with novel and simple-yet-effective patch re-scaling module (PRM) that functions identical but lightweight in comparison to the multi-scale approaches. Experimental evaluation demonstrates that the proposed networks outperform the state-of-the-art methods in majority cases on four different benchmark datasets with up to 12.6% improvement in terms of the RMSE evaluation metric. Better cross-dataset performance also validates the better generalization ability of our schemes. For the multimodal input, the effective feature-extraction (FE) and strong information fusion between two modalities remain a big challenge. Thus, the aim in the multimodal environment is to investigate different fusion techniques with improved FE mechanism for better crowd estimation. The multi-scale uni-modal attention networks are also proven to be more effective in other deep leaning domains, as applied successfully on seven different scene-text recognition datasets with better performance.

Machine learning (ML) is transforming a wide range of applications, promising to bring immense economic and social benefits. However, it also raises substantial security and privacy challenges.  In this dissertation we describe a framework for efficient, collaborative and secure ML training using a federation of client devices that jointly train a ML model using their private datasets in a process called Federated Learning (FL). First, we present the design of a blockchain-enabled Privacy-preserving Federated Transfer Learning (PPFTL) framework for resource-constrained IoT applications. PPFTL addresses the privacy challenges of FL and improves efficiency and effectiveness through model personalization. The framework overcomes the computational limitation of on-device training and the communication cost of transmitting high-dimensional data or feature vectors to a server for training. Instead, the resource-constrained devices jointly learn a global model by sharing their local model updates. To prevent information leakage about the privately-held data from the shared model parameters, the individual client updates are homomorphically encrypted and aggregated in a privacy-preserving manner so that the server only learns the aggregated update to refine the global model. The blockchain provides provenance of the model updates during the training process, makes contribution-based incentive mechanisms deployable, and supports traceability, accountability and verification of the transactions so that malformed or malicious updates can be identified and traced to the offending source. The framework implements model personalization approaches (e.g. fine-tuning) to adapt the global model more closely to the individual client's data distribution.

In the second part of the dissertation, we turn our attention to the limitations of existing FL algorithms in the presence of adversarial clients who may carry out poisoning attacks against the FL model. We propose a privacy-preserving defense, named CONTRA, to mitigate data poisoning attacks and provide a guaranteed level of accuracy under attack.  The defense strategy identifies malicious participants based on the cosine similarity of their encrypted gradient contributions and removes them from FL training. We report the effectiveness of the proposed scheme for IID and non-IID data distributions. To protect data privacy, the clients' updates are combined using secure multi-party computation (MPC)-based aggregation so that the server only learns the aggregated model update without violating the privacy of users' contributions.

Swarms of unmanned aerial systems (UASs) usage is becoming more prevalent in the world. Many private companies and government agencies are actively developing analytical and technological solutions for multi-agent cooperative swarm of UASs.  However, majority of existing research focuses on developing guidance, navigation, and control (GNC) algorithms for swarm of UASs and proof of stability and robustness of those algorithms. In addition to profound challenges in control of swarm of UASs, a reliable and fast intercommunication between UASs is one of the vital conditions for success of any swarm.  Many modern UASs have high inertia and fly at high speeds which means if latency or throughput are too low in swarms, there is a higher risk for catastrophic failure due to intercollision within the swarm. This work presents solutions for scaling number of collaborative agents in swarm of UASs using frequency-based hierarchy. This work identifies shortcomings and discusses traditional swarm communication systems and how they rely on a single frequency that will handle distribution of information to all or some parts of a swarm. These systems typically use an ad-hoc network to transfer data locally, on the single frequency, between agents without the need of existing communication infrastructure. While this does allow agents the flexibility of movement without concern for disconnecting from the network and managing only neighboring communications, it doesn’t necessarily scale to larger swarms. In those large swarms, for example, information from the outer agents will be routed to the inner agents. This will cause inner agents, critical to the stability of a swarm, to spend more time routing information than transmitting their state information. This will lead to instability as the inner agents’ states are not known to the rest of the swarm. Even if an ad-hoc network is not used (e.g. an Everyone-to-Everyone network), the frequency itself has an upper limit to the amount of data that it can send reliably before bandwidth constraints or general  interference causes information to arrive too late or not at all.

We propose that by using two frequencies and creating a hierarchy where each layer is a separate frequency, we can group large swarms into manageable local swarms. The intra-swarm communication (inside the local swarm) will be handled on a separate frequency while the inter-swarm communication will have its own. A normal mesh network was tested in both hardware in the loop (HitL) scenarios and a collision avoidance flight test scenario. Those results were compared against dual-frequency HitL simulations. The dual-frequency simulations showed overall improvement in the latency and throughput comparatively to both the simulated and flight-tested mesh network.

In the last decade, the discovery of noncoding RNA (ncRNA) has exploded. Classifying these ncRNA is critical to determining their function. This thesis proposes a new method employing deep convolutional neural networks (CNNs) to classify ncRNA sequences. To this end, this thesis first proposes an efficient approach to convert the RNA sequences into images characterizing their base-pairing probability. As a result, classifying RNA sequences is converted to an image classification problem that can be efficiently solved by available CNN-based classification models. This thesis also considers the folding potential of the ncRNAs in addition to their primary sequence. Based on the proposed approach, a benchmark image classification dataset is generated from the RFAM database of ncRNA sequences. In addition, three classical CNN models and three Siamese network models have been implemented and compared to demonstrate the superior performance and efficiency of the proposed approach. Extensive experimental results show the great potential of using deep learning approaches for RNA classification.

In recent years, the problem of real-time scheduling has increasingly become more important as well as more complicated. The former is due to the proliferation of safety critical systems into our day-to-day life and the latter is caused by the escalating demand for high performance which is driving the multicore architecture towards consolidation of various kinds of heterogeneous computing resources into smaller and smaller SoCs. Motivated by these trends, this dissertation tackles the following fundamental question: how can we guarantee predictable real-time execution while preserving high utilization on heterogeneous multicore SoCs?

This dissertation presents new real-time scheduling techniques for predictable and efficient scheduling of mixed criticality workloads on heterogeneous SoCs. The contributions of this dissertation include the following: 1) a novel CPU-GPU scheduling framework, called BWLOCK++, that ensures predictable execution of critical GPU kernels on integrated CPU-GPU platforms 2) a novel gang scheduling framework called RT-Gang, which guarantees deterministic execution of parallel real-time tasks on the multicore CPU cluster of a heterogeneous SoC. 3) optimal and heuristic algorithms for gang formation that increase real-time schedulability under the RT-Gang framework and their extension to incorporate scheduling on accelerators in a heterogenous SoC. 4) A case-study evaluation using an open-source autonomous driving application that demonstrates the analytical and practical benefits of the proposed scheduling techniques.

So much of what we do on a daily basis is dependent on computers: email, social media, online gaming, banking, online shopping, virtual conference calls, and general web browsing to name a few. Most of the devices we depend on for these services are computers or servers that we do not own, nor do we have direct physical access to. We trust the underlying network to provide access to these devices remotely. But how do we know which computers/servers are safe to access, or verify that they are who they claim to be? How do we know that a distant server has not been hacked and compromised in some way?

Remote attestation is a method for establishing trust between remote systems. An "appraiser" can request information from a "target" system. The target responds with "evidence" consisting of run-time measurements, configuration information, and/or cryptographic information (i.e. hashes, keys, nonces, or other shared secrets). The appraiser can then evaluate the returned evidence to confirm the identity of the remote target, as well as determine some information about the operational state of the target, to decide whether or not the target is trustworthy.

A tool that may prove useful in remote attestation is the TPM, or "Trusted Platform Module". The TPM is a dedicated microcontroller that comes built-in to nearly all PC and laptop systems produced today. The TPM is used as a root of trust for storage and reporting, primarily through integrated cryptographic keys. This root of trust can then be used to assure the integrity of stored data or the state of the system itself. In this thesis, I will explore the various functions of the TPM and how they may be utilized in the development of the remote attestation language, "Copland".

First Topic: Ice sheets impact sea-level change and hence their response to climatic variations needs to be continually monitored and studied. We propose to apply multipass differential interferometric synthetic aperture radar (DInSAR) techniques to data from the Multichannel Coherent Radar Depth Sounder (MCoRDS) to measure the vertical displacement of englacial layers within an ice sheet. DInSAR’s accuracy is usually on the order of a small fraction of the wavelength (e.g. millimeter to centimeter precision is common) in monitoring ground displacement along the radar line of sight (LOS).  In the case of ice sheet internal layers, vertical displacement is estimated by compensating for the spatial baseline using precise trajectory information and estimates of the cross-track layer slope from direction of arrival analysis. Preliminary results from a high accumulation region near Camp Century in northwest Greenland and Summit Station in central Greenland are presented here. We propose to extend this work by implementing a maximum likelihood estimator that jointly estimates the vertical velocity, the cross-track internal layer slope, and the unknown baseline error due to GPS and INS errors. The multipass algorithm will be applied to additional flights from the decade long NASA Operation IceBridge airborne mission that flew MCoRDS on many repeated flight tracks. We also propose to improve the accuracy of tomographic swaths produced from multipass measurements and investigate the possibility to use focusing matrices to improve wideband tomographic processing.

Second Topic: With the increased demand for bandwidth-hungry applications in the telecommunications industry, radar applications can no longer enjoy the generous frequency allocations within the UHF band. Spectral efficiency, if achievable, leads to the freeing of portions of the radar bandwidth to facilitate spectrum sharing between radar and other wireless systems. A decrease in bandwidth leads to worse radar resolution. In certain scenarios, reduced resolution is acceptable, and bandwidth may be compromised for spectral efficiency. An iterative reduced rank MMSE algorithm based on marginal Fisher information is proposed and investigated to minimize the loss of resolution with the tradeoff of degraded side-lobe performance. The algorithm is applied to the radar measurement model with simulated range profiles and performance results discussed.

 Even though fiber optics communication is providing a high bandwidth channel to achieve high speed data transmission, there is still a need for higher spectral efficiency, faster data processing speeds while reduced resource requirements due to ever increasing data and media traffic. Various multilevel modulation and demodulation techniques are used to improve spectral efficiency. Although, spectral efficiency is improved, there are other challenges that arise while doing so such as requirement for high speed electronics, receiver sensitivity, chromatic dispersion, operational flexibility etc. Here, we investigate complex high speed field modulation techniques in direct detection systems to improve spectral efficiency while focusing to reduce resources required for implementation, compensating for linear and nonlinear impairments in fiber optics communication systems.

We first demonstrated a digital-analog hybrid subcarrier multiplexing (SCM) technique which can reduce the requirement of high speed electronics such as ADC and DAC, while providing wideband capability, high spectral efficiency, operational flexibility and controllable data-rate granularity.

With conventional Quadrature Phase Shift Keying (QPSK), to achieve maximum spectral efficiency, we need high spectral efficient Nyquist filters which takes high FPGA resources for digital signal processing (DSP). Hence, we investigated Quadrature Duobinary (QDB) modulation as a solution to reduce the FPGA resources required for DSP while achieving spectral efficiency of 2bits/s/Hz. Currently we are investigating all analog single sideband (SSB) complex field modulated direct detection system. Here, we are trying to achieve higher spectral efficiency by using QDB modulation scheme in comparison to QPSK while avoiding signal-signal beat interference (SSBI) by providing a guard-band based approach.

Another topic we investigated, both through simulation and experiments, is a way to compensate for nonlinearities generated by semiconductor optical amplifiers (SOA) when operated in gain saturation in a field modulated direct detection systems. We successfully, compensated for the SOA nonlinearities in the presence of fiber chromatic dispersion, which was post compensated using electronic dispersion compensation after restoring the phase information of the received signal using Kramers-Kronig receiver.

Multichannel synthetic aperture radar (SAR) sounders with cross-track antenna arrays map ice sheet basal morphology in three dimensions with a single pass using tomography.  The tomographic ice-sheet imaging method leverages parametric direction-finding techniques like the Maximum Likelihood Estimator and the Multiple Signal Classification algorithm to resolve scattering interfaces in elevation.  These techniques have received considerable attention because of their potential to exceed the Rayleigh resolution limit of the receive array under certain conditions.  This performance is predicated on having perfect knowledge of the frequency-dependent response of the array to directional sources, referred to as the array manifold.  Even modest amounts of mismatch between the assumed and actual manifold model degrade the accuracy of parametric angle estimators and erode their sought-after superresolution potential.

Array manifold calibration refers to the step in the array processor of refining our representation of the directional array-response vectors by accounting for factors such as mutual coupling, geometric uncertainties, and channel-to-channel gain imbalances.  Pilot calibration requires measuring the in-situ array over its field of view and storing the manifold in a look-up-table.  Alternatively, the array transfer function may be modeled parametrically to levy an estimation framework for characterizing mismatch.  Parametric calibration theory for sensor position perturbations has been established for several decades.  However, there remains a marked disconnect between the signal processing and antennas communities regarding how to include mutual coupling within the parametric framework.  To date, literature lacks validated studies that address parameterization of the embedded element patterns for direction-finding arrays.

A manifold calibration methodology is proposed for an airborne, multichannel ice-penetrating SAR.  The methodology departs from conventional approaches by extracting calibration targets from SAR imagery of well-understood terrain to empirically characterize the directional responses of the integrated array's embedded element patterns.  This work presents a Maximum Likelihood Estimator for nonlinear parameters common across disjoint calibration sets that has the potential to improve the accuracy of our estimated geometric uncertainties by increasing the total Fisher information in our observations.  The investigation contributes to specific gaps in array signal processing and remote sensing literature by treating the unique challenge of calibrating in-situ arrays used in direction-finding applications.

Densifying the network by deploying many small cells has attracted significant interests from wireless industries for exploring its potential to facilitating the proposed many data-intensive use cases in fifth-generation (5G) networks. While such efforts are essential, there are gaps in fundamental research and practical deployment of small cells. It is clear that increased interference from adjacent cells, called intercell interference, is the major limiting factor.  In order to address this issue, each base station's parameters should be properly controlled to mitigate the intercell interference. We call the task of designing the base station's parameters the base station optimization (BSO) problem in this work. Due to the large numbers of small cells and mobile users distributed over the network, solving BSO by precisely modeling the network conditions is almost infeasible. One of the popular approaches that has attracted many researchers recently is a data-based framework called machine learning (ML). While supervised ML is prevalent, it requires pre-labeled off-line data that are not available in many wireless scenarios. Unlike supervised ML, reinforcement learning (RL) can handle this situation because it is based on designing a good policy to find the best exploration-\&-exploitation tradeoff without the pre-labeled training dataset. Thus, in this work, we present a new approach to the problem of BSO, based on the application of deep reinforcement learning (DRL) to enhance the quality of service (QoS) experienced by mobile users. To speed up the exploration of DRL, we employ particle swarm optimization (PSO), which shows improved QoS and convergence compared to conventional DRL.