Defense Notices

Person re-identification (Re-ID) and multi-object tracking in unconstrained surveillance environments pose significant challenges within the field of computer vision. These complexities stem mainly from occlusion, variability in appearance, and identity switching across various camera views. This research outlines a comprehensive and innovative agenda aimed at tackling these issues, employing a series of increasingly advanced deep learning architectures, culminating in a groundbreaking occlusion-aware Vision Transformer framework.

At the heart of this work is the introduction of Deep SORT with Multiple Inputs (Deep SORT-MI), a cutting-edge real-time Re-ID system featuring a dual-metric association strategy. This strategy adeptly combines Mahalanobis distance for motion-based tracking with cosine similarity for appearance-based re-identification. As a result, this method significantly decreases identity switching compared to the baseline SORT algorithm on the MOT-16 benchmark, thereby establishing a robust foundation for metric learning in subsequent research.

Expanding on this foundation, a novel pose-estimation framework integrates 2D skeletal keypoint features extracted via OpenPose directly into the association pipeline. By capturing the spatial relationships among body joints along with appearance features, this system enhances robustness against posture variations and partial occlusion. Consequently, it achieves substantial reductions in false positives and identity switches compared to earlier methods, showcasing its practical viability.

Furthermore, a Diverse Detector Integration (DDI) study meticulously assessed the influence of detector choices—including YOLO v4, Faster R-CNN, MobileNet SSD v2, and Deep SORT—on the efficacy of metric learning-based tracking. The results reveal that YOLO v4 consistently delivers exceptional tracking accuracy on both the MOT-16 and MOT-17 datasets, establishing its superiority in this competitive landscape.

In conclusion, this body of research notably advances occlusion-aware person Re-ID by illustrating a clear progression from metric learning to pose-guided feature extraction and ultimately to transformer-based global attention modeling. The findings underscore that lightweight, meticulously parameterized Vision Transformers can achieve impressive generalization for occlusion detection, even under constrained data scenarios. This opens up exciting prospects for integrated detection, localization, and re-identification in real-world surveillance systems, promising to enhance their effectiveness and reliability.

This thesis addresses two related problems in the optical manipulation of microscopic particles: the efficient generation of holograms for holographic optical tweezers and the simulation of multi-particle optical binding. Holographic optical tweezers use phase-only spatial light modulators to create programmable optical trapping fields, enabling dynamic control over the number, position, and relative strength of optical traps. Because the quality of the trapping field depends strongly on the computed hologram, the first part of this work focuses on improving hologram-generation methods used in these systems.

A new phase-induced compressive sensing algorithm is presented for holographic optical tweezers, along with weighted and unweighted variants. These methods are developed from the Gerchberg-Saxton framework and are designed to improve computational efficiency while preserving favorable trapping characteristics such as uniformity and optical efficiency. By combining compressive sensing with phase induction, the proposed algorithms reduce the computational burden associated with iterative hologram generation while maintaining strong performance across a variety of trapping arrangements. Comparative simulations are used to evaluate these methods against several established hologram-generation algorithms, and the results show that the proposed approaches offer meaningful improvements in convergence behavior and overall performance.

The second part of this thesis examines optical binding, a phenomenon in which multiple particles interact through both the incident optical field and the fields scattered by neighboring particles. To study this process, a numerical simulation is developed that incorporates gradient forces, radiation pressure, and light-mediated particle-particle interactions in both two- and three-dimensional configurations. The simulation is used to investigate how particles evolve under different initial conditions and illumination states, and how collective effects influence the formation of stable or semi-stable arrangements. These results provide insight into the role of scattering-mediated forces in many-particle optical systems and highlight differences between two-dimensional and three-dimensional behavior.

Although hologram generation and optical binding are treated as separate problems in this work, they are connected by a common goal: understanding how structured optical fields can be designed and applied to control microscopic matter. Together, the results of this thesis contribute to the broader study of computational beam shaping and many-body optical interactions, with relevance to advanced optical trapping, particle organization, and dynamically reconfigurable light-driven systems.

Runtime estimation plays a structural role in reservation-based scheduling for High Performance Computing (HPC) systems, where predicted walltimes directly influence reservation timing, backfilling feasibility, and overall queue dynamics. This raises a fundamental question of whether improved runtime prediction accuracy necessarily translates into improved scheduling performance. In this work, we conduct an empirical study of runtime estimation under EASY Backfilling using an application-driven workload consisting of MRI-based brain segmentation jobs. Despite identical configurations and uniform metadata, runtimes exhibit substantial variability driven by intrinsic input structure. To capture this variability, we develop a feature-driven machine learning (ML) framework that extracts region-wise features from MRI volumes to predict job runtimes without relying on historical execution traces or scheduling metadata. We integrate these ML-derived predictions into an EASY Backfilling scheduler implemented in the Batsim simulation framework. Our results show that regression accuracy alone does not determine scheduling performance. Instead, scheduling performance depends strongly on estimation bias and its effect on reservation timing and runtime exceedances. In particular, mild multiplicative calibration of ML-based runtime estimates stabilizes scheduler behavior and yields consistently competitive performance across workload and system configurations. Comparable performance can also be observed with certain levels of uniform overestimation; however, calibrated ML predictions provide a systematic mechanism to control estimation bias without relying on arbitrary static inflation. In contrast, underestimation consistently leads to severe performance degradation and cascading job terminations. These findings highlight runtime estimation as a structural control input in backfilling-based HPC scheduling and demonstrate the importance of evaluating prediction models jointly with scheduling dynamics rather than through regression metrics alone.

Modern intelligent systems increasingly rely on continuous sensor data streams for perception, decision-making, and control, making sensors a critical yet underexplored attack surface. While prior research has demonstrated the feasibility of sensor-based attacks, recent advances in mobile operating systems and machine learning-based defenses have significantly reduced their practicality, rendering them more detectable, resource-intensive, and constrained by evolving permission and context-aware security models.

This dissertation revisits sensor exploitation under these modern constraints and develops a unified, cross-layer perspective that improves both practicality and stealth of sensor-enabled attacks. We identify three fundamental challenges: (i) the difficulty of reliably manipulating physical sensor signals in noisy, real-world environments; (ii) the effectiveness of context-aware defenses in detecting anomalous sensor behavior on mobile devices, and (iii) the lack of lightweight coordination for practical sensor-based side- and covert-channels.

To address the first challenge, we propose a physical-domain attack framework that integrates signal modeling, simulation-guided attack synthesis, and real-time adaptive targeting, enabling robust adversarial perturbations with high attack success rates even under environmental uncertainty. As a case study, we demonstrate an infrared laser-based adversarial example attack against face recognition systems, which achieves consistently high success rates across diverse conditions with practical execution overhead.

To improve attack stealth against context-aware defenses, we introduce an auto-contextualization mechanism that synchronizes malicious sensor actuation with legitimate application activity. By aligning injected signals with both statistical patterns and semantic context of benign behavior, the approach renders attacks indistinguishable from normal system operations and benign sensor usage. We validate this design using three Android logic bombs, showing that auto-contextualized triggers can evade both rule-based and learning-based detection mechanisms.

Finally, we extend sensor exploitation beyond the traditional attack-channel plane by introducing a lightweight control-plane protocol embedded within sensor data streams. This protocol encodes control signals directly into sensor observations and leverages simple signal-processing primitives to coordinate multi-stage attacks without relying on privileged APls or explicit inter-process communication. The resulting design enables low-overhead, stealthy coordination of cross-device side- and covert-channels.

Together, these contributions establish a new paradigm for sensor exploitation that spans physical, contextual, and control-plane dimensions. By bridging these layers, this dissertation demonstrates that sensor-based attacks remain not only feasible but also practical and stealthy in modern computer systems.

***Currently being reviewed, unavailable***

Optical Neural Networks (ONNs) have gained traction as an alternative to the conventional computing architectures used in modern CPUs and GPUs, largely because light enables massive parallelism, ultrafast inference, and minimal power consumption. 

As with conventional deep neural networks (DNNs), free-space ONNs require two main layers: (1) a nonlinear activation function which exists to separate adjacent linear layers, and (2) weighting layers that applies a linear transformation given an input.

Firstly, a Resonant Semiconductor Saturable Absorption Mirror (RSAM) was investigated as a viable nonlinear activation function. Several mechanisms have been used to create nonlinear activation functions, such as cold atoms, vapor absorption cells, and polaritons, but these implementations are bulky and must operate under tightly controlled environments while RSAMs is a passive device. Compared to typical SESAMs, the resonance structure of RSAM also reduces the saturation fluence compared to non-resonant SAMs, allowing low power laser sources to be used. A fiber-based optical testbed demonstrated notable improvement of 8.1% in classification accuracy compared to a linear only network trained with the MNIST dataset.

Secondly, Micro-electromechanical-system-based phase light modulators (PLMs) were evaluated as an alternative to LC-SLMs for in-situ reinforcement learning. PLMs can operate at kilohertz-scale frame rates at a substantially lower cost compared to LC-SLMs but have lower phase resolution and non-uniform quantization which impacts fidelity. Despite these disadvantages, the high-speed nature of PLMs allows for significant decrease in optimization time, which not only allows for reduction in training time, but also allows for larger datasets and more complex models with more learnable parameters. A single layer optical network was implemented using policy-based learning with discrete action-space to minimize impact of quantization. The testbed achieves 90.1%, 79.7%, and 76.9% training, validation, and test accuracy, respectively, on 3,000 images from the MNIST dataset. Additionally, we achieved 79.9%, 72.1%, and 71.7% accuracy on 3,000 images from the Fashion MNIST dataset. At 14 minutes per epoch during training, it is at least a magnitude lower in training time compared to LC-SLMs based models.

Checkpointing strategies in high-performance computing traditionally employ the Young-Daly for-

mula to determine the (first-order) optimal duration between checkpoints, which assumes a known

mean time between faults (MTBF). However, in practice, the MTBF may not be known accurately

or may vary, causing Young-Daly checkpointing to perform sub-optimally. In 2021, Sigdel et al.

introduced the CHORE (CHeckpointing Overhead and Rework Equated) checkpointing strategy,

which is MTBF-agnostic yet demonstrates a bounded increase in overhead compared to the op-

timal strategy. This thesis analyzed and extends the CHORE framework in several ways. First,

it verifies Sigdel et al.’s claims about the relative overhead of the CHORE strategy through both

event-driven simulations and expected runtimes derived from the underlying probablistic model.

Second, it extends the CHORE strategy to silent errors, which must be deliberately checked for to

be detected. In this scenario, the overhead compared to optimal checkpointing is once more ana-

lyzed through simulations and expected runtimes. Third, a heuristic is proposed to offer improved

performance of the CHORE algorithm under typical runtime scenarios by interpreting CHORE as

an additive-increase multiplicative-decrease model and tuning the parameters.

Metagenomic sequencing has become a central paradigm for studying complex microbial communities and their interactions with the host, with emerging applications in clinical prediction and disease modeling. In this work, we first investigate two representative application scenarios: predicting immune checkpoint inhibitor response in non-small cell lung cancer using gut microbial signatures, and characterizing host–microbiome interactions in neonatal systems. The proposed reference-free neural network captures both compositional and functional signals without reliance on reference genomes, while the neonatal study demonstrates how environmental and genetic factors reshape microbial communities and how probiotic intervention can mitigate pathogen-induced immune activation.

These studies highlight both the promise and the inherent difficulty of metagenomic analysis: transforming raw sequencing data into clinically actionable insights remains an algorithmically fragmented and computationally intensive process. This challenge arises from two key limitations: the lack of a unified algorithmic foundation for sequence alignment and the absence of systematic approaches for selecting and organizing analytical tools. Motivated by these challenges, we present a unified computational framework for metagenomic analysis that integrates complementary algorithmic and systems-level solutions.

First, to resolve fragmentation at the alignment level, we develop the Versatile Alignment Toolkit (VAT), a unified algorithmic system for biological sequence alignment across diverse applications. VAT introduces an asymmetric multi-view k-mer indexing scheme that integrates multiple seeding strategies within a single architecture and enables dynamic seed-length adjustment via longest common prefix (LCP)–based inference without re-indexing. A flexible seed-chaining mechanism further supports diverse alignment scenarios, including collinear, rearranged, and split alignments. Combined with a hardware-efficient in-register bitonic sorting algorithm and dynamic index-loading strategy, VAT achieves high efficiency and broad applicability across read mapping, homology search, and whole-genome alignment. Second, to address the challenge of tool selection and pipeline construction, we develop SNAIL, a natural language processing system for automated recognition of bioinformatics tools from large-scale and rapidly growing scientific literature. By integrating XGBoost and Transformer-based models such as SciBERT, SNAIL enables structured extraction of analytical tools and supports automated, reproducible pipeline construction.

Together, this work establishes a unified framework that is grounded in real-world applications and addresses key bottlenecks in metagenomic analysis, enabling more efficient, scalable, and clinically actionable workflows.

Graph Neural Networks (GNNs) achieve state-of-the-art performance in modeling complex biological and behavioral systems, yet their "black-box" nature limits their utility for scientific discovery and clinical translation. Standard post-hoc explainability methods typically attribute importance to low-level features, such as individual nodes or edges, which often fail to map onto the high-level, domain-specific concepts utilized by experts. To address this gap, this thesis explores diverse methodological strategies for achieving Concept-Level Interpretability in GNNs, demonstrating how deep learning models can be structurally and analytically aligned with expert domain knowledge. This theme is explored through two distinct methodological paradigms applied to critical challenges in neuroscience and clinical psychology. First, we introduce an interpretable-by-design approach for modeling brain structure-function coupling. By employing an ensemble of GNNs conceptually biased via input graph filtering, the model enforces verifiably disentangled node embeddings. This allows for the quantitative testing of specific structural hypotheses, revealing that a minority of strong anatomical connections disproportionately drives functional connectivity predictions. Second, we present a post-hoc conceptual alignment paradigm for quantifying atypical motor signatures in Autism Spectrum Disorder (ASD). Utilizing a Spatio-Temporal Graph Autoencoder (STGCN-AE) trained on normative skeletal data, we establish an unsupervised anomaly detection system. To provide clinical interpretability, the model's reconstruction error is systematically aligned with a library of human-interpretable kinematic features, such as postural sway and limb jerk. Explanatory meta-modeling via XGBoost and SHAP analysis further translates this abstract loss into a multidimensional clinical signature. Together, these applications demonstrate that integrating concept-level interpretability through either architectural design or systematic post-hoc alignment enables GNNs to serve as robust tools for hypothesis testing and clinical assessment.

The adoption of Commercial off-the-shelf (COTS) components has become a dominant paradigm in modern system design due to their reduced development cost, faster time-to-market, and widespread availability. However, the reliance on globally distributed and untrusted supply chains introduces significant security risks, particularly the possibility of malicious hardware modifications such as Trojans, embedded during design or fabrication. In such settings, traditional methods that depend on golden models, full design visibility, or trusted fabrication are no longer sufficient, creating the need for new security assurance approaches under a zero-trust model. This proposed research addresses security challenges in COTS microprocessors through two complementary solutions: runtime resilience and pre-deployment trust verification. First, a multi-variant-execution-based framework is developed that leverages functionally equivalent program variants to induce diverse microarchitectural execution patterns. By comparing intermediate outputs across variants, the framework enables runtime detection and tolerance of Trojan induced payload effects without requiring hardware redundancy or architectural modifications. To enhance the effectiveness of variant generation, a reinforcement learning assisted framework is introduced, in which the reward function is defined by security objectives rather than traditional performance optimization, enabling the generation of variants that are more robust against repeated Trojan activation. Second, to enable black-box trust verification prior to deployment, this work presents a framework that can efficiently test the presence of hardware Trojans by identifying microarchitectural rare events and transferring activation knowledge from existing processor designs to trigger highly susceptible internal nodes. By leveraging ISA-level knowledge, open-source RTL references, and LLM-guided test generation, the framework achieves high trigger coverage without requiring access to proprietary designs or golden references. Building on these two scenarios, a future research direction is outlined for evolving trust in COTS hardware through continuous runtime observation, where multi-variant execution is extended with lightweight monitoring mechanisms that capture key microarchitectural events and execution traces. These observations are accumulated as hardware trust counters, enabling the system to progressively establish confidence in the underlying hardware by verifying consistent behavior across diverse execution patterns over time. Together, these directions establish a foundation for analyzing and mitigating security risks across zero-trust COTS supply chains.

Android is the most widely adopted mobile operating system, supporting billions of devices and driven by a robust app ecosystem.  Its permission-based security model aims to enforce the Principle of Least Privilege (PoLP), restricting apps to only the permissions it needs.  However, many apps still request excessive permissions, increasing the risk of data leakage and malicious exploitation. Previous research on overprivileged permission has become ineffective due to outdated methods and increasing technical complexity.  The introduction of runtime permissions and scoped storage has made some of the traditional analysis techniques obsolete.  Additionally, developers often are not transparent in explaining the usage of app permissions on the Play Store, misleading users unknowingly and unwillingly granting unnecessary permissions. This combination of overprivilege and poor transparency poses significant security threats to Android users.  Recently, the rise of local large language models (LLMs) has shown promise in various security fields. The main focus of this study is to analyze whether an app is overpriviledged based on app description provided on the Play Store using Local LLM. Finally, we conduct a manual evaluation to validate the LLM’s findings, comparing its results against human-verified response.

Electronic Health Records (EHRs) provide rich longitudinal patient information, but their high dimensionality, sparsity, heterogeneity, and temporal complexity make robust representation learning difficult. This dissertation studies how to improve patient and medical concept representation learning in EHRs and consequently enhance healthcare predictive tasks by integrating domain knowledge, knowledge graphs, large language models (LLMs), and contrastive learning. First, it introduces an ontology-aware temporal contrastive framework for survival analysis that learns discriminative patient representations from censored and observed trajectories by modeling temporal distinctiveness in longitudinal EHR data. Second, it proposes a multi-ontology representation learning framework that jointly propagates knowledge within and across diagnosis, medication, and procedure ontologies, enabling richer medical concept embeddings, especially under limited data and for rare conditions. Third, it develops an LLM-enriched, text-attributed medical knowledge graph framework that combines EHR-derived statistical evidence with type-constrained LLM reasoning to infer semantic relations, generate contextual node and edge descriptions, and co-learn concept embeddings through joint language-model and graph-neural-network training. Together, these studies advance a unified view of EHR representation learning in which structured medical knowledge, textual semantics, and temporal patient trajectories are jointly leveraged to build more accurate, interpretable, and robust healthcare prediction models.

Individuals with Autism Spectrum Disorder (ASD) frequently experience elevated stress and are at higher risk for mood disorders such as anxiety and depression. Sensory over-responsivity, social challenges, and difficulties with emotional recognition and regulation contribute to such heightened stress. This study presents a proof-of-concept system that detects and mitigates stress through interactions with a virtual pet. Designed for young adults with high-functioning autism, and potentially useful for people beyond that group, the system monitors simulated heart rate, skin resistance, body temperature, and environmental sound and light levels. Upon detection of stress or potential triggers, the system alerts the user and offers stress-reduction activities via a virtual pet, including guided deep-breathing exercises and interactive engagement with the virtual companion. Through combining real-time stress detection with interactive interventions on a single platform, the system aims to help autistic individuals recognize and manage stress more effectively.

Facial recognition systems increasingly rely on machine learning services, yet they remain vulnerable to cyber-attacks. While traditional adversarial attacks target input data, an underexplored threat comes from weight manipulation attacks, which directly modify model parameters and can compromise deployed systems in cyber-physical settings. This paper investigates defenses against Weight Surgery, a weight manipulation attack that modifies the final linear layer of neural networks to merge or shatter classes without requiring access to training data. We propose a computationally lightweight defense capable of detecting sample pairs affected by Weight Surgery at low false-positive rates. The defense is designed to operate in realistic deployment scenarios, selecting its sensitivity parameter 𝛾 using only benign samples to meet a target false-positive rate. Evaluation on 1000 independently attacked models demonstrates that our method achieves over 95% recall at a target false-positive rate of 0.001. Performance remains strong even under stricter conditions: at FPR = 0.0001, recall is 92.5%, and at 𝛾=0.98, FPR drops to 0.00001 while maintaining 88.9% recall. These results highlight the robustness and practicality of the defense, offering an effective safeguard for neural networks against model-targeted attacks.

Microelectronics supply chains increasingly rely on globally distributed design, fabrication, integration, and deployment processes, making traditional assumptions of trusted hardware inadequate. Security in this setting can be understood through a zero-trust microelectronics supply-chain model, in which neither manufacturing partners nor procured hardware platforms are assumed trustworthy by default. Two complementary threat scenarios are considered in the proposed research. In the first scenario, custom Integrated Circuits (ICs) fabricated through potentially untrusted foundries are examined, where design-for-security protections intended to prevent piracy, overproduction, and intellectual-property theft can themselves become vulnerable to attacks. In this scenario, hardware Trojan-assisted meta-attacks are used to show that such protections can be systematically identified and subverted by fabrication-stage adversaries. In the second scenario, commercial off-the-shelf ICs are considered from the perspective of end users and procurers, where internal design visibility is unavailable and hardware trustworthiness cannot be directly verified. For this setting, runtime-oriented protection mechanisms are developed to safeguard sensitive computation against malicious hardware behavior and side-channel leakage. Building on these two scenarios, a future research direction is outlined for side-channel-driven vulnerability discovery in off-the-shelf devices, motivated by the need to evaluate and test such platforms prior to deployment when no design information is available. The proposed direction explores gray-box security evaluation using power and electromagnetic side-channel analysis to identify anomalous behaviors and potential vulnerabilities in opaque hardware platforms. Together, these directions establish a foundation for analyzing and mitigating security risks across zero-trust microelectronics supply chains.