Defense Notices

Many important machine learning and data mining algorithms rely on a measure to provide a notion of distance or dissimilarity. Naive metrics such as the Euclidean distance are incapable of leveraging task-specific information, and consider all features as equal. A learned distance metric can become much more effective by honing in on structure specific to a task. Additionally, it is often extremely desirable for a metric to be sparse, as this vastly increases the ability to interpret the distance metric. In this dissertation, we explore several current problems in distance metric learning and put forth solutions which make use of structured sparsity.

The first contribution of this dissertation begins with a classic approach in distance metric learning and address a scenario where distance metric learning is typically inapplicable, i.e., the case of learning on heterogeneous data in a high-dimensional input space. We construct a projection-free distance metric learning algorithm which utilizes structured sparse updates and successfully demonstrate its application to learn a metric with over a billion parameters.

The second contribution of this dissertation focuses on an intriguing regression-based approach to distance metric learning. Under this regression approach there are two sets of parameters to learn; those which parameterize the metric, and those defining the so-called ``virtual points''. We begin with an exploration of the metric parameterization and develop a structured sparse approach to robustify the metric to noisy, corrupted, or irrelevant data. We then focus on the virtual points and develop a new method for learning the metric and constraints together in a simultaneous manner. It is demonstrate through empirical means that our approach results in a distance metric which is more effective than the current state of-the-art.

Machine learning algorithms have recently become ingrained in an incredibly diverse amount of technology. The focus of this dissertation is to develop more effective techniques to learn a distance metric. We believe that this work has the potential for broad-reaching impacts, as learning a more effective metric typically results in more accurate metric-based machine learning algorithms.

Data mining is a computing process of finding meaningful patterns in large sets of data. These patterns are then analyzed and used to make predictions for the future. One form of data mining is to extract rules from data sets. There are various rule induction algorithms, such as LEM1 (Learning from Examples Module Version 1), LEM2 (Learning from Examples Module Version 2) and MLEM2(Modified Learning from Examples Module Version 2). Most of the rule induction algorithms require the input data with only discretized attributes. If the input data contains numerical attributes, we need to convert them into discrete values (intervals) before performing rule induction, this process is called discretization. In this project, we discuss an implementation of LEM2 which generates the rules from data with numerical and symbolic attributes. The accuracy of the rules generated by LEM2 is measured by computing the error rate by a program called rule checker using ten-fold cross-validation and holdout methods. ​

Dynamic Binary Translators(DBTs) have a variety of uses, like instrumentation, profiling, security, portability, etc. In order for the desired application to run with these enhanced additional features(not originally part of its design), it is to be run under the control of Dynamic Binary Translator. The application can be thought of as the guest application, to be run with in a controlled environment of the translator, which would be the host application. That way, the intended application execution flow can be enforced by the translator, thereby inducing the desired behavior in the application on the host platform(combination of Operating System and Hardware). Depending on the implementation of the translator(host application), the guest application can either have code compiled for the host platform, or a different platform. It would be the responsibility of the translator to make appropriate code/binary translation of the guest application code, to be run on the host platform.

However, there will be a run-time/execution-time overhead in the translator, when performing the additional tasks to run the guest application in a controlled fashion. This run-time overhead has been limiting the usage of DBT's on a large scale, where response times can be critical. There is often a trade-off between the benefits of using a DBT against the overall application response time. So, there is a need to research/explore ways of faster application execution through DBT's(given their large code-base).

With the evolution of the multi-core and GPU hardware architectures, paralleization of software can be employed through multiple threads, which can concurrently run parts of code and potentially doing more work at the same time. The proper design of parallel applications or parallelizing parts of existing code, can lead to faster application run-time's, by taking advantage of the hardware architecture support to parallel programs.

We explore the possibility of improving the performance of a DBT named DynamoRIO. The basic idea is to improve its performance by speeding-up the process of guest code translation, through multiple threads translating multiple pieces of code concurrently. In an ideal case, all the required code blocks for application execution would be available ahead of time(eager translation), without any wait/overhead at run-time, and also giving it the enhanced features through the DBT. For efficient run-time eager translation there is also a need for heuristics, to better predict the next likely code block to be executed. That could potentially bring down the less productive code translations at run-time. The goal is to get application speed-up through eager translation, coupled with block prediction heuristics, leading to an execution time close to that of native run.

With the advancement of arbitrary waveform generation techniques, new radar transmission modes can be designed via precise control of the waveform's time-domain signal structure. The finer degree of emission control for a waveform (or multiple waveforms via a digital array) presents an opportunity to reduce ambiguities in the estimation of parameters within the radar backscatter. While this freedom opens the door to new emission capabilities, one must still consider the practical attributes for radar waveform design. Constraints such as constant amplitude (to maintain sufficient power efficiency) and continuous phase (for spectral containment) are still considered prerequisites for high-powered radar waveforms. These criteria are also applicable to the design of multiple waveforms emitted from an antenna array in a multiple-input multiple-output (MIMO) mode.

In this work, two spatially-diverse radar emission design methods are introduced that provide constant amplitude, spectrally-contained waveforms. The first design method, denoted as spatial modulation, designs the radar waveforms via a polyphase-coded frequency-modulated (PCFM) framework to steer the coherent mainbeam of the emission within a pulse. The second design method is an iterative scheme to generate waveforms that achieve a desired wideband and/or widebeam radar emission. However, a wideband and widebeam emission can place a portion of the emitted energy into what is known as the `invisible' space of the array, which is related to the storage of reactive power that can damage a radar transmitter. The proposed design method purposefully avoids this space and a quantity denoted as the Fractional Reactive Power (FRP) is defined to assess the quality of the result.

The design of FM waveforms via traditional gradient-based optimization methods is also considered. A waveform model is proposed that is a generalization of the PCFM implementation, denoted as coded-FM (CFM), which defines the phase of the waveform via a summation of weighted, predefined basis functions. Therefore, gradient-based methods can be used to minimize a given cost function with respect to a finite set of optimizable parameters. A generalized integrated sidelobe metric is used as the optimization cost function to minimize the correlation range sidelobes of the radar waveform.

In Data mining, Decision Tree is a type of classification model which uses a tree-like data structure to organize the data to obtain meaningful information. We may use Decision Tree for important predictive analysis in data mining. 

In this project, we compare two decision tree generating algorithms CART and the modified ID3 algorithm using different datasets with discrete and continuous numerical values. A new approach to handle the continuous numerical values is implemented in this project since the basic ID3 algorithm is inefficient in handling the continuous numerical values. In the modified ID3 algorithm, we discretize the continuous numerical values by creating cut-points. The decision trees generated by the modified algorithm contain fewer nodes and branches compared to basic ID3. 

The results from the experiments indicate that there is statistically insignificant difference between CART and modified ID3 in terms of accuracy on test data. On the other hand, the size of the decision tree generated by CART is smaller than the decision tree generated by modified ID3. 

Recommendation systems are becoming more and more important with increasing popularity of e-commerce platforms. An ideal recommendation system recommends preferred items to the user. In this project, an algorithm named item-item collaborative filtering is implemented as premise. The recommendations are smarter by going through movies similar to the movies of different ratings by the user, calculating predictions and recommending those movies which have high predictions. The primary goal of the proposed recommendation algorithm is to include user’s preference and to include lesser known items in recommendations. The proposed recommendation system was evaluated on basis of Mean Absolute Error(MAE) and Root Mean Square Error(RMSE) against 1 Million movie rating involving 6040 users and 3900 movies. The implementation is made as a web-application to simulate the real-time experience for the user.  

In the current era of big data, huge amount of data can be easily collected, but the unprocessed data is not useful on its own. It can be useful only when we are able to find interesting patterns or hidden knowledge. The algorithm to find interesting patterns is known as Rule Induction Algorithm. Rule induction is a special area of data mining and machine learning in which formal rules are extracted from a dataset. The extracted rules may represent some general or local (isolated) patterns related to the data.
In this report, we will focus on the IRIM (Interesting Rule Inducing Module) which induces strong interesting rules that covers most of the concept. Usually, the rules induced by IRIM provides interesting and surprising insight to the expert in the domain area.
The IRIM algorithm was implemented using Python and pySpark library, which is specially customize for data mining. Further, the IRIM algorithm was extended to handle the different types of missing data. Then at the end the performance of the IRIM algorithm with and without missing data feature was analyzed. As an example, interesting rules induced from IRIS dataset are shown.

Vision-based autonomous navigation of UAVs in real-time is a very challenging problem, which requires obstacle detection, tracking, and depth estimation. Although the problems of obstacle detection and tracking along with 3D reconstruction have been extensively studied in computer vision field, it is still a big challenge for real applications like UAV navigation. The thesis intends to address these issues in terms of robustness and efficiency. First, a vision-based fast and robust obstacle detection and tracking approach is proposed by integrating a salient object detection strategy within a kernelized correlation filter (KCF) framework. To increase its performance, an adaptive obstacle detection technique is proposed to refine the location and boundary of the object when the confidence value of the tracker drops below a predefined threshold. In addition, a reliable post-processing technique is implemented for an accurate obstacle localization. Second, we propose an efficient approach to detect the outliers present in noisy image pairs for the robust fundamental matrix estimation, which is a fundamental step for depth estimation in obstacle avoidance. Given a noisy stereo image pair obtained from the mounted stereo cameras and initial point correspondences between them, we propose to utilize reprojection residual error and 3-sigma principle together with robust statistic based Qn estimator (RES-Q) to efficiently detect the outliers and accurately estimate the fundamental matrix. The proposed approaches have been extensively evaluated through quantitative and qualitative evaluations on a number of challenging datasets. The experiments demonstrate that the proposed detection and tracking technique significantly outperforms the state-of-the-art methods in terms of tracking speed and accuracy, and the proposed RES-Q algorithm is found to be more robust than other classical outlier detection algorithms under both symmetric and asymmetric random noise assumptions.

The DC to AC power Converters (so-called Inverters) are widely used in industrial applications. The multilevel inverters are becoming increasingly popular in industrial apparatus aimed at medium to high power conversion applications.  In comparison to the conventional inverters, they feature superior characteristics such as lower total harmonic distortion (THD), higher efficiency, and lower switching voltage stress.  Nevertheless, the superior characteristics come at the price of a more complex topology with an increased number of power electronic switches. The increased number of power electronics switches results in more complicated control strategies for the inverter. Moreover, as the number of power electronic switches increases, the chances of fault occurrence of the switches increases, and thus the inverter’s reliability decreases. Due to the extreme monetary ramifications of the interruption of operation in commercial and industrial applications, high reliability for power inverters utilized in these sectors is critical.  As a result, developing simple control strategies for normal and fault-tolerant operation of multilevel inverters has always been an interesting topic for researchers in related areas.  The purpose of this dissertation is to develop new control and fault-tolerant strategies for the multilevel power inverter.  For the normal operation of the inverter, a new high switching frequency technique is developed.  The proposed method extends the utilization of the dc link voltage while minimizing the dv/dt of the switches. In the event of a fault, the line voltages of the faulty inverters are unbalanced and cannot be applied to the three phase loads. For the faulty condition of the inverter, three novel fault-tolerant techniques are developed. The proposed fault-tolerant strategies generate balanced line voltages without bypassing any healthy and operative inverter element, makes better use of the inverter capacity and generates higher output voltage. These strategies exploit the advantages of the Selective Harmonic Elimination (SHE) and Space Vector Modulation (SVM) methods in conjunction with a slightly modified Fundamental Phase Shift Compensation (FPSC) technique to generate balanced voltages and manipulate voltage harmonics at the same time.  The proposed strategies are applicable to several classes of multilevel inverters with three or more voltage levels.

Aiming to achieve the learning capabilities possessed by intelligent beings, especially human, researchers in machine learning field have the long-standing tradition of borrowing ideas from human learning, such as reinforcement learning, active learning, and curriculum learning.  Motivated by a philosophical theory called  "constructivism", in this work, we propose a new machine learning paradigm, constructivism learning.   The constructivism theory has had wide-ranging impact on various human learning theories about how human acquire knowledge.  To adapt this human learning theory to the context of machine learning, we first studied how to improve leaning performance by exploring inductive bias or prior knowledge from multiple learning tasks with multiple data sources, that is multi-task multi-view learning, both in offline and lifelong setting.  Then we formalized a Bayesian nonparametric approach using sequential Dirichlet Process Mixture Models to support constructivism learning.  To further exploit constructivism learning, we also developed a constructivism deep learning method utilizing Uniform Process Mixture Models.