Task Relationship Modeling in Lifelong Multitask Learning

Multitask Learning with task relationship modeling is a learning framework which identifies and shares training information among multiple related tasks to improve the generalization error of each task. The utilization of task relationships in static multitask learning framework, where all the tasks are known beforehand and all the data is present before the training, has been studied in considerable detail for past several years. However, in the case of lifelong multitask learning, where the tasks arrive in an online fashion and information about all the tasks is not known beforehand, modeling the task relationship is very challenging. The main contribution of this thesis is to propose a framework for modeling task relationships in lifelong multitask learning. The task relationship models in lifelong multitask learning needs to be flexible and dynamic such that it can be easily updated with each new task coming in. Also, a new task needs to readily learn its position in the existing task network using the task relationship model. Traditionally, task relationships are represented using fixed sized matrices, which describe the task network. These matrices are not capable of dynamically changing with each incoming task, and can be rather expensive to update. Here, we propose learning functions to represent the relationships between tasks. Learning functions is faster and computationally less expensive for depicting the task relationship models. The functions partition the task space such that the similar tasks remain in the same region and enforce similar tasks to depend on similar features. Learning both the task parameters and relationships is done in a supervised manner. In this thesis, we show that the algorithm we developed provides significantly better accuracy and is much faster than the state of the art lifelong learning algorithm. For some dataset, our algorithm provides a better accuracy than even the static multitask learning method.