Enhancing Parameter-Efficient Fine-Tuning of Large Language Models with Alignment Adapters and LoRA

Large Language Models (LLMs) have become integral to natural language processing, involving initial broad pretraining on generic data followed by fine-tuning for specific tasks or domains. While advancements in Parameter Efficient Fine-Tuning (PEFT) techniques have made strides in reducing resource demands for LLM fine-tuning, they possess individual constraints. This project addresses the challenges posed by PEFT in the context of transformers architecture for sequence-to-sequence tasks, by integrating two pivotal techniques: Low-Rank Adaptation (LoRA) for computational efficiency and adaptive layers for task-specific customization. To overcome the limitations of LoRA, we introduce a simple yet effective hyper alignment adapter, that leverages a hypernetwork to generate decoder inputs based on encoder outputs, thereby serving as a crucial bridge to improve alignment between the encoder and the decoder. This fusion strikes a balance between the fine-tuning complexity and task performance, mitigating the individual drawbacks while improving the encoder-decoder alignment. As a result, we achieve more precise and contextually relevant sequence generation. The proposed solution improves the overall efficiency and effectiveness of LLMs in sequence-to-sequence tasks, leading to better alignment and more accurate output generation.