Search Results for author:
Found 5 papers, 5 papers with code
SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning
Large Language Models (LLMs) have highlighted the necessity of effective unlearning mechanisms to comply with data regulations and ethical AI practices.
From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers
Building upon this insight, in this work, we propose a novel density loss that encourages higher activation sparsity (equivalently, lower activation density) in the pre-trained models.
Uncovering the Hidden Cost of Model Compression
This empirical investigation underscores the need for a nuanced understanding beyond mere accuracy in sparse and quantized settings, thereby paving the way for further exploration in Visual Prompting techniques tailored for sparse and quantized models.
Robust Graph Neural Networks using Weighted Graph Laplacian
For demonstration, the experiments are conducted with Graph convolutional neural network(GCNN) architecture, however, the proposed framework is easily amenable to any existing GNN architecture.
APP: Anytime Progressive Pruning
With the latest advances in deep learning, there has been a lot of focus on the online learning paradigm due to its relevance in practical settings.
