BumbleBee: Dynamic KV-Cache Streaming Submodular Summarization for Infinite-Context Transformers
Lilly Kumari, Shengjie Wang, Tianyi Zhou, Nikhil Sarda, Anthony Rowe, Jeff Bilmes
Proceedings of First Conference on Language Modeling (COLM), 2024
Resume
Get my resume in PDF format here.
Education
- Ph.D Candidate in University of Washington, 2025 (expected)
- Department of Electrical and Computer Engineering Department
- Machine Learning, Optimization, and Data Interpretation
- Advisor: Professor Jeff Bilmes
- Bachelors in Electrical Engineering, Indian Institute of Technology, Roorkee, 2016
Work experience
University of Washington Seattle (Sept 18 - Present) - Graduate Research Assistant
Adobe Systems, Bangalore (Jun 16 - Jul 18) : Member of Technical Staff
- Research group in Intelligent Services team
Internship experience
- Google Labs (Jan 24 - Aug 24)) : Student Researcher
- Proposed a novel submodular optimization framework called BumbleBee for efficient long-context KV cache summarization in LLMs and achieved state-of-the-art results on various long-context NLP tasks
- Google Labs (Jun 23 - Sep 23) : Research Intern
- Proposed kNN-based and Submodular Information Measures based subset selection methods to selectively augment the input context of LLMs for long-range grounded dialog modeling.
- Google Research (Jun 22 - Sep 22) : Research Intern
- Advisors: Srikumar Ramalingam, Ayan Chakrabarti
- Proposed sample re-weighting and replay algorithms for faster convergence of neural networks and a novel distillation-based loss objective to perform efficient importance sampling using lightweight models.
- Adobe Research (May 15 - July 15) : Research Intern
- Advisors: Ritwik Sinha, Atanu Sinha
- Developed an algorithm for audience segmentation using different machine learning techniques and predicting the segment membership of a new reader for effective personalization.
Publications
An End-to-End Submodular Framework for Data-Efficient In-Context Learning
Lilly Kumari, Shengjie Wang, Arnav Das, Tianyi Zhou, Jeff Bilmes
Proceedings of Findings of the Association for Computational Linguistics: NAACL, 2024COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Learning
Arnav Das, Gantavya Bhatt, Lilly Kumari, Sahil Verma, Jeff Bilmes
ICML Workshop on Data-Centric Machine Learning Research, 2024High resolution point clouds from mmwave radar
Akarsh Prabhakara, Tao Jin, Arnav Das, Gantavya Bhatt, Lilly Kumari, Elahe Soltanaghai, Jeff Bilmes, Swarun Kumar, Anthony Rowe
Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2023Retrieval Augmented Generation for Dialog Modeling
Lilly Kumari, Usama Shafqat, Nikhil Sarda
NeurIPS Workshop on Efficient Natural Language and Speech Processing, 2023Retrospective Adversarial Replay for Continual Learning
Lilly Kumari, Shengjie Wang, Tianyi Zhou, Jeff Bilmes
Proceedings of Advances in neural information processing systems (NeurIPS), 2022Submodular Span with Applications to Conditional Data Summarization
Lilly Kumari, Jeff Bilmes
Proceedings of the AAAI Conference on Artificial Intelligence, 2021Botcha: detecting malicious non-human traffic in the wild
Sunny Dhamnani, Ritwik Sinha, Vishwa Vinay, Lilly Kumari, Margarita Savova
RecSys Workshop on Online Misinformation- and Harm-Aware Recommender Systems, 2020Audience Prism: Segmentation and Early Classification of Visitors Based on Reading Interests
Lilly Kumari, Sunny Dhamnani, Akshat Bhatnagar, Atanu R Sinha, Ritwik Sinha
Proceedings of the 3rd IKDD Conference on Data Science, 2016
Teaching
TinyML (EEP 595) , University of Washington, Apr 2024 - Jun 2024
Introduction to Statistical Learning (EE 511) , University of Washington, Jan 2024 - Mar 2024
Deep Learning (EEP 596) , University of Washington, Apr 2021 - Jun 2021
Advanced Introduction to Machine Learning (EEP 596) , University of Washington, Jan 2021 - Mar 2021
Patents
Classification of website sessions using one-class labeling techniques, US-PTO 10785318
Detecting robotic internet activity across domains utilizing one-class and domain adaptation machine-learning models, US-PTO 15982393
Makeup identification using deep learning, US-PTO 10755447
Services
- Program Committee (Reviewer):
- SubsetML@ICML 2021
- ICML 2022-2023
- ICLR 2023
- NeurIPS 2022-2024
- ENLSP@NeurIPS 2023-2024
- ARR 2023-Present
Honors and Awards
- Institute Silver Medal - Department Rank 1 at IIT, Roorkee 2016
- NeurIPS Scholar Award 2022
- NVIDIA Academic Hardware Grant Award, 2022
- Azure Compute Grant Award for $40k, 2020
Skills
- Python, C++, R, PyTorch, Tensorflow