See you in Vancouver!
See you in West Meeting Room 210, Sun 15 Dec noon EST — 2:50 p.m. EST. Check our NeurIPS 2024 schedule!
Our accepted technical reports are at OpenReview page of LLM Merging Compeition NeurIPS 2024
Announcing Competition Winners
Performance Track
| Placement | Winner |
|---|---|
| 1st Place 🌟 | A Model Merging Method Qiang Gao, Jisheng fang, Hao Mo |
| 2nd Place ⭐ | Simple Llama Merge: What Kind of LLM Do We Need? Yinuo Zhang |
| 3rd Place 💫 | LLM Merging Competition Technical Report: Efficient Model Merging with Strategic Model Selection, Merging, and Hyperparameter Optimization Zixiang Di, Yaoming Yang, Mei Jiang, Bingdong Li, Hong Qian, Aimin Zhou |
Efficiency Track
| Winner 🚀 |
|---|
| LLM Merging Competition Technical Report for NeurIPS 2024: Efficiently Building Large Language Models through Merging Yizhen Zhang, Yang Ding, Jie Wu, Yujiu Yang |
Most Creative Paper Write-up Track
| Winner 🎆 |
|---|
| Model Merging using Geometric Median of Task Vectors Siddharth Gupta, Aakash Gupta |
The codebase of Winners
Please download via the following link: Google drive
Aims and Focus
Join our discord here !
Training high-performing large language models (LLMs) from scratch is a notoriously expensive and difficult task, costing hundreds of millions of dollars in compute alone. These pretrained LLMs, however, can cheaply and easily be adapted to new tasks via fine-tuning, leading to a proliferation of models that suit specific use cases. Recent work has shown that specialized fine-tuned models can be rapidly merged to combine capabilities and generalize to new skills.
Get started
The competition allows any current model that follows the general conditions (e.g., existed when the competition was announced and is up to 8Gb) see Rules for explicit conditions.
A starter kit with an end-to-end submission flow can be found here:
https://github.com/llm-merging/LLM-Merging
- Please submit a report describing your merging method to our OpenReview LMC 2024 page. Please follow the standard NeurIPS format template. There are no strict restrictions or limitations for the report, but we suggest that the page limit not exceed 4 pages. All submitted reports will be publicly accessible on our website.
For more details, please check the Challenge, Rules, and Starter Kit tab.
Important Dates
| Submission Open | |
| Code Submission Deadline | |
| Report Submission Deadline | |
| Winners Notification | |
| Competition Presentation | |
- Currently the deadline is October 11th, 2024. Please check our website and the announcements in the Discord channel for updates.*
Sponsors
 |
 |
 |