Join our discord here !
Aims and Focus
Training high-performing large language models (LLMs) from scratch is a notoriously expensive and difficult task, costing hundreds of millions of dollars in compute alone. These pretrained LLMs, however, can cheaply and easily be adapted to new tasks via fine-tuning, leading to a proliferation of models that suit specific use cases. Recent work has shown that specialized fine-tuned models can be rapidly merged to combine capabilities and generalize to new skills.
Get started
The competition allows any current model that follows the general conditions (e.g., existed when the competition was announced and is up to 8Gb) see Rules for explicit conditions.
A starter kit with an end-to-end submission flow can be found here:
https://github.com/llm-merging/LLM-Merging
- Please submit a report describing your merging method to our OpenReview LMC 2024 page. Please follow the standard NeurIPS format template. There are no strict restrictions or limitations for the report, but we suggest that the page limit not exceed 4 pages. All submitted reports will be publicly accessible on our website.
For more details, please check the Challenge, Rules, and Starter Kit tab.
Important Dates
| Submission Open | |
| Code Submission Deadline | |
| Report Submission Deadline | November 1st, 2024 |
| Winners Notification | Mid-November, 2024 |
| Competition Presnetation | December, 2024 |
- Currently the deadline is October 11th, 2024. Please check our website and the announcements in the Discord channel for updates.*
Sponsors
 |
 |
 |