Join our discord here !

Aims and Focus

Training high-performing large language models (LLMs) from scratch is a notoriously expensive and difficult task, costing hundreds of millions of dollars in compute alone. These pretrained LLMs, however, can cheaply and easily be adapted to new tasks via fine-tuning, leading to a proliferation of models that suit specific use cases. Recent work has shown that specialized fine-tuned models can be rapidly merged to combine capabilities and generalize to new skills.

Get started

The competition allows any current model that follows the general conditions (e.g., existed when the competition was announced and is up to 8Gb) see Rules for explicit conditions.

A starter kit with an end-to-end submission flow can be found here:
https://github.com/llm-merging/LLM-Merging

For more details, please check the Challenge, Rules, and Starter Kit tab.

Important Dates

Submission Open Early June, 2024
Code Submission Deadline September 13th, 2024, New! October 11th, 2024
Report Submission Deadline November 1st, 2024
Winners Notification Mid-November, 2024
Competition Presnetation December, 2024

Sponsors