daryaZare/iris-olmo-2-1b-mixed-k10-ep6

IRIS OLMo 2 1B Mixed K10

LoRA adapters for allenai/OLMo-2-0425-1B-Instruct, fine-tuned for relevance scoring with label-distribution distillation on the mixed IRIS and multihop training regime.

Training

• Learning rate: 5e-5
• Effective batch size: 16
• Seed: 42
• Training duration: 6 epochs
• Checkpoint interval: 0.5 epoch
• LoRA rank: 16
• LoRA alpha: 32

Checkpoints

Epoch	Checkpoint
0.5	checkpoint-142
1.0	checkpoint-284
1.5	checkpoint-426
2.0	checkpoint-568
2.5	checkpoint-710
3.0	checkpoint-852
3.5	checkpoint-994
4.0	checkpoint-1136
4.5	checkpoint-1278
5.0	checkpoint-1420
5.5	checkpoint-1562
6.0	checkpoint-1704

The reported checkpoint in the IRIS evaluation is checkpoint-852 (epoch 3).

Loading

Each checkpoint is a PEFT LoRA adapter and requires the base model.

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_id = "allenai/OLMo-2-0425-1B-Instruct"
adapter_id = "daryaZare/iris-olmo-2-1b-mixed-k10-ep6"
checkpoint = "checkpoint-852"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(base_model_id)
model = PeftModel.from_pretrained(
    base_model,
    adapter_id,
    subfolder=checkpoint,
)