LLM_Crystal_CIF — LoRA adapters for CIF generation

LoRA adapters fine-tuning Qwen2.5-7B-Instruct to emit a full CIF crystal structure from a prompt of reduced composition + target space-group number. Part of a controlled composition-sweep study (MP-20 : MPTS-52 training ratio at fixed volume/steps).

Trained with Unsloth, LoRA r=32 / α=64, lr 1e-4, 24,000 unique training crystals, pinned 4,500 steps (no early stopping), 16-bit. Adapter base: unsloth/Qwen2.5-7B-Instruct (loads fine on stock Qwen/Qwen2.5-7B-Instruct).

Adapters (by training composition, MP-20% : MPTS-52%)

Subfolder	MP-20 %	Best-of-10 match (full 8,096 MPTS-52 test)
comp_mp20_00	0 (pure MPTS-52 baseline)	30.1%
comp_mp20_25	25	30.4%
comp_mp20_50	50	29.5%
comp_mp20_75	75	28.0%
comp_mp20_100	100 (pure MP-20)	26.6%

Finding: at matched volume + steps, match declines monotonically as the MP-20 fraction rises — the gain reported for "combined" data was volume, not symmetry composition.

Each subfolder contains: model/ (LoRA adapter + adapter_config.json), tokenizer/, config.json, and training_stats.json.

GRPO adapters

grpo_crystext_reward/step2150 is a GRPO (RL) continuation of the rank_r32_s3407 SFT adapter, using the reward function from CrysText copied verbatim (+0.5 parses · +0.5 valid · +0.5 formula · +0.25/+0.25/+1.0 StructureMatcher at high/mid/low tolerance · −2 on failure), with their hyperparameters and batch geometry (lr 1e-6, 12 completions per step = 2 crystals × 6 generations). See grpo_crystext_reward/config.json for the full record.

Checkpoint	Best-of-10 (full 8,096 MPTS-52 test)	Strict-RMS (med Å)
rank_r32_s3407 (SFT start point)	29.9%	0.050
GRPO step 150	29.8%	0.053
GRPO step 1150	24.3%	0.108
grpo_crystext_reward/step2150 (uploaded)	not evaluated	—

Finding: GRPO did not improve on SFT here. The CrysText reward preserves SFT performance early (29.8% vs 29.9%) where this project's own reward designs cost ~2 pp, but the run degrades with more steps. The uploaded adapter is the last surviving checkpoint of a degrading run — it is published for provenance, not as a recommended model. For best quality use rank_r32_s3407 or comp_mp20_25. The two evaluated checkpoints (steps 150 and 1150) no longer exist as weights; their full predictions and validation panels are in the GitHub repo under results/grpo_crystext_reward/.

Prompt note. GRPO adapters were trained with a different system message than the SFT adapters — "You are an expert in materials science and crystallography. Return only one complete CIF file and nothing else." Prompt grpo_crystext_reward/* with that message; use the shorter one in the example below for every other adapter.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "shehrozashoaib/LLM_Crystal_CIF", subfolder="comp_mp20_50/model")
tok = AutoTokenizer.from_pretrained("shehrozashoaib/LLM_Crystal_CIF", subfolder="comp_mp20_50/tokenizer")

messages = [
    {"role": "system", "content": "You are an expert in materials science and crystallography."},
    {"role": "user", "content": "Generate CIF for the given material description\n\n"
                                 "Material composition is FeCuS2. It has a space group number 122."},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device),
                     max_new_tokens=3072, do_sample=True, temperature=0.6, top_p=0.9)
print(tok.decode(out[0], skip_special_tokens=True))

Code, datasets, and full results: see the companion GitHub repo shehrozashoaib/LLM_Crystal_CIF.