I am fine tuning llama 3.2 8bit lora using chetgpt, but I am doubtful whether the direction is correct

[kbuwel]

I’m trying to fine-tune using things like the Harry Potter novels.
I performed fine tuning several times with llama-3-Korean-Bllossom-8B.
I wanted to create a creative llama model rather than an instructional one.
Let me show you an example.
I made one sample from Chapter 1: The Surviving Child to Chapter 2: In Front of the Vanishing Window.
The results of doing as gpt said were disastrous.
In order to divide the scenes into chunks, there were times when scenes were cut randomly and the scenes were messed around, saying they had to be stitched together.
So I cut one sample like that.
The jsonl file contains {“text”: “Chapter 1 The Child Who Survived\nThe Dursleys living at 4 Privet Drive were people who were very proud of being normal.
Everything.”}
I created a sample like this.
Here is the code:

-- coding: utf-8 --

“”"
train_auto.py

QLoRA continuation/style learning automation script

Goal:

• train.jsonl {“text”: “…”} Automatic data analysis
• Automatic learning amount determination based on data size/token/block
• Prevents the problem of small data lasting dozens of epochs
• Test creation of the same prompt before and after learning
• Automatically save before_sample.txt / after_sample.txt / train_report.txt
• Automatically generate verification prompts for short/long posts

Note:

• This script is for learning “writing style/sequence.”
• Not for learning chat/RP format.
  “”"

import gc
import json
import math
import random
import inspection
import time
from dataclasses import dataclass, asdict
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple

import torch
from datasets import Dataset
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
from transformers import (
AutoModelForCausalLM;
Auto Tokenizer,
BitsAndBytesConfig,
DataCollatorForLanguageModeling,
EarlyStoppingCallback,
GenerationConfig,
Trainer,
TrainerCallback,
TrainingArguments,
)

===========================================================

Default settings

===========================================================

MODEL_NAME = “Llama-3.1-8B”
DATA_PATH = “train.jsonl”
OUTPUT_DIR = “blossom-8b-qlora-story-auto”

MAX_SEQ_LENGTH = 1536
VALID_RATIO = 0.05
RANDOM_SEED = 42
MIN_CHARS = 40

Auto tuning on/off

AUTO_TUNE = True

Verification of generation before/after learning on/off

AUTO_GENERATION_TEST = True
GENERATION_TEST_DIR = “generation_check”
GENERATION_MAX_NEW_TOKENS = 120

If you want to specify your own verification prompt, put it here.

If left blank, it is automatically extracted from train.jsonl.

GENERATION_TEST_PROMPTS: List[str] =

Saving/evaluating too small data only takes time, so it is automatically turned off.

SAVE_TOTAL_LIMIT = 2

USE_BF16 = torch.cuda.is_available() and torch.cuda.is_bf16_supported()

===========================================================

data structure

===========================================================

@dataclass
classDataStats:
file_size_bytes: int
file_size_label: str
samples: int
total_chars: int
total_tokens: int
avg_chars: float
avg_tokens: float
min_chars: int
max_chars: int
min_tokens: int
max_tokens: int
estimated_blocks: int
skipped_short: int
skipped_duplicate: int
skipped_json: int

@dataclass
class TrainPlan:
profile: str
mode: str
train_blocks: int
eval_blocks: int
max_steps: int
num_train_epochs: int
gradient_accumulation_steps: int
optimize_updates: int
learning_rate: float
warmup_ratio: float
eval_strategy: str
save_strategy: str
eval_steps: Optional[int]
save_steps: Optional[int]
load_best_model_at_end: bool
early_stopping: bool

@dataclass
class PromptItem:
prompt: str
position: str
source_chars: int
line_no: int

===========================================================

utility

===========================================================

def set_seed(seed: int) → None:
random.seed(seed)
torch.manual_seed(seed)
if torch.cuda.is_available():
torch.cuda.manual_seed_all(seed)

def file_size_label(path: str) → Tuple[int, str]:
size = Path(path).stat().st_size
if size < 1024:
return size, f"{size} B"
if size < 1024 * 1024:
return size, f"{size / 1024:.1f} KB"
return size, f"{size / (1024 * 1024):.2f} MB"

def ceil_div(a: int, b: int) → int:
return (a + b - 1) // b

def normalize_text(text: str) → str:
return " “.join(str(text).replace(”\r\n", “\n”).replace(“\r”, “\n”).split()).strip()

===========================================================

Data load/analysis

===========================================================

def load_jsonl(path: str, tokenizer: AutoTokenizer) → Tuple[Dataset, DataStats, List[Dict[str, Any]]]:
rows: List[Dict[str, Any]] =
raw_items: List[Dict[str, Any]] =
seen = set()

skipped_short = 0
skipped_duplicate = 0
skipped_json = 0

total_chars = 0
total_tokens = 0
min_chars: Optional[int] = None
max_chars = 0
min_tokens: Optional[int] = None
max_tokens = 0

with open(path, “r”, encoding=“utf-8-sig”) as f:
for line_no, line in enumerate(f, start=1):
line = line.strip()
if not line:
continue

try:
obj = json.loads(line)
except json.JSONDecodeError:
skipped_json += 1
print(f"[skip] JSON parsing failed: line {line_no}")
continue

text = str(obj.get(“text”, “”)).replace(“\r\n”, “\n”).replace(“\r”, “\n”).strip()
text = “\n”.join(x.rstrip() for x in text.splitlines()).strip()

if len(text) < MIN_CHARS:
skipped_short += 1
continue

        if text in seen:
            skipped_duplicate += 1
            continue

        token_count = len(tokenizer.encode(text, add_special_tokens=False))
        char_count = len(text)

        seen.add(text)
        rows.append({"text": text, "chars": char_count, "tokens": token_count})
        raw_items.append({"text": text, "chars": char_count, "tokens": token_count, "line_no": line_no})

        total_chars += char_count
        total_tokens += token_count
        min_chars = char_count if min_chars is None else min(min_chars, char_count)
        max_chars = max(max_chars, char_count)
        min_tokens = token_count if min_tokens is None else min(min_tokens, token_count)
        max_tokens = max(max_tokens, token_count)

if not rows:
    raise ValueError(f"학습 가능한 text가 없습니다: {path}")

size_bytes, size_label = file_size_label(path)
stats = DataStats(
    file_size_bytes=size_bytes,
    file_size_label=size_label,
    samples=len(rows),
    total_chars=total_chars,
    total_tokens=total_tokens,
    avg_chars=total_chars / len(rows),
    avg_tokens=total_tokens / len(rows),
    min_chars=min_chars or 0,
    max_chars=max_chars,
    min_tokens=min_tokens or 0,
    max_tokens=max_tokens,
    estimated_blocks=max(1, ceil_div(total_tokens, MAX_SEQ_LENGTH)),
    skipped_short=skipped_short,
    skipped_duplicate=skipped_duplicate,
    skipped_json=skipped_json,
)

print_data_stats(stats)
return Dataset.from_list(rows), stats, raw_items

def print_data_stats(stats: DataStats) → None:
print(“=” * 60)
print(“[data stats]”)
print(f"파일 크기 : {stats.file_size_label}“)
print(f"샘플 수 : {stats.samples:,}”)
print(f"총 문자 수 : {stats.total_chars:,}“)
print(f"총 토큰 수 : {stats.total_tokens:,}”)
print(f"평균 문자 수 : {stats.avg_chars:,.1f}“)
print(f"평균 토큰 수 : {stats.avg_tokens:,.1f}”)
print(f"최소/최대 문자 수 : {stats.min_chars:,} / {stats.max_chars:,}“)
print(f"최소/최대 토큰 수 : {stats.min_tokens:,} / {stats.max_tokens:,}”)
print(f"예상 블록 : {stats.estimated_blocks:,}개 @ MAX_SEQ_LENGTH={MAX_SEQ_LENGTH}“)
print(f"제외 : 짧음={stats.skipped_short:,}, 중복={stats.skipped_duplicate:,}, JSON오류={stats.skipped_json:,}”)
print(“=” * 60)

============================================================

토큰화/패킹

============================================================

def tokenize_for_continuation(examples: Dict[str, List[str]], tokenizer: AutoTokenizer) → Dict[str, Any]:
texts = [text + tokenizer.eos_token for text in examples[“text”]]
return tokenizer(texts, add_special_tokens=False)

def pack_blocks(examples: Dict[str, List[List[int]]]) → Dict[str, List[List[int]]]:
“”"
MAX_SEQ_LENGTH 단위 packing.
마지막 remainder를 버리지 않는다.
KB급 데이터가 1536토큰 미만이라는 이유로 0블록이 되는 문제를 막는다.
“”"
result: Dict[str, List[List[int]]] = {}

for key in ("input_ids", "attention_mask"):
    concatenated: List[int] = []
    for item in examples[key]:
        concatenated.extend(item)

    if not concatenated:
        result[key] = []
        continue

    result[key] = [
        concatenated[i : i + MAX_SEQ_LENGTH]
        for i in range(0, len(concatenated), MAX_SEQ_LENGTH)
    ]

return result

============================================================

자동 학습 계획

============================================================

def choose_profile(stats: DataStats, train_blocks: int) → Tuple[str, int, int, float, float, int]:
“”"
반환:
profile, target_optimizer_updates, grad_accum, learning_rate, warmup_ratio, save_eval_steps

기준:
- 파일 크기 하나만 보지 않는다.
- 총 토큰 수와 packed block 수를 같이 본다.
- KB급 파일이 block 수 때문에 large로 튀지 않게 방지한다.
"""
size_mb = stats.file_size_bytes / (1024 * 1024)
tokens = stats.total_tokens
blocks = train_blocks

rules = [
    # profile, max_size_mb, max_tokens, max_blocks, updates, grad_accum, lr, warmup, save_eval
    ("tiny / KB급",      0.10,     12_000,      8,     8,   1, 1.0e-4, 0.00, 0),
    ("small / 소형",     0.50,     50_000,     32,    16,   1, 9.0e-5, 0.00, 0),
    ("medium / 중형",    2.00,    200_000,    128,    32,   2, 8.0e-5, 0.02, 50),
    ("large / MB급",    10.00,  1_000_000,    768,    64,   4, 7.0e-5, 0.03, 100),
    ("xlarge / 대형",   50.00,  5_000_000,  4_096,   128,   8, 6.0e-5, 0.03, 200),
]

for profile, max_size, max_tokens, max_blocks, updates, grad_accum, lr, warmup, save_eval in rules:
    if size_mb <= max_size and tokens <= max_tokens and blocks <= max_blocks:
        return profile, updates, grad_accum, lr, warmup, save_eval

return "huge / 초대형", 0, 8, 5.0e-5, 0.03, 500

def build_train_plan(stats: DataStats, train_blocks: int, eval_blocks: int) → TrainPlan:
train_blocks = max(1, int(train_blocks))
eval_blocks = int(eval_blocks)

profile, target_updates, grad_accum, lr, warmup, save_eval = choose_profile(stats, train_blocks)

if target_updates > 0:
    # 작은~대형 대부분은 update 수 기준으로 제한
    mode = "max_steps"
    max_steps = max(1, target_updates * grad_accum)
    num_epochs = 1
    optimizer_updates = target_updates
else:
    # 진짜 초대형만 epoch 기반
    mode = "epoch"
    max_steps = -1
    num_epochs = 2
    optimizer_updates = max(1, (train_blocks * num_epochs) // grad_accum)

# 작은 데이터는 eval/save가 오히려 시간 낭비
if save_eval <= 0 or train_blocks <= 32 or eval_blocks <= 0:
    eval_strategy = "no"
    save_strategy = "no"
    eval_steps = None
    save_steps = None
    load_best = False
    early_stopping = False
else:
    eval_strategy = "steps"
    save_strategy = "steps"
    eval_steps = save_eval
    save_steps = save_eval
    load_best = True
    early_stopping = True

return TrainPlan(
    profile=profile,
    mode=mode,
    train_blocks=train_blocks,
    eval_blocks=eval_blocks,
    max_steps=max_steps,
    num_train_epochs=num_epochs,
    gradient_accumulation_steps=grad_accum,
    optimizer_updates=optimizer_updates,
    learning_rate=lr,
    warmup_ratio=warmup,
    eval_strategy=eval_strategy,
    save_strategy=save_strategy,
    eval_steps=eval_steps,
    save_steps=save_steps,
    load_best_model_at_end=load_best,
    early_stopping=early_stopping,
)

def print_train_plan(plan: TrainPlan) → None:
print(“=” * 60)
print(“[auto training plan]”)
print(f"profile : {plan.profile}“)
print(f"mode : {plan.mode}”)
print(f"train/eval blocks : {plan.train_blocks:,} / {plan.eval_blocks:,}“)
print(f"epochs : {plan.num_train_epochs}”)
print(f"max steps : {plan.max_steps}“)
print(f"grad accumulation : {plan.gradient_accumulation_steps}”)
print(f"optimizer updates : {plan.optimizer_updates:,}“)
print(f"learning rate : {plan.learning_rate}”)
print(f"warmup ratio : {plan.warmup_ratio}“)
print(f"eval/save : {plan.eval_strategy} / {plan.save_strategy}”)
print(“=” * 60)

============================================================

검증 프롬프트 자동 생성

============================================================

def safe_cut_at_sentence(text: str, min_len: int = 35, max_len: int = 100) → str:
text = normalize_text(text)
if not text:
return “”

if len(text) <= max_len:
    return text

endings = ["다.", "요.", "까?", "까.", "!", "?", ".", "”", "\""]
best = -1
for ending in endings:
    pos = text.rfind(ending, min_len, max_len)
    if pos >= 0:
        best = max(best, pos + len(ending))

if best >= min_len:
    return text[:best].strip()

cut = text.rfind(" ", min_len, max_len)
if cut < min_len:
    cut = max_len
return text[:cut].strip()

def make_segment_prompt(text: str, ratio: float, target_len: int = 95) → str:
“”"
긴 글의 앞/중/후반 위치에서 자연스러운 짧은 이어쓰기 프롬프트를 만든다.
긴 원문 전체를 넣지 않는다.
“”"
text = normalize_text(text)
if not text:
return “”

n = len(text)
start = int(n * ratio)
start = max(0, min(start, max(0, n - target_len - 1)))

window_start = max(0, start - 140)
window_end = min(n, start + target_len + 180)
window = text[window_start:window_end]

local_start = start - window_start

# 문장 중간에서 시작하는 것을 줄인다.
prev_candidates = []
for ending in ["다.", "요.", "!", "?", "."]:
    prev_candidates.append(window.rfind(ending, 0, local_start))
prev = max(prev_candidates)

if prev >= 0 and local_start - prev <= 120:
    local_start = prev + 2

segment = window[local_start : local_start + target_len + 80].strip()
return safe_cut_at_sentence(segment, min_len=30, max_len=target_len)

def build_prompt_items(raw_items: List[Dict[str, Any]], max_prompts: int = 6) → List[PromptItem]:
if GENERATION_TEST_PROMPTS:
manual: List[PromptItem] =
for idx, p in enumerate(GENERATION_TEST_PROMPTS, start=1):
prompt = safe_cut_at_sentence(p, min_len=20, max_len=100)
if prompt:
manual.append(PromptItem(prompt=prompt, position=“manual”, source_chars=len(p), line_no=idx))
return manual[:max_prompts]

candidates: List[PromptItem] = []

for item in raw_items:
    text = item["text"]
    chars = int(item["chars"])
    line_no = int(item["line_no"])

    if chars < 600:
        positions = [("beginning", 0.00)]
    elif chars < 2500:
        positions = [("beginning", 0.00), ("middle", 0.45)]
    else:
        positions = [("beginning", 0.00), ("middle", 0.45), ("late", 0.78)]

    for pos_name, ratio in positions:
        prompt = make_segment_prompt(text, ratio=ratio)
        if len(prompt) >= 20:
            candidates.append(PromptItem(prompt=prompt, position=pos_name, source_chars=chars, line_no=line_no))

if not candidates:
    return [
        PromptItem("오늘은 이상하게 마음이 복잡했다.", "fallback", 0, 0),
        PromptItem("나는 조용히 고개를 들었다.", "fallback", 0, 0),
        PromptItem("그때 문득 이상한 생각이 들었다.", "fallback", 0, 0),
    ]

def score(item: PromptItem) -> Tuple[int, int, int]:
    # 긴 글의 middle/late를 우선 확보
    pos_score = {"middle": 0, "late": 1, "beginning": 2}.get(item.position, 3)
    return (pos_score, -item.source_chars, item.line_no)

candidates.sort(key=score)

selected: List[PromptItem] = []
seen_prompt_keys = set()
seen_positions = set()

# 1차: position 다양성 확보
for item in candidates:
    if len(selected) >= max_prompts:
        break
    key = item.prompt[:80]
    if key in seen_prompt_keys:
        continue
    if item.position in seen_positions and len(seen_positions) < 3:
        continue
    selected.append(item)
    seen_prompt_keys.add(key)
    seen_positions.add(item.position)

# 2차: 남은 칸 채우기
for item in candidates:
    if len(selected) >= max_prompts:
        break
    key = item.prompt[:80]
    if key in seen_prompt_keys:
        continue
    selected.append(item)
    seen_prompt_keys.add(key)

return selected[:max_prompts]

============================================================

생성 검증/리포트

============================================================

def generate_samples(
model: AutoModelForCausalLM,
tokenizer: AutoTokenizer,
prompts: List[PromptItem],
label: str,
out_dir: str,
) → Path:
out_path = Path(out_dir)
out_path.mkdir(parents=True, exist_ok=True)
file_path = out_path / f"{label}_sample.txt"

was_training = model.training
model.eval()

gen_config = GenerationConfig(
    max_new_tokens=GENERATION_MAX_NEW_TOKENS,
    do_sample=True,
    temperature=0.72,
    top_p=0.88,
    repetition_penalty=1.18,
    no_repeat_ngram_size=4,
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=tokenizer.eos_token_id,
)

lines: List[str] = []
lines.append("=" * 80)
lines.append(f"[{label} generation sample]")
lines.append("=" * 80)

with torch.no_grad():
    for idx, item in enumerate(prompts, start=1):
        prompt = item.prompt
        inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
        input_len = inputs["input_ids"].shape[-1]

        output_ids = model.generate(**inputs, generation_config=gen_config)

        continuation = tokenizer.decode(output_ids[0][input_len:], skip_special_tokens=True).strip()
        full_text = tokenizer.decode(output_ids[0], skip_special_tokens=True)

        lines.append("")
        lines.append("-" * 80)
        lines.append(
            f"[prompt {idx}] position={item.position}, source_chars={item.source_chars}, line={item.line_no}"
        )
        lines.append(prompt)
        lines.append("")
        lines.append(f"[continuation {idx}]")
        lines.append(continuation if continuation else "(생성된 continuation 없음)")
        lines.append("")
        lines.append(f"[full output {idx}]")
        lines.append(full_text)

file_path.write_text("\n".join(lines), encoding="utf-8")

if was_training:
    model.train()

print(f"[generation] saved: {file_path.resolve()}")
return file_path

def simple_repetition_score(text: str) → Dict[str, Any]:
words = normalize_text(text).split()
if not words:
return {“words”: 0, “unique_words”: 0, “unique_ratio”: 0.0, “dot_count”: text.count(“.”)}

unique_words = len(set(words))
return {
    "words": len(words),
    "unique_words": unique_words,
    "unique_ratio": round(unique_words / max(1, len(words)), 4),
    "dot_count": text.count("."),
}

def write_train_report(
stats: DataStats,
plan: TrainPlan,
prompts: List[PromptItem],
before_path: Optional[Path],
after_path: Optional[Path],
elapsed_seconds: Optional[float],
) → Path:
report_dir = Path(GENERATION_TEST_DIR)
report_dir.mkdir(parents=True, exist_ok=True)
report_path = report_dir / “train_report.txt”

lines: List[str] = []
lines.append("=" * 80)
lines.append("[train report]")
lines.append("=" * 80)
lines.append("")
lines.append("[config]")
lines.append(f"model              : {MODEL_NAME}")
lines.append(f"data path          : {DATA_PATH}")
lines.append(f"output dir         : {OUTPUT_DIR}")
lines.append(f"max seq length     : {MAX_SEQ_LENGTH}")
lines.append(f"bf16/fp16          : {USE_BF16} / {not USE_BF16}")
lines.append("")

lines.append("[data stats]")
for k, v in asdict(stats).items():
    lines.append(f"{k:24}: {v}")
lines.append("")

lines.append("[train plan]")
for k, v in asdict(plan).items():
    lines.append(f"{k:24}: {v}")
lines.append("")

if elapsed_seconds is not None:
    lines.append("[time]")
    lines.append(f"elapsed_seconds         : {elapsed_seconds:.1f}")
    lines.append(f"elapsed_minutes         : {elapsed_seconds / 60:.2f}")
    lines.append("")

lines.append("[generation files]")
lines.append(f"before             : {before_path}")
lines.append(f"after              : {after_path}")
lines.append("")

lines.append("[prompts]")
for idx, item in enumerate(prompts, start=1):
    lines.append(
        f"{idx}. position={item.position}, source_chars={item.source_chars}, "
        f"line={item.line_no} | {item.prompt}"
    )
lines.append("")

lines.append("[judge guide]")
lines.append("- before/after continuation의 문체 변화가 있는지 본다.")
lines.append("- after가 원문을 그대로 베끼면 과적합 가능성이 있다.")
lines.append("- after가 점(.) 반복, 같은 구절 반복, 화자명 반복을 보이면 update 수를 줄인다.")
lines.append("- after가 before와 거의 같으면 update 수를 늘리거나 데이터량을 늘린다.")
lines.append("- 짧은 글은 beginning continuation, 긴 글은 middle/late continuation 변화까지 같이 본다.")
lines.append("- KB급 데이터는 지식 주입보다 문체 힌트 정도로 보는 것이 안전하다.")

report_path.write_text("\n".join(lines), encoding="utf-8")
print(f"[report] saved: {report_path.resolve()}")
return report_path

============================================================

TrainingArguments 호환 처리

============================================================

def add_training_args_compat(kwargs: Dict[str, Any]) → Dict[str, Any]:
sig = inspect.signature(TrainingArguments.init)

eval_value = kwargs.pop("_eval_strategy_value", "steps")
kwargs = {k: v for k, v in kwargs.items() if v is not None}

if "eval_strategy" in sig.parameters:
    kwargs["eval_strategy"] = eval_value
elif "evaluation_strategy" in sig.parameters:
    kwargs["evaluation_strategy"] = eval_value

if "optim" in sig.parameters:
    kwargs["optim"] = "paged_adamw_8bit"

if "gradient_checkpointing" in sig.parameters:
    kwargs["gradient_checkpointing"] = True

return kwargs

class PeriodicCleanupCallback(TrainerCallback):
def init(self, every_n_steps: int = 50):
self.every_n_steps = every_n_steps

def on_step_end(self, args, state, control, **kwargs):
    if state.global_step and state.global_step % self.every_n_steps == 0:
        gc.collect()
        if torch.cuda.is_available():
            torch.cuda.empty_cache()

============================================================

main

============================================================

def main() → None:
set_seed(RANDOM_SEED)

tokenizer = AutoTokenizer.from_pretrained(
    MODEL_NAME,
    use_fast=True,
    trust_remote_code=True,
)

if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

tokenizer.padding_side = "right"

dataset, stats, raw_items = load_jsonl(DATA_PATH, tokenizer)

split = dataset.train_test_split(
    test_size=VALID_RATIO,
    seed=RANDOM_SEED,
    shuffle=True,
)

def _tokenize(examples):
    return tokenize_for_continuation(examples, tokenizer)

tokenized = split.map(
    _tokenize,
    batched=True,
    remove_columns=split["train"].column_names,
    desc="Tokenizing",
)

train_dataset = tokenized["train"].map(
    pack_blocks,
    batched=True,
    desc="Packing train blocks",
)

eval_dataset = tokenized["test"].map(
    pack_blocks,
    batched=True,
    desc="Packing eval blocks",
)

if len(train_dataset) == 0:
    raise ValueError("train_dataset이 0블록입니다. 데이터 또는 MAX_SEQ_LENGTH 설정을 확인하세요.")

if len(eval_dataset) == 0:
    print("[warn] eval block이 0개라서 eval을 끕니다.")

print("=" * 60)
print("[packed blocks]")
print(f"train blocks       : {len(train_dataset):,}")
print(f"eval blocks        : {len(eval_dataset):,}")
print(f"tokens per block   : {MAX_SEQ_LENGTH:,}")
print("=" * 60)

plan = build_train_plan(stats, len(train_dataset), len(eval_dataset)) if AUTO_TUNE else TrainPlan(
    profile="manual",
    mode="epoch",
    train_blocks=len(train_dataset),
    eval_blocks=len(eval_dataset),
    max_steps=-1,
    num_train_epochs=50,
    gradient_accumulation_steps=8,
    optimizer_updates=max(1, (len(train_dataset) * 50) // 8),
    learning_rate=8e-5,
    warmup_ratio=0.03,
    eval_strategy="steps",
    save_strategy="steps",
    eval_steps=100,
    save_steps=100,
    load_best_model_at_end=True,
    early_stopping=True,
)
print_train_plan(plan)

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16 if USE_BF16 else torch.float16,
)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_NAME,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)

model.config.use_cache = False
model = prepare_model_for_kbit_training(model)

peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules=[
        "q_proj", "k_proj", "v_proj", "o_proj",
        "gate_proj", "up_proj", "down_proj",
    ],
)

model = get_peft_model(model, peft_config)
model.print_trainable_parameters()

prompts = build_prompt_items(raw_items, max_prompts=6) if AUTO_GENERATION_TEST else []
before_path: Optional[Path] = None
after_path: Optional[Path] = None

if AUTO_GENERATION_TEST:
    before_path = generate_samples(
        model=model,
        tokenizer=tokenizer,
        prompts=prompts,
        label="before",
        out_dir=GENERATION_TEST_DIR,
    )

collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False)

training_args_kwargs = dict(
    output_dir=OUTPUT_DIR,
    _eval_strategy_value=plan.eval_strategy,

    per_device_train_batch_size=1,
    gradient_accumulation_steps=plan.gradient_accumulation_steps,

    num_train_epochs=plan.num_train_epochs,
    max_steps=plan.max_steps,
    learning_rate=plan.learning_rate,
    lr_scheduler_type="cosine",
    warmup_ratio=plan.warmup_ratio,
    weight_decay=0.01,

    logging_steps=10,
    eval_steps=plan.eval_steps,
    save_strategy=plan.save_strategy,
    save_steps=plan.save_steps,
    save_total_limit=SAVE_TOTAL_LIMIT,

    load_best_model_at_end=plan.load_best_model_at_end,
    metric_for_best_model="eval_loss" if plan.load_best_model_at_end else None,
    greater_is_better=False if plan.load_best_model_at_end else None,

    bf16=USE_BF16,
    fp16=not USE_BF16,

    report_to="none",
    remove_unused_columns=False,
    dataloader_num_workers=0,
)

training_args = TrainingArguments(**add_training_args_compat(training_args_kwargs))

callbacks: List[TrainerCallback] = [PeriodicCleanupCallback(every_n_steps=50)]
if plan.early_stopping:
    callbacks.insert(0, EarlyStoppingCallback(early_stopping_patience=3))

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset if plan.eval_strategy != "no" else None,
    data_collator=collator,
    callbacks=callbacks,
)

start_time = time.time()
trainer.train()
elapsed = time.time() - start_time

if AUTO_GENERATION_TEST:
    after_path = generate_samples(
        model=model,
        tokenizer=tokenizer,
        prompts=prompts,
        label="after",
        out_dir=GENERATION_TEST_DIR,
    )

    write_train_report(
        stats=stats,
        plan=plan,
        prompts=prompts,
        before_path=before_path,
        after_path=after_path,
        elapsed_seconds=elapsed,
    )

model.save_pretrained(OUTPUT_DIR)
tokenizer.save_pretrained(OUTPUT_DIR)

print(f"[done] saved adapter/tokenizer to: {Path(OUTPUT_DIR).resolve()}")
if AUTO_GENERATION_TEST:
    print(f"[done] generation check dir: {Path(GENERATION_TEST_DIR).resolve()}")

if name == “main”:
main()
I was trying to use the train.jsonl file universally. This is an attempt to automatically process cases where the number of samples is large but the amount of text is small, or, conversely, when the number of samples is small but the amount of text is large.
It was a learning experience.
However, it ended up being an explanatory model rather than the creative model I wanted.
Chetgpt says it’s my fault.
I’m insisting that I chose the wrong base model.
Chet gpt is the one who recommended llama-3-Korean-Bllossom-8B, and now he is saying that I chose the wrong base model, which is absurd.
What do you guys think?

[John6666]

Hmm… From what you describe, I get the impression that several different tasks may have been mixed together. In that situation, ChatGPT tends by default to prioritize the task as the user framed it and search for an answer “inside” that frame, so it can sometimes lead you into a dead end. I cannot tell whether that is what happened in your case, but it may be safer to first locate where you currently are in the larger problem space:

---

Short answer

Your concern about the starting checkpoint is reasonable.

MLP-KTLim/llama-3-Korean-Bllossom-8B is not a plain Korean base model. Its model card says that it received Korean instruction tuning and DPO-based alignment, and its official example uses a helpful-assistant system prompt and a chat template.

That makes the checkpoint a plausible contributor to explanatory or assistant-like output. It is not the cleanest baseline for testing pure prose continuation.

However, I would separate two questions:

1. Is the current training objective aimed at the final writing behavior you want?
2. Did this particular training run work as intended?

Both may matter.

A raw-prose LoRA can plausibly become a useful prose-continuation or style adapter. It does not, by itself, train the whole stack normally involved in long-form writing: long-output behavior, planning, persistent story state, continuity checking and revision.

Separately, the current script has a few mechanical and evaluation points worth checking before using its output as evidence that the whole direction failed.

The most informative checks before another full run would be:

• compare the checkpoint without the adapter under raw continuation, chat request and assistant-prefill inputs;
• verify the actual optimizer-update count;
• inspect whether genuine EOS tokens remain supervised after collation;
• compare adapter-off and adapter-on outputs with unseen prompts and paired generation settings.

Where the current training run appears to sit

Long-form writing
│
├─ Local prose generation
│    ├─ vocabulary and sentence style
│    ├─ dialogue / description balance
│    ├─ short scene continuation
│    └─ prose-mode behavior
│         ↑
│         └─ The current raw-text LoRA mainly operates here
│
├─ Long-output behavior
├─ Instruction-to-story behavior
├─ Plot and chapter planning
├─ Persistent character / world / timeline state
├─ Continuity checking
└─ Revision and selection

The current dataset format, {"text": "..."}, and causal next-token objective mainly teach local prose distribution and continuation.

That is a valid component. It is not the same thing as a complete long-form writing system.

A useful distinction is:

Long context
≠
Long output
≠
A coherent long-form narrative

A model may accept a long input but still stop after a short response. It may generate many words but lose plot constraints. It may write attractive individual scenes while forgetting character knowledge, chronology or unresolved plot threads across chapters.

Choose the route by the intended final behavior

Intended behavior	More direct route	Possible role of the current LoRA
Continue an existing passage for a few paragraphs	Korean-capable base model + raw-prose LM/LoRA	Main component
Write a scene or chapter from instructions	Instruct model + prompt/completion or conversational SFT	Optional prose/style component
Produce a very long response in one request	Long-output examples, length conditioning and/or decomposition	May improve prose, but does not provide length control alone
Maintain a coherent multi-chapter story	Planner + persistent story state + writer + checker/revision	Writer component
Interactive RP or collaborative writing	Chat/RP behavior + character/lore/session memory	Optional narration or character-style component

So there is not only one possible verdict.

If the goal is short prose continuation

The current direction still has a destination. I would test a Korean-capable base checkpoint, repair the evaluation, and measure whether the adapter improves unseen continuation.

If the goal is instruction-to-story generation

Raw prose alone does not teach the relationship between an instruction and the desired story response. Prompt/completion examples would be more direct.

If the goal is coherent multi-chapter writing

The current LoRA may remain useful, but mainly as the prose-writing layer inside a larger workflow.

Three reasonable next paths

These are alternatives, not a list of mandatory homework.

Path A — Repair and interpret the current experiment

Choose this if the main target is prose continuation and you want to know whether the current adapter actually works.

Focus on:

• checkpoint behavior before training;
• optimizer-step count;
• EOS supervision;
• held-out paired evaluation.

Path B — Test the model inside an existing writing workflow

Choose this if the larger goal is long-form writing and you want to see how much planning, memory and revision change the result without building everything yourself.

For example, the same adapter could be tested as the writer model inside an open-source writing environment or pipeline.

Path C — Retrain with a checkpoint and objective matched to the target

Choose this if the current objective does not match the intended interface.

• raw continuation → base checkpoint + raw prose;
• instruction-to-story → instruct checkpoint + prompt/completion data;
• RP/co-writing → chat/RP data and runtime character/lore state;
• long-output writing → long completions and/or decomposed generation.

Why long-form writing is normally split into several capabilities

The TRL SFTTrainer documentation and dataset-format guide distinguish standard language-modeling data, prompt/completion data and conversational data.

They may all use next-token prediction internally, but the examples teach different contracts.

Raw language modeling

{
  "text": "The rain had continued for three days. Yuna stood by the window..."
}

This directly teaches prose distribution and continuation.

Prompt/completion writing

{
  "prompt": "Write a restrained first-person horror scene set in an empty railway station. Output only the prose.",
  "completion": "The last train had already left when I noticed that the clock..."
}

This teaches:

instruction and constraints
→
story response

Planning

story so far
+ unresolved plot threads
+ chapter objective
→
next chapter plan

Revision

draft
+ continuity report
→
revised draft

The important statement is not that fine-tuning can never teach planning, memory or revision. It can, with suitable examples and objectives.

The narrower point is:

The current raw-prose objective does not directly supervise those capabilities.

The LongWriter paper reported that models with long input contexts could still struggle to produce long outputs when their supervised post-training data lacked long completions. Its AgentWrite implementation decomposes an ultra-long task into planned sections; the resulting long outputs were then used as SFT data.

Story-generation systems also commonly add explicit structure around the writer:

• Re3 uses planning, repeated story-state prompting, reranking and revision.
• Its open-source implementation shows how these parts are separated.
• DOME combines dynamic hierarchical outlining with temporal memory.
• DOC uses detailed outline control for longer stories.

These are examples rather than mandatory architectures. Their common lesson is that attractive next-token generation is only one part of long-form writing.

Checkpoint behavior and input-format checks

The Bllossom model card explicitly describes:

• Korean instruction tuning;
• DPO-based alignment;
• a helpful-assistant system prompt;
• chat-template usage;
• EOS and <|eot_id|> as generation terminators.

This does not prove that the checkpoint caused the observed result. The missing control is its output before fine-tuning.

Using one newly written Korean story prefix, compare:

A. Raw continuation

Tokenize only the prose prefix and continue it.

B. Chat request

Ask the model to continue the passage using its official chat template.

C. Assistant prefill

Place the prose prefix inside the final assistant message and continue that message.

The Transformers chat-template guide describes continue_final_message=True as a response-prefill mechanism.

Possible interpretations:

Observation	More likely explanation
Raw continuation is already explanatory without the adapter	Existing checkpoint behavior
Only the chat request is explanatory	Prompt/template path
Assistant prefill produces prose	Response-start format
All three behave similarly	Look more strongly at training, data and decoding
Only adapter-on output becomes more explanatory	Training-run or dataset issue

It is also worth saving raw generated token IDs.

Bllossom’s official example uses both EOS and <|eot_id|> as terminators, while the script appears to pass only tokenizer.eos_token_id. Since skip_special_tokens=True hides special tokens, an EOT token followed by additional generation may be invisible in the decoded comparison.

Check:

Was EOS generated?
Was <|eot_id|> generated?
Did generation continue after EOT?

Also record the exact repository and revision. The thread title, the placeholder model name in the script and the described Bllossom checkpoint do not all use the same Llama version label. Debugging should be based on the exact model ID, tokenizer and revision rather than only “Llama 3.1” or “Llama 3.2.”

Mechanical checks in the current training script

1. max_steps and gradient accumulation

The plan appears to do:

max_steps = target_updates * grad_accum
optimizer_updates = target_updates

The official Trainer documentation and TrainerState documentation describe global_step and max_steps in terms of completed update steps.

Gradient accumulation controls how many micro-batches contribute to one update. It is normally not multiplied into the desired optimizer-update count a second time.

If target_updates really means optimizer updates, the expected form would normally be:

max_steps = target_updates

Under the current profiles:

Profile	Printed target updates	Gradient accumulation	max_steps passed to Trainer
tiny	8	1	8
small	16	1	16
medium	32	2	64
large	64	4	256
xlarge	128	8	1024

That would make medium, large and xlarge perform 2×, 4× and 8× the update count printed as optimizer_updates, assuming normal Trainer semantics.

This does not directly explain an instructional tone, but it changes:

• effective token exposure;
• passes over the data;
• scheduler behavior;
• overfitting risk;
• repetition and memorization risk;
• the interpretation of the report.

A tiny instrumentation run can settle this:

micro-batches consumed
backward passes
optimizer steps
trainer.state.global_step
tokens seen
approximate dataset passes

2. EOS supervision

The script:

1. appends EOS to every source row;
2. assigns EOS as PAD if the tokenizer has no PAD token;
3. uses DataCollatorForLanguageModeling.

Transformers issue #23530 demonstrates a configuration where pad_token == eos_token caused genuine EOS labels to be replaced with -100, not only padding positions.

That report used a particular tokenizer and library version, so it is not proof that the same thing happened here. It is simply inexpensive to audit.

eos_positions = batch["input_ids"].eq(tokenizer.eos_token_id)
supervised_eos = eos_positions & batch["labels"].ne(-100)
masked_attended_eos = (
    eos_positions
    & batch["attention_mask"].eq(1)
    & batch["labels"].eq(-100)
)

print("EOS in input:", eos_positions.sum().item())
print("Supervised EOS:", supervised_eos.sum().item())
print("Attended but masked EOS:", masked_attended_eos.sum().item())

If all genuine document-ending EOS tokens are masked, the model is not learning those boundaries.

3. Automatic training-plan heuristics

The script chooses learning rate, updates, accumulation and evaluation behavior from file size, token count and block count.

That is convenient, but equal token counts can describe very different data:

• one novel;
• several unrelated novels;
• independent scenes;
• one author;
• many authors;
• a base checkpoint;
• an instruction/DPO checkpoint.

Rather than assuming one calculated update count is correct, a small checkpoint curve can be more informative:

8 updates
16 updates
32 updates
64 updates

Evaluate all four on the same held-out prompts.

That shows whether style adaptation improves gradually, whether repetition begins, and whether memorization appears before the final checkpoint.

4. Early stopping

If evaluation is infrequent relative to the total number of update steps, early stopping may never receive enough evaluation events to act.

For example, a patience of three evaluations cannot meaningfully stop a run that performs only one scheduled evaluation before completion.

5. QLoRA itself

The script uses 4-bit NF4 quantization, double quantization, prepare_model_for_kbit_training, and LoRA on the main attention and MLP projection layers.

That is recognizably within the ordinary QLoRA family described in the PEFT quantization guide.

It does not establish that every hyperparameter is ideal, but 4-bit QLoRA alone is not a strong explanation for a semantic shift toward explanatory prose.

I would inspect checkpoint behavior, steps, labels, data and evaluation before treating quantization as the primary cause.

Why the current before/after comparison is difficult to interpret

The comparison generation uses:

do_sample=True
temperature=0.72
top_p=0.88
repetition_penalty=1.18
no_repeat_ngram_size=4

A seed is set near the beginning of the program, but the same seed is not restored immediately before both the before-training and after-training generations.

Training, model initialization and data loading consume random state. Therefore, the two samples are not necessarily paired random draws.

The automatic comparison prompts also appear to be created from the original raw_items, before the train/test split. A prompt may therefore come from a training row.

This mixes:

• adapter effects;
• random sampling variation;
• memorization;
• generalization;
• decoding constraints.

Diagnostic comparison

Use:

same unseen prompt
adapter off / adapter on
do_sample=False
same stopping tokens
same output limit
minimal repetition constraints

Creative-quality comparison

Use:

same prompt set
same generation configuration
same explicit seed list before and after
multiple seeds per prompt
blind comparison where practical

Useful holdout levels

1. A withheld passage from the same work.
2. A withheld chapter or work.
3. A newly written passage with similar stylistic goals.
4. A neutral passage in another style or genre.

If rows are scenes from the same novel, a random row-level split still shares characters, world facts, author style and plot context across train and evaluation.

Depending on the question, chapter-group or work-level holdout may be more informative.

Separate local-adapter evaluation from long-form-system evaluation

Local prose evaluation

• Does it continue in prose mode?
• Is the local transition coherent?
• Does it maintain POV and tense?
• Did the target style appear on unseen text?
• Did repetition increase?
• Does it stop naturally?
• Is it reproducing training text?

Long-form-system evaluation

• Does it maintain character state?
• Does it preserve chronology and causality?
• Does it follow the outline?
• Does it track unresolved plot threads?
• Does quality degrade over chapters?
• Does revision repair detected contradictions?

A LoRA can succeed on the first list while the overall writing system still fails on the second. That would be a component-boundary result, not necessarily a failed adapter.

How to read the possible outcomes

Observation	Likely next interpretation
Bllossom is explanatory before training in raw continuation	Checkpoint behavior is a strong contributor
Only chat requests are explanatory	Prompt/template path is important
Assistant prefill fixes the mode	Response-start formatting may be sufficient for some uses
Adapter-on alone increases explanatory output	Inspect data, steps and training trajectory
Genuine EOS labels are masked	Repair labels/collation before retraining
Actual updates exceed the printed target	Repair the training-plan calculation
Only training-derived prompts improve	Memorization or overfitting
Unseen prompts show stable stylistic improvement	LoRA works within its intended local scope
Prose improves but multi-chapter consistency does not	The adapter is a writer component, not the whole system
A base checkpoint adapts while Bllossom does not	Interaction with existing alignment becomes more plausible
A capable model plus planning/memory works without custom training	Orchestration may be a larger bottleneck than adaptation
Every model fails at the same stage in one framework	Inspect the framework’s prompt/state contract

Existing open-source systems that may help define or test the destination

The choice is not only:

keep fine-tuning
or
build a large writing pipeline from scratch

Open-source systems exist at several levels.

Human-guided continuation: KoboldCpp

KoboldCpp is an AGPL-licensed local GGUF runtime with the KoboldAI Lite interface.

It includes:

• Storywriter mode;
• persistent stories;
• Memory;
• Author’s Note;
• World Info;
• editing, retry and undo;
• swappable local models and samplers.

It is not an automatic multi-chapter planner. It is useful for checking whether a checkpoint or adapter works as a human-guided prose-continuation engine under a more appropriate writing interface.

RP, lore and collaborative writing: SillyTavern

SillyTavern is a mature open-source frontend for chat, RP and interactive writing.

Its World Info/lorebook and Data Bank mechanisms are examples of keeping mutable story information outside model weights and injecting relevant information at runtime.

It is most relevant if the intended goal is character-driven co-writing, RP or persistent lore rather than autonomous novel generation.

Integrated writing workflow: InkOS

InkOS is an AGPL-licensed open-source story-creation system for novels, scripts and related long-form projects.

Its source exposes a broader workflow involving planning, context composition, drafting, persistent files, auditing and revision.

Its value here is not proof that it is the best novel generator. It is a concrete, inspectable reference for what a full writing stack may contain beyond the writer model.

Smaller model-swappable pipeline: GOAT Storytelling Agent

GOAT Storytelling Agent implements a top-down flow roughly like:

topic
→ book specification
→ chapter plot
→ scene split
→ scene generation

It is closer to a developer or research pipeline than a polished writing application, but it is useful for testing a model as one stage of a larger system.

Research references

• LongWriter / AgentWrite: long-output decomposition and long-output training data.
• Re3 implementation: planning, state prompting, reranking and revision.
• DOME: dynamic outlines and temporal memory.
• DOC: detailed outline control.

Research repositories may have older dependencies, proprietary API assumptions or narrow experimental setups. They are best treated as design references unless you intend to reproduce the research environment.

Using an existing system as a diagnostic baseline

Connect the same model conditions to the same workflow:

A. Bllossom without the adapter
B. Bllossom with the current adapter
C. A comparison checkpoint

Keep constant:

outline
story state
chapter objective
prompt templates
sampler
revision count
framework version

Then compare:

Result	Possible interpretation
Planning/state improves all models	Missing orchestration was important
Adapter improves only prose quality	Adapter is useful as the writer layer
Adapter worsens explanation or repetition	Training-run problem remains
Another model reaches the goal without fine-tuning	Model selection or operation may matter more
All models fail at the same workflow stage	Framework contract may be the bottleneck

Checkpoint choices if retraining is still appropriate

For pure continuation, compare against a Korean-capable base checkpoint rather than immediately switching to another RP or instruction model.

Examples of comparison baselines—not universal recommendations—include:

• kakaocorp/kanana-1.5-8b-base
• Qwen/Qwen3-8B-Base

The useful comparison is:

plain base checkpoint
versus
instruction/DPO-aligned checkpoint

under the same raw-prose objective.

For instruction-to-story behavior, an instruction checkpoint may be appropriate, but the data should explicitly demonstrate the desired interface:

premise / constraints
→
story text without explanatory preamble

For RP or collaborative writing, a chat/RP checkpoint and runtime lore/character state may be more relevant than raw continuation training.

A community creative or RP model can also be useful as a behavioral baseline, but its model card should be checked for:

• base model;
• prompt format;
• training-data disclosure;
• language coverage;
• licensing;
• independent evaluation;
• intended distinction between RP dialogue and novel prose.

What I would not conclude from the current evidence

I would not yet conclude that:

• Bllossom is definitely the sole cause;
• raw-text fine-tuning is inherently wrong;
• QLoRA caused the explanatory tone;
• fixed-length packing destroyed the story structure;
• more epochs will solve the problem;
• switching to an RP model will automatically create a long-form writer;
• a longer context window automatically creates a coherent novel;
• the current samples prove improvement or degradation;
• the present adapter is useless.

The adapter may turn out to be:

• a failed training run;
• a functioning prose adapter evaluated incorrectly;
• a useful adapter attached to a mismatched checkpoint;
• a scene writer that was expected to be the whole writing system;
• unnecessary if a stronger model and an existing workflow already meet the actual goal.

Those outcomes lead to different decisions.

Information that would make later diagnosis more concrete

Any subset of the following would help future readers distinguish checkpoint behavior, training behavior and evaluation noise:

exact model repository and revision
tokenizer repository and revision
Transformers / PEFT / bitsandbytes / Torch versions
GPU and precision configuration
train_report.txt
training and evaluation loss history
trainer.state.global_step
actual total tokens seen
trainable parameter count
EOS / PAD / EOT IDs
one collator-label audit
adapter-off and adapter-on outputs for the same unseen prompt
generation seeds and configuration
number of JSONL rows
total tokens
what one row represents: paragraph, scene, chapter or full work
number of distinct works and authors
whether evaluation prompts appeared in training

These are not all prerequisites for continuing. Each one simply rules out a different class of explanation.

Bottom line

The concern about Bllossom is justified: it is already instruction-tuned and DPO-aligned, so it was not the cleanest starting point for a pure prose-continuation experiment.

But changing the checkpoint alone would not answer the larger question.

The current script can plausibly train a prose-continuation/style adapter. If the final goal is coherent long-form writing, that adapter is only one possible component. Long-output behavior, instruction following, planning, persistent story state, continuity checking and revision are separate layers.

The central decision is therefore not simply:

keep this LoRA
or
discard it

It is:

Is the artifact being trained intended to be:

1. the whole writing system,
2. an instruction-to-story model,
or
3. a prose-generation component inside a larger workflow?

Once that is fixed, the route becomes clearer:

Short prose continuation
→ base checkpoint + raw-prose LoRA

Instruction-to-scene or chapter
→ instruct checkpoint + prompt/completion SFT

Very long output
→ long-output examples and/or decomposition

Coherent multi-chapter writing
→ planner + persistent story state + writer + checker/revision
  with optional prose fine-tuning

Interactive RP or co-writing
→ chat/RP behavior + lore/character/session memory

Before committing to another full training run, the adapter-off input-format comparison, actual optimizer-step count, EOS-label audit and paired held-out generation should reveal whether the present result is mainly a checkpoint mismatch, a training accident, an evaluation problem, a scope mismatch—or some combination of them.

[masonreed11]

I think the biggest issue is the choice of base model. Instruction-tuned models naturally drift toward explaining instead of continuing stories. A base model trained with well-structured narrative data is usually a much better fit for creative writing, while your training pipeline already looks reasonable for experimentation.