Deflated · Workbench Dispatches · 11 entries
Build logs.
Notes from training runs, shipped tools, and the things that broke along the way.
- two projects archived. two lessons i don't want to forget.Read entry →
Bayya's launch-day red team found four bugs. One IDK-1 pre-training run burned 53,850 steps on a data-order bug. both are archived; the lessons aren't.
- i launched bayya, then spent a day trying to break it myselfRead entry →
4 real bugs found by red-teaming my own shipped product: a crash, a jailbreak, a moderation gap, a garbage-output edge case. all fixed same day, then a full redesign on top.
- it took four SFT rounds to teach IDK-1 to count to threeRead entry →
106M params, 4810 instruction pairs, val loss 1.3670. IDK-1-Instruct is live on HuggingFace.
- i trained the first sundanese language model from scratch and it took 83 minutesRead entry →
40 million speakers, zero dedicated LMs. so i built one — 9M params, 122M tokens, nanoGPT from scratch.
- i built a clean indonesian text corpus because every existing one is brokenRead entry →
CulturaX taught IDK-1 nothing. so i built Cleanesia — 17,761 docs, 19M tokens, open API.
- i cancelled 53,000 steps of training because of one line of codeRead entry →
the loss wasn't flat because of learning rate. it was flat because i was training on scrambled tokens.
- IDK-1 at 47,500 steps: what the output actually looks likeRead entry →
47.5% done. loss still flat. ran inference for the first time. here's what came out.
- IDK-1 week 1: what 8,000 steps of training actually looks likeRead entry →
val loss 10.7 → 7.79. free compute, real numbers, zero drama.
- why i went from 500M to 100M parameters (it's not what you think)Read entry →
DFD-1 wasn't a failure. it was a tradeoff. here's the math.
- i trained a crisis PR model and it said 'bitch investigations' onceRead entry →
630K params, character-level tokenizer, val loss 1.03. it works if you squint.
- what i learned building an indonesian slm as a non-cs studentRead entry →
i'm a digital PR major. here's what three models taught me that no course would.