Evaluations

Testing models against the job: evaluation design, results and what they mean for choosing a model.

Evaluations 17 April 2026

How much bitrate does an audio-multimodal LLM actually need?

An eval across 12 audio-capable models on OpenRouter, 5 MP3 bitrates, 4 dictation samples. The default upload quality in most dictation apps is 2–4× overprovisioned — and the more interesting finding had nothing to do with bitrate.

6 min read

#Audio
#Dictation
#Evals

Evaluations 25 March 2026

Bias and censorship evaluation tests for large language models

Test prompts and notes exploring cultural bias, censorship, and worldview differences across Western and Eastern large language models.

5 min read

#Bias
#Censorship
#Open Source