{ "markdown": "# rendercheck\n\n\n\n\n[![PyPI](https://img.shields.io/pypi/v/rendercheck)](https://pypi.org/project/rendercheck/)\n[![CI](https://github.com/rogermsc/rendercheck/actions/workflows/ci.yml/badge.svg)](https://github.com/rogermsc/rendercheck/actions/workflows/ci.yml)\n[![Python](https://img.shields.io/badge/python-3.10%20%E2%80%93%203.14-blue)](https://github.com/rogermsc/rendercheck)\n[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)\n\n**The worst bugs in generated media don't throw.**\n\n> \"the audio often cuts off the final sentence […] though the API returns\n> success without error signals\"\n>\n> — a developer on the [OpenAI forum](https://community.openai.com/t/1379584),\n> April 2026, describing production output\n\n![rendercheck demo](docs/demo.gif)\n\nIf you generate speech or video with a model — TTS, voice agents, podcasts,\navatars, AI video — your tests catch the exception that never happens. They do\nnot catch the narration that reads at 300 words per minute, the voice track\nsitting 18 dB below the footage it's cut against, the clip that rendered at 42%\nlength and got cached as a success, the captions that describe the audio three\nseconds before it happens, or the file whose audio track is missing entirely.\n\nThe 2026 state of the art for catching these is *a person listening to the\noutput*. That works, and it costs more than everything else in your pipeline\ncombined.\n\n`rendercheck` makes them throw.\n\n```python\nfrom rendercheck import assert_pace, assert_loudness, looks_ok\n\nassert_pace(\"episode-12.mp3\", \"episode-12.vtt\")\nassert_loudness(\"episode-12.mp3\")\nlooks_ok(\"slide-14.png\", [\"the title fits on one line\"])\n```\n\nPlain assert functions. No framework, no runner, no service. They raise\n`AssertionError`, so they already work in pytest, in CI, or in a five-line\nscript. Eighteen of the nineteen checks have **no dependencies and make no\nnetwork calls** — if you have `ffmpeg`, you're ready.\n\n---\n\n## Quickstart\n\nYou need `ffmpeg` on your PATH (`brew install ffmpeg`, `apt-get install ffmpeg`,\nor `winget install ffmpeg`). Then:\n\n```bash\npip install rendercheck\nrendercheck demo\n```\n\nOr drop a file into the **[playground](https://rogermsc.github.io/rendercheck/playground/)**\n— same checks, running on ffmpeg compiled to WebAssembly, nothing uploaded.\n\n`demo` synthesises ten defective files and runs the real checks against them,\nso you can see it fire without owning a broken render. Verbatim, first two of\neight:\n\n```\nNarration too fast\n A voice picked to match a presenter's face read English at machine-gun speed. Valid audio, correct timing, perfectly in sync.\n\n $ rendercheck check machine-gun.wav --script narration.vtt\n\n FAIL pace narration pace 300 WPM exceeds 245 (300 words in 60.0s) -- this reads as machine-gun delivery and listeners cannot follow it: machine-gun.wav\n PASS loudness -16.1 LUFS\n PASS dead air 0.0 s silence\n PASS truncation 8.9 dB of fall-off at the end\n PASS clipping 0 samples at 0 dBFS\n\nLevels that don't match\n Synthesised narration landed 18 dB under the footage it was cut against. Nobody noticed until viewers rode the volume knob.\n\n $ rendercheck check too-quiet.wav\n\n SKIP pace no --script given\n FAIL loudness -34.2 LUFS is 18.2 dB quieter than the -16 target -- it will sound inaudible next to correctly-levelled audio cut alongside it: too-quiet.wav\n PASS dead air 0.0 s silence\n PASS truncation 8.8 dB of fall-off at the end\n PASS clipping 0 samples at 0 dBFS\n```\n\n…and one of the two added in 0.3.0:\n\n```\nCaptions against the wrong clock\n A concatenation added three seconds of pre-roll after the captions were written. Both files are perfectly valid on their own.\n\n $ rendercheck check late-captions.wav\n\n SKIP pace no --script given\n PASS loudness -16.0 LUFS\n PASS dead air 0.0 s silence\n PASS truncation 74.3 dB of fall-off at the end\n PASS clipping 0 samples at 0 dBFS\n FAIL captions late-captions.vtt runs 3.0s late against late-captions.wav, past the 0.75s limit -- every line arrives at the wrong moment, and both files are individually valid so nothing else catches it\n```\n\nThen point it at your own output:\n\n```bash\nrendercheck check episode-12.mp3 --script episode-12.vtt --preset podcast\n```\n\nExit code is 1 if anything failed — **or if nothing could be measured**, because\na run that looked at nothing is not a clean one. A path you typo'd exits 2.\n`--json` gives you the same report for pipelines in any language, and `--strict`\nrejects partial runs too.\n\n## Where is this file going?\n\n\"How loud should this be?\" has no single answer — it depends entirely on where\nthe file ends up, and every platform publishes a different number. `--preset`\nturns that table into something a build can enforce:\n\n```\n$ rendercheck presets\n\n preset target tol peak source\n youtube -14L 1.0dB -1.0TP YouTube normalises playback to -14 LUFS\n spotify -14L 1.0dB -1.0TP Spotify, including podcasts, at -14 LUFS\n tiktok -14L 1.5dB -1.0TP TikTok and Instagram, measured rather than published\n podcast -16L 1.0dB -1.0TP AES71 / Apple Podcasts: -16 LUFS stereo, -19 mono\n apple -16L 1.0dB -1.0TP Apple Music Sound Check, -16 LUFS\n web -16L 2.0dB -- spoken-word web video -- rendercheck's own defaults\n ebu -23L 1.0dB -1.0TP EBU R128, European broadcast\n atsc -24L 2.0dB -2.0TP ATSC A/85, North American broadcast\n netflix -27L 2.0dB -2.0TP Netflix delivery, dialog-gated\n```\n\nNone of those numbers are ours. The contribution is that `--preset ebu` is a\ndecision a reviewer can read, where `--target-lufs -23` is a magic number the\nnext person will not dare touch. A preset that states a ceiling also switches on\nthe **true-peak** check, which catches a master measuring clean locally and\ndistorting after upload. `web` exists only to *name* the built-in defaults, so\nit states none and behaves exactly like passing no preset at all.\n\nProject-wide settings go in `rendercheck.toml` (or `[tool.rendercheck]` in\n`pyproject.toml`) so a CI step is not eight flags on one line:\n\n```toml\npreset = \"podcast\"\nmax_silence = 5.0\n```\n\nFlags you type still beat the file, and the file beats the built-in defaults.\n\nIn pytest they're just asserts — no plugin, no fixtures:\n\n```python\n@pytest.mark.parametrize(\"episode\", EPISODES)\ndef test_episode_is_shippable(episode):\n assert_pace(episode.audio, episode.vtt)\n assert_loudness(episode.audio)\n assert_no_dead_air(episode.audio)\n```\n\n## \"Isn't this forty lines of pyloudnorm?\"\n\nFor one of the nineteen checks, roughly yes. None of these measurements are novel,\nand it would be dishonest to imply otherwise:\n\n| The measurement | Already available from |\n|---|---|\n| Integrated loudness, true peak | [pyloudnorm](https://github.com/csteinmetz1/pyloudnorm), ffmpeg's `loudnorm` |\n| Silence detection | [pydub](https://github.com/jiaaro/pydub)`.silence`, ffmpeg's `silencedetect` |\n| Duration, stream layout, frame rate | `ffprobe` |\n| Black frames, freezes | ffmpeg's `blackdetect`, `freezedetect` |\n| Caption↔audio offset | [ffsubsync](https://github.com/smacke/ffsubsync) — which *corrects* it |\n| Container and codec conformance | [MediaConch](https://mediaarea.net/MediaConch) — policy-driven, pass/fail, from the CLI |\n| Speaker identity | [resemblyzer](https://github.com/resemble-ai/resemblyzer) |\n| Video quality metrics | [VMAF](https://github.com/Netflix/vmaf), [ffmpeg-quality-metrics](https://github.com/slhck/ffmpeg-quality-metrics) |\n\nMost of those hand a **number to a researcher**. The two that are already gates\ngate a different thing: MediaConch checks that a file conforms to a container\npolicy, which is a preservation question, not a perceptual one — a file can pass\nevery MediaConch rule and still be narrated at 300 WPM. ffsubsync will happily\nrealign captions that were never wrong, because it has no opinion about whether\nthey needed it.\n\nWhat is actually missing, and what this is:\n\n- **A threshold that came from a defect**, not from a paper. 245 WPM because a\n real voice narrated at 280 and shipped. −16 LUFS because narration landed at\n −34 against footage at −13.\n- **A message that says what a person would notice.** \"−34.0 LUFS\" is a reading.\n \"18.0 dB quieter than the −16 target — it will sound inaudible next to\n correctly-levelled audio cut alongside it\" is a bug report.\n- **Fail-open on infrastructure, fail-closed on a defect**, so it can sit in CI\n without becoming the thing that breaks the build for its own reasons.\n- **Exit codes and one command over a directory**, rather than a notebook.\n\nAgainst the LLM-eval tools the difference is structural rather than a matter of\ncoverage. [promptfoo](https://github.com/promptfoo/promptfoo),\n[DeepEval](https://github.com/confident-ai/deepeval) and\n[RAGAS](https://github.com/vibrantlabsai/ragas) are excellent and none of them\ncan do this: their test case is a **string**. There is no assertion to add,\nbecause there is nowhere to put the file. Use them for the script; use this for\nwhat the script turned into.\n\nAnd if you already run broadcast QC — Interra BATON, Telestream Vidchecker,\nQCTools — you have had most of this for twenty years. It just isn't in your git\nhooks.\n\n## In your pipeline\n\n**GitHub Actions** — installs ffmpeg and fails the build on a defect:\n\n```yaml\n- uses: rogermsc/rendercheck@v0\n with:\n files: out/\n preset: podcast\n strict: \"true\"\n```\n\n**Node, Remotion, anything that renders in a build step:**\n\n```bash\nnpx rendercheck check out/\n```\n\n**Docker**, if you would rather not have a Python toolchain at all — ffmpeg is\nalready in the image:\n\n```bash\ndocker run --rm -v \"$PWD:/work\" ghcr.io/rogermsc/rendercheck check /work/out.mp4\n```\n\n**Coding agents**, via MCP. An agent that just wrote a render pipeline and ran\nit has no way to tell whether the file that came back is any good; the other\nmedia MCP servers cut and transcode, which hands it *more* media rather than an\nanswer:\n\n```bash\nclaude mcp add rendercheck -- rendercheck mcp\n```\n\nIt is listed in the [MCP registry](https://registry.modelcontextprotocol.io)\nas `io.github.rogermsc/rendercheck`, so clients that read the registry can\ninstall it without being told where it lives. No key is needed — every check\nexcept `looks ok` is deterministic.\n\n`check_media` returns one verdict per check with the measured value, so the\nmodel can act on \"−34 LUFS, 18 dB under target\" rather than on a file it cannot\nhear.\n\n**promptfoo** — its assertions are all string-shaped, so an eval can confirm the\nnarration script and tell you nothing about the audio. `examples/promptfoo/`\ncloses that half in thirty lines:\n\n```yaml\nassert:\n - type: python\n value: file://rendercheck_assert.py:get_assert\n```\n\n**Anything else** — `--json` on stdout, one object per file, plus exit codes.\n\n---\n\n## Six checks, six incidents\n\nEach default is a threshold set by a defect that actually shipped, not a number\nchosen for symmetry. All output below is verbatim. Every threshold is an\nargument — see the [reference](docs/README.md) for tuning.\n\n**Narration too fast.** A voice picked to match a presenter's *face* narrated\nEnglish at ~280 WPM. The audio was valid, correctly timed, perfectly in sync. It\njust sounded like a machine gun.\n\n```python\nassert_pace(\"episode-12.mp3\", \"episode-12.vtt\", max_wpm=245)\n```\n```\nnarration pace 300 WPM exceeds 245 (300 words in 60.0s) -- this reads as\nmachine-gun delivery and listeners cannot follow it: episode-12.mp3\n```\n\n**Levels that don't match.** Synthesised narration landed at −34 LUFS and was\nconcatenated with footage at −13. Same file, a 20 dB step in the middle. Nobody\nnoticed until viewers spent 45 minutes riding the volume knob.\n\n```python\nassert_loudness(\"episode-12.mp3\", target_lufs=-16, tol=2.0)\n```\n```\n-34.0 LUFS is 18.0 dB quieter than the -16 target -- it will sound inaudible\nnext to correctly-levelled audio cut alongside it: episode-12.mp3\n```\n\n**Truncated renders cached as successes.** Encode failures produced clips a\nfraction of their intended length, which the pipeline cached as *succeeded*.\nRetries only re-ran the ones that had **errored** — and these hadn't.\n\n```python\nassert_duration(\"segment-07.mp4\", expected_seconds=24.0)\n```\n```\nsegment-07.mp4 is 10.0s -- 42% of the expected 24.0s. A render this short is a\nsilent encode failure, not a short take; re-render rather than retry\n```\n\n**Holes in the middle.** Compositing failed transiently *and silently*, leaving\ndead stretches mid-file. Right length, right average loudness. The hole only\nexisted in the middle.\n\n```python\nassert_no_dead_air(\"episode-12.mp3\", max_silence=3.0)\n```\n```\n6.2s of silence starting at 0:41 exceeds the 3s limit -- a gap this long\nmid-file is a dropped segment, not a pause (2 found in total): episode-12.mp3\n```\n\n**The wrong person speaking.** A script said \"I'm Jordan\" while the system had\nassigned Alex. A whole module rendered with the wrong face and the wrong voice.\n**Every other gate passed.**\n\n```python\nassert_speaker(script, expected=\"Alex\", known_names=[\"Alex\", \"Jordan\", \"Sam\"])\n```\n```\nthe script introduces the presenter as \"Jordan\" but Alex is assigned -- the\nrendered avatar would introduce itself with someone else's name. Fix whichever\nis wrong: the assigned presenter, or the name in the script\n```\n\nThe `known_names` roster is required, and it's the whole trick: without it, a\ncharacter in a scenario saying *\"I'm Rosa, a nurse\"* trips the check on every\nscript that tells a story. Only a name belonging to someone who could actually\nhave been cast counts as a claim about the speaker.\n\n**Things you can only see.** Overflowing titles, colliding logos, half-empty\ncanvases, figures cropped mid-caption. All rendered without error.\n\n```python\nlooks_ok(\"slide-14.png\", [\"the title fits on one line\", \"no text is clipped\"])\n```\n```\n[major] slide-14.png: the title wraps to three lines and overlaps the logo in\nthe top-right corner -- failed rubric item: 'the title fits on one line'\n```\n\nThis is the only check that needs a key: `pip install \"rendercheck[vision]\"`.\n\n## Thirteen more, for defects other people keep reporting\n\nThe six above came out of one pipeline. These came from reading other people's\nbug reports — the same complaint, filed against every provider in turn:\n\n| Check | The defect |\n|---|---|\n| `assert_no_truncation` | Speech that stops mid-sentence while the API returns success. The single most-reported defect in generated audio; measured against the file's own average, so it holds for quiet and loud content alike. |\n| `assert_has_sound` | A clip that comes back silent — an upscale step drops the audio track, a mux points at the wrong stream, a synthesis writes zeroes. |\n| `assert_no_clipping` | A gain stage pushed the waveform past full scale. Crackles on consonants, and turning it down afterwards does not undo it. |\n| `assert_true_peak` | Measures clean locally, distorts after upload. Loudness and peak are different problems: a lossy codec reconstructs the waveform *between* samples, and clips where it goes over. |\n| `assert_no_black_frames` | Generated video truncating to black instead of erroring: right length, valid container, nothing in the last third. |\n| `assert_not_frozen` | The picture stops moving. Every frame present, every frame the same frame. |\n| `assert_captions_aligned` | Captions written against one clock, audio rendered against another. Every line arrives at the wrong moment, and both files are individually perfect. |\n| `assert_streams_aligned` | Sound and picture that do not cover the same stretch of time — a mux that ran out of one input, or a concatenation that mistimed its first segment. |\n| `assert_format` | A render that quietly fell back to 720p, came out at the wrong frame rate, or is variable-rate where the pipeline downstream assumes constant. |\n| `assert_not_blank` | An image generator that failed and returned an empty canvas — reported against DALL·E, Stable Diffusion, Qwen, Gemini and Krita, always the same way: no error, no warning, correct dimensions, nothing on it. Catches any flat canvas, not only a black one; `blackdetect` sees none of the others. |\n| `assert_loudness_range` | A file with no single workable volume setting: turned up for the quiet passages, the loud ones startle. Different question from `assert_loudness`, which only asks where the middle sits. |\n| `assert_audio_format` | Mono delivered where stereo was specified, or 44.1 kHz where the spec says 48 — resampled downstream by whichever converter happens to be in the chain. |\n| `assert_lip_sync` | A talking head whose mouth is out of time with its voice. The container is correct, the streams are valid, the durations agree, and `assert_streams_aligned` passes — because that reads the container's timing and this reads the content's. Off by default, and it declines to answer more often than it answers: see the note under *What it does not check*. |\n\n**The blank check is the one that needs no key.** Until it existed, a still had\nto go through the vision tier before this tool would say anything about it at\nall, so `rendercheck check slide.png` measured nothing and exited non-zero. It\nreads the spread between the bottom and top of the luma distribution rather than\nminimum to maximum, which is what makes it hold up: a blank frame carrying one\nstray artifact spans the full range on min/max and still reads as blank here.\n\n**The caption check is the one with no equivalent anywhere.** [ffsubsync] and\nfriends *correct* drift; the online validators lint the `.srt` on its own —\noverlapping cues, reading speed, empty rows. Neither asks whether this caption\nfile matches this audio, which is the only question a build needs answered. It\nworks by matching the shape of the talking against the shape of the cues, and it\nreports a constant offset and a *drift* separately: an offset is one shift from\ncorrect, and drift is not fixable by any single shift.\n\nEvery one of these **skips rather than passes** when it cannot measure — no\nvideo stream, no silence structure to align against, no per-stream duration in\nthe container. `blackdetect` on a `.wav` reports nothing, and nothing would\notherwise read as \"looked, all clean\". That is the same trap as the regression\nbelow, and it is the reason a run where everything skipped exits non-zero.\n\n[ffsubsync]: https://github.com/smacke/ffsubsync\n\n---\n\n## Found on real files\n\nNot a synthetic benchmark. Pointed at the output of a production pipeline that\nrenders narrated video at scale:\n\n| | result |\n|---|---|\n| A course known to be good | clean — 162 WPM, −14.1 LUFS, no dead air |\n| Content re-rendered *after* a loudness fix landed | **passes** at −14.2 LUFS |\n| Four episodes rendered *before* that fix | **fails** at −19.4 to −21.3 LUFS |\n\nIt drew the line exactly where the fix landed, on files it was never told\nanything about, agreeing with a conclusion humans had reached months earlier.\n\n**That table is an anecdote, and it is labelled as one.** The media is private,\nso nobody — including its author — can rerun it. The numbers you *can* check are\nin [docs/calibration.md](docs/calibration.md), produced by `bench/bench.py`:\n\n```bash\npython bench/bench.py sync # no network, no corpus, about a minute\n```\n\nThat script injects defects of known size into media with a known answer and\ncounts what each check actually said, with abstentions in the denominator and a\nrule-of-three bound printed next to every zero. Where a threshold in this\nlibrary has a measured provenance, that is where it came from.\n\n## Two promises\n\n**It fails open on infrastructure.** No ffmpeg, no key, no network, no\nmeasurement → it warns and passes. A gate that blocks your pipeline because of\nits *own* breakage gets deleted within a week, and then it protects nothing. A\ndefect fails closed; the checker fails open. (A missing file is your typo, not\ninfrastructure — that still raises.)\n\n**Silence is never mistaken for success.** A check that couldn't run says so, as\na `rendercheck.Skipped` warning and in the CLI output. An empty run never reads\nas a clean one.\n\nWe had to earn the second one, twice.\n\nThe first cut of this library returned **PASS** for a file with no audio track\nat all — `silencedetect` reports nothing when there is nothing to analyse, and\nthat read as \"no silence found\". It is now the loudest failure in the suite,\nwith a regression test named after it, and the line it taught is the rule\neverything else follows: *if we measured and it is wrong, fail closed; if we\ncould not measure, fail open.*\n\nThen, before releasing under this name, we audited the tool against its own\npremise and found **seven more**. Every one of them reported success without\nhaving established it:\n\n- An all-skipped run exited `0`. No ffmpeg on the runner meant a green build.\n- A **typo'd file path** exited `0`, contradicting the promise two paragraphs up.\n- A typo'd `--script` path was read as narration — one word — and produced a\n confident, wrong verdict *about the audio*: `1 WPM is below 110`.\n- `--presenter` without `--known-names` defaulted the roster to the assigned\n presenter, which made the speaker check **structurally incapable of firing**.\n It printed `PASS` on a script naming somebody else.\n- `looks_ok` blamed a missing API key for every exception, so an SDK mismatch\n passed forever and Bedrock users were sent chasing the wrong thing.\n- ffprobe failures leaked a raw Python list into the message.\n- Without ffmpeg the test suite crashed on collection instead of skipping.\n\nAll seven are fixed, each with a test that fails without the fix, and the exit\ncodes are now a contract: `0` measured and clean, `1` a defect **or nothing\nmeasured**, `2` a path that isn't there. Details in the\n[changelog](CHANGELOG.md).\n\nA tool that catches silent failures is worth exactly as much as its own honesty\nabout them.\n\n## What it does not check\n\nBeing explicit, because a QA tool that implies more coverage than it has is\nworse than none:\n\n- **Lip sync, to the tolerance that matters.** `assert_lip_sync` correlates\n picture motion against the speech envelope and catches a mouth most of half a\n second out of time with its voice, with no face detection and no model. Two\n things it cannot do. It cannot resolve the errors that actually matter\n perceptually — EBU R37 puts detectability at +40 ms of audio lead and −60 ms\n of lag, and this catches under a third of 200 ms errors. And it declines to answer on\n about two files in three, because on those the correlation peak does not stand\n clear enough to read a number off. It finds a broken pipeline; it does not\n certify a good one, and that still needs a model\n ([SyncNet](https://github.com/joonson/syncnet_python) and friends).\n- **Perceptual video quality.** No PSNR, SSIM, or VMAF — those need a reference\n encode to compare against, which generated media does not have.\n- **Whether the narration is *correct*** — only how fast it's read, and whether\n the captions line up in time. Nothing here transcribes the audio, so a voice\n reading the wrong script at a reasonable pace passes everything. That check\n wants Whisper, which is a dependency out of proportion to the rest of this.\n Groundedness and factual accuracy are a different problem again, well covered\n by the LLM-eval tools.\n- **Speech intelligibility.** Loudness is not clarity; a correctly-levelled\n track can still be mumbled.\n- **Music, mixing, or anything non-speech.** The defaults assume spoken word.\n- **The rubric you didn't write.** `looks_ok` only checks what you ask it to.\n\n## More\n\n- [Symptoms](docs/symptoms.md) — \"the TTS cut off the last sentence\", \"the clip\n came back silent\", and which check catches each\n- [Reference and tuning](docs/README.md) — every threshold, and when to turn a\n check off\n- [Contributing](CONTRIBUTING.md) — the bar for a new check\n- [Changelog](CHANGELOG.md)\n\nMIT.\n", "bytes": 24273, "sha": "11a59c9b723650e8c35d7483767049406a64826f2d479beddf38107f7a33dc2d", "repo_slug": "rogermsc/rendercheck", "fonte": "repo", "truncated": false, "api": "https://agentalog.com/api/listings/mcp_io_github_rogermsc_rendercheck_d760b409/readme" }