YT Auto - (Local)

I set out to automate a full YouTube content pipeline — researching trending topics, writing scripts, generating natural-sounding Telugu narration, rendering animated video, and publishing — without paying for cloud AI APIs. That single constraint turned out to be the hardest part of the project.

Cloud LLM/TTS APIs (I used Gemini early on) are reliable by default: near-unlimited scale, consistent quality, and someone else's infrastructure keeping the model alive. Running the same pipeline locally on a single RTX 3070 Ti laptop GPU (8GB VRAM) meant I had to build all of that reliability myself:

Free-tier quota exhaustion — Gemini's free tier ran out constantly (text, voice, and even image generation), which pushed the whole architecture to a local-first design: a local LLM via Ollama for scripts, and a local speech-synthesis model for narration, with Gemini only as an occasional fallback. A silent production outage traced to a GPU driver bug — one model started failing every run with a generic error. I had to dig through Ollama's own server logs to find a CUDA kernel crash specific to that model's architecture on this GPU, confirm it wasn't fixed by the latest update, and route around it with an automatic model-fallback chain instead of waiting on an upstream fix. Telugu is a second-class citizen in most ML tooling — the standard text-rendering library couldn't shape Telugu script correctly, and small open speech models default to a flat, robotic delivery. Both needed custom fixes: proper glyph shaping through a different rendering path, and hand-tuned pitch/pace/emotion parameters to get natural-sounding speech. Small local models hallucinate more than large cloud ones — I had to build my own fact-grounding layer that checks every number and claim in a generated script against the real source material before it's allowed on screen, something a bigger hosted model would need much less of.

The result runs a full daily content pipeline for effectively zero marginal cost per video, which wouldn't have been possible by just calling a cloud API and hoping it stays within budget.

1. Free-tier quota exhaustion → local-first architecture with automatic fallback chains

Rather than architecting around one AI provider, every stage (script LLM, voice, even thumbnail generation) got a chain of engines, not a single one: local model first, cloud API only as a last resort. TTS_ENGINE=local → gemini → edge for voice, and the same pattern for the script-writing LLM. Each stage tries the next option automatically and logs which one it actually used, so a quota wall on one provider degrades the pipeline instead of stopping it.

1. A GPU driver crash traced through raw server logs → self-healing model chain

When one model started failing every run, I didn't just retry it — I went into Ollama's own server logs and found the actual crash: a CUDA kernel-capability check failing for that specific model's architecture on this GPU. I isolated it by testing other models on the same GPU (they worked fine, proving it wasn't a hardware or driver problem generally), restarted and updated Ollama to rule out a stale process, and confirmed the crash was deterministic and unfixed upstream. The permanent fix: an automatic model-fallback chain that detects this exact crash signature, blacklists that model for an hour so it stops wasting time retrying something that will fail again immediately, and falls through to the next local model — turning a hard outage into a one-line warning in the logs.

1. Telugu as a second-class citizen in ML tooling → custom rendering and voice tuning Text: the standard image library (Pillow) draws Telugu with broken conjuncts because it has no font-shaping engine. I routed all on-screen Telugu through ffmpeg's HarfBuzz-based text renderer instead — rendering each string to a cached alpha mask and compositing that, so the video's on-screen text is correctly shaped. Voice: got the local speech model from flat and robotic to something with real delivery through several rounds of measurable tuning — per-speaker pitch, pace, and clarity settings, natural variable pauses instead of fixed gaps, and an "emotion director" step where the script's own local LLM reads the narration and decides the delivery style per line. For a two-host format, I found the same "energetic" style that helped the male voice made the female voice sound harsh, so each speaker got separately tuned settings rather than one shared preset — validated by building a small evaluation harness that renders both before/after versions and scores them for reproducibility, timing, and (with a disclosed-as-unreliable) automated listener, rather than just eyeballing a single sample.
2. Small local models hallucinate more → a grounding layer the cloud version didn't need

Built a fact-checking pass that sits between the LLM and the final video: the topic-research step extracts facts verbatim from real sources instead of letting the model paraphrase them (paraphrasing was quietly losing real numbers and specific names), the script-writing prompt explicitly forbids inventing any figure — including a fabricated statistic spelled out as a word instead of a digit, which is how one bad number slipped through the original numeric-only check — and any on-screen stat/comparison graphic gets dropped if its number doesn't trace back to a real source. The same discipline applies to links in the video description: only a link that's been scraped from a real source or matched to a hand-verified official URL is used, and even then only after a live reachability check right before publishing, so a dead or fabricated link never goes out.

Net result

A pipeline that runs a full research → script → voice → video → publish cycle on its own schedule, for the cost of electricity rather than a per-video API bill, with the same kind of reliability engineering (fallback chains, crash recovery, fact validation) that a paid cloud stack would otherwise have handled for me.