● active2019

Recipes

A personal recipe manager six years in the making — from a flat JSON archive to a Turbo monorepo with an AI-assisted scraping pipeline.

Next.jsTanStack StartPostgreSQLMeilisearchSeaweedFSOpenAITurborepotRPC

highlights

  • Three-pass AI scraping pipeline with real-time SSE progress
  • Related-recipe discovery via Meilisearch + GPT tool calls with human approval in the admin UI
  • 545 recipes collected since 2019
  • Replaced Google App Engine with a self-hosted Komodo stack on the homelab

Overview

Started as a flat JSON archive published to NPM in late 2019, this project has gone through three distinct eras over six years. It’s now a Turbo monorepo with a public Next.js site and a TanStack Start admin app purpose-built for the scraping pipeline. The site is live at recipes.cortlan.dev.

Six Years of Recipes

  1. 2019JSON archive + @cortl/recipes NPM package
  2. 2020Gatsby frontend on Firebase Hosting
  3. 2021Next.js + TypeScript + PostgreSQL on GAE
  4. 2022AI scraping pipeline introduced
  5. 2026Turbo monorepo · TanStack admin · SeaweedFS

The Scraping Pipeline

The original approach was a parser-per-site setup — one file per domain, brittle to layout changes, locked to a handful of sites. That was the only real option in 2019, before LLMs were available. The current pipeline replaced it with a three-pass AI approach:

  1. HTML → Markdown — first pass strips noise and normalizes structure
  2. Markdown → JSON — second pass applies the response schema
  3. Ingredients, tags, and timings — a third isolated prompt handles the fields most prone to schema drift

Related-recipe linking uses Meilisearch paired with GPT tool calls. The model brainstorms candidates, queries the index, and proposes matches — the admin UI lets me approve before anything is written.

Architecture

recipes/
├── apps/admin    ← TanStack Start + tRPC  (pipeline, draft management)
└── apps/ui       ← Next.js 16 + GraphQL   (public site, full-text search)

PostgreSQL, Meilisearch, SeaweedFS, and a headless Chrome instance for Puppeteer all run as a single Komodo stack on the homelab.

learnings

  • GAE cold starts were the forcing function to move infrastructure home
  • A separate prompt for ingredients, tags, and timings solves most schema conformance issues
  • Meilisearch + LLM tool calls is an effective pattern for semantic search and linking
(END)q back