Evaluate OpenRouter video generation with traces and benchmarks

OpenRouter routes requests across many models through one API — useful for swapping video endpoints without rewiring your app. The hard part is knowing whether a model change actually improved output quality. Frametail records each call as a trace with model, latency, and artifact references, then lets you compare runs on a fixed dataset.

You keep OpenRouter for inference and model choice. Frametail is the evaluation layer: benchmarks that do not drift, scorers your team agrees on, and experiments that become linkable release artifacts.

The SDK wraps `@openrouter/sdk` for `chat.send` (including streaming) and `videoGeneration` operations — generate, poll jobs, fetch video content, and list video models. Each span carries provider metadata, model id, and sanitized inputs/outputs suitable for debugging beside clips in the dashboard.

Use the same project for OpenRouter and fal if you run multiple providers; benchmarks stay comparable when dataset rows and scorers are pinned.

Enable tracing in staging, generate video through OpenRouter while you iterate on prompts, then pin representative rows and run a benchmark before promoting a new model slug to production. When live traffic regresses, open the trace for that generation — not a generic log line.