ScrapeMole
Home / Guides / YouTube transcripts for RAG with timestamp citations

YouTube transcripts for RAG: chunk by timestamp and cite the video moment (Python)

Feed YouTube transcripts to an LLM the RAG way: fetch timestamped segments in one HTTP call, chunk them with start times, retrieve, and cite answers as links to the exact moment. Python with real output.

Open the YouTube Transcript Scraper →By ScrapeMole · 26 October 2026

Disclosure: I'm the maker of the YouTube Transcript Scraper on the Apify Store and the ScrapeMole x402 API used below. Other transcript tools exist; this guide shows how to use ours, with real output.

Dumping a whole YouTube transcript into an LLM works for one short video. It stops working when you have a channel's worth of videos, long talks, or you want answers that point to where in the video something was said. For that you want retrieval-augmented generation (RAG): split transcripts into chunks, index them, retrieve the few relevant chunks per question, and let the model answer with citations.

The trick for video is to chunk by timestamped caption segments instead of by raw characters. Every chunk then keeps a start time, so every citation can be a clickable youtube.com/watch?v=...&t=638s link that jumps to the moment.

If you only need the transcript text (curl, n8n, simple summaries), start with YouTube transcripts from Python, curl and n8n. This guide is the next step: chunks, timestamps, citations and ingestion into a vector store.

1. Get transcripts with timestamps (one HTTP call)

The Actor's run-sync-get-dataset-items endpoint runs it and returns the dataset rows in the same response. Only requests is needed:

# pip install requests
import os, json, requests

ACTOR = "scrapemole~youtube-transcript-scraper"
URL = f"https://api.apify.com/v2/acts/{ACTOR}/run-sync-get-dataset-items"

def fetch_transcripts(videos, languages=("en",)):
    r = requests.post(
        URL,
        headers={"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
        json={"videos": videos, "languages": list(languages)},
        timeout=300,
    )
    r.raise_for_status()
    return r.json()  # one row per video

A real row for 3Blue1Brown's "But what is a neural network?" (run on 26 October 2026), trimmed to the fields used here:

{
  "videoId": "aircAruvnKk",
  "status": "ok",
  "title": "But what is a neural network? | Deep learning chapter 1",
  "channelName": "3Blue1Brown",
  "language": "en",
  "isAutoGenerated": false,
  "segmentCount": 286,
  "wordCount": 3357,
  "segments": [
    { "start": 4.22, "duration": 1.18, "end": 5.4, "text": "This is a 3." },
    { "start": 6.06, "duration": 4.653, "end": 10.713, "text": "It's sloppily written and rendered at an extremely low resolution of 28x28 pixels," }
  ]
}

The row also carries transcriptText, url, channelId, durationSeconds, publishDate, viewCount, description and availableLanguages, which are useful as metadata filters later. Videos without captions come back with status: "error" and an errorCode (not charged), so always check status.

2. Chunk by segments, keep the timestamps

Caption segments are short (a line or two), so group them until a chunk reaches about 200 words, and carry one segment over into the next chunk so a sentence split at the boundary is still findable:

def chunk_segments(item, max_words=200, overlap_segments=1):
    """Group timestamped segments into ~max_words chunks, keeping start/end times."""
    chunks, cur, words, fresh = [], [], 0, 0
    for seg in item["segments"]:
        cur.append(seg)
        fresh += 1
        words += len(seg["text"].split())
        if words >= max_words:
            chunks.append(cur)
            cur = cur[-overlap_segments:] if overlap_segments else []
            words = sum(len(s["text"].split()) for s in cur)
            fresh = 0
    if fresh:  # leftover segments that aren't in a chunk yet
        chunks.append(cur)
    out = []
    for i, c in enumerate(chunks):
        start = int(c[0]["start"])
        out.append({
            "id": f'{item["videoId"]}-{i:04d}',
            "text": " ".join(s["text"] for s in c),
            "videoId": item["videoId"],
            "title": item["title"],
            "channel": item["channelName"],
            "start": c[0]["start"],
            "end": c[-1]["end"],
            "url": f'https://www.youtube.com/watch?v={item["videoId"]}&t={start}s',
        })
    return out

def mmss(sec):
    sec = int(sec)
    return f"{sec // 60}:{sec % 60:02d}"

rows = fetch_transcripts(["https://www.youtube.com/watch?v=aircAruvnKk"])
chunks = []
for item in rows:
    if item["status"] != "ok":
        print("skip", item["videoId"], item.get("errorCode"))
        continue
    chunks += chunk_segments(item)

with open("chunks.jsonl", "w") as f:
    for c in chunks:
        f.write(json.dumps(c) + "\n")

Real output for that 18-minute video:

18 chunks
aircAruvnKk-0000 0:04 - 1:10 https://www.youtube.com/watch?v=aircAruvnKk&t=4s
   This is a 3. It's sloppily written and rendered at an extremely low resolution of 28x28 pixels, but your brain has no tr ...
aircAruvnKk-0001 1:07 - 2:03 https://www.youtube.com/watch?v=aircAruvnKk&t=67s
   Unless you've been living under a rock, I think I hardly need to motivate the relevance and importance of machine learni ...

Notice chunk 1 starts at 1:07, before chunk 0 ends at 1:10: that is the one-segment overlap. Tune max_words to your embedding model: 150 to 300 words is a good range for most, and a chunk of about a minute of speech is also a nice unit to cite.

3. Retrieve and answer with citations

Each chunk is a self-contained record (text, video, start time, deep link), so retrieval and citations need nothing else. Here is a dependency-free version you can run as is; it uses a tiny keyword scorer so the example works without an API key:

import json, math, re
from collections import Counter

chunks = [json.loads(line) for line in open("chunks.jsonl")]

def tokens(t):
    return re.findall(r"[a-z0-9]+", t.lower())

# Tiny BM25-style scorer so the example runs with no extra installs.
# Swap in embeddings + a vector DB for real use (see below).
N = len(chunks)
df = Counter(w for c in chunks for w in set(tokens(c["text"])))
def score(query, c):
    tf = Counter(tokens(c["text"]))
    return sum(tf[w] * math.log(1 + N / df[w]) for w in tokens(query) if w in tf)

question = "What does the bias do in a neuron?"
top = sorted(chunks, key=lambda c: score(question, c), reverse=True)[:3]

context = "\n\n".join(
    f'[{i+1}] {c["title"]} @ {mmss(c["start"])} ({c["url"]})\n{c["text"]}'
    for i, c in enumerate(top)
)
prompt = (
    "Answer the question using only the sources below. "
    "Cite sources as [n] after each claim.\n\n"
    f"{context}\n\nQuestion: {question}"
)
# send `prompt` to the LLM of your choice

The three chunks it picked for that question, from the real transcript:

[1] 10:38 https://www.youtube.com/watch?v=aircAruvnKk&t=638s
[2] 12:54 https://www.youtube.com/watch?v=aircAruvnKk&t=774s
[3] 13:58 https://www.youtube.com/watch?v=aircAruvnKk&t=838s

Source [1] is the part where the video introduces the bias ("Maybe you only want it to be active when the sum is bigger than say 10"), so the model's answer can cite [1] and your UI can render it as a link that opens the video at 10:38. Because the prompt only contains numbered sources with times, it's easy to check that every claim maps to a real moment in a real video.

4. Ingest into a vector store

For more than a handful of videos, replace the keyword scorer with embeddings. The chunk records map directly onto any vector database: id as the document ID, text as the content to embed, and everything else as metadata. In pseudo-code that fits Chroma, pgvector, Qdrant, LanceDB or a LangChain/LlamaIndex loader:

for c in chunks:
    store.upsert(
        id=c["id"],
        text=c["text"],             # what gets embedded
        metadata={k: c[k] for k in ("videoId", "title", "channel", "start", "end", "url")},
    )

A few things that pay off in practice:

  • Stable IDs (videoId plus chunk number) make re-ingestion idempotent: re-run a channel and you upsert, not duplicate.
  • Filter on metadata (channel, videoId, or publishDate from the row) before vector search when users ask about one channel or a recent period.
  • Store the video title in the chunk text as well if your questions often name the video ("in the neural network video, ..."); embeddings then match on it.
  • Summaries as a second layer: for long videos, summarize each chunk once with any LLM and index the summaries next to the raw chunks. Questions like "what is this video about" hit the summaries; detailed questions hit the raw text. Keep the same start and url so summaries stay citable.
  • Many videos at once: pass a list of URLs in videos. To get every video of a channel first, use the YouTube Channel Scraper, then feed its URLs here.

Alternative for AI agents: pay per call with x402 (no account)

If the thing fetching transcripts is an AI agent rather than your own script, there's also a pay-per-call HTTP API with no signup and no API key. It costs $0.002 per transcript in USDC on Base via x402, charged only on a 200 response:

curl -i "https://scrapemole-api-mainnet.scrapemole.workers.dev/v1/youtube/transcript?video=aircAruvnKk&lang=en"
# -> HTTP 402 with the payment requirements; an x402 client pays and retries automatically

The paid response has the same videoId, title, channelName and segments (start, duration, end, text) fields, so chunk_segments() above works on it unchanged. Agents that speak MCP can use the remote MCP server at https://scrapemole-api-mainnet.scrapemole.workers.dev/mcp (Streamable HTTP); its youtube_transcript tool returns the same data and is paid through the x402 MCP transport. Docs: openapi.json and llms.txt.

What it costs

On Apify you pay per transcript delivered: $0.004 on the free plan, down to $0.002 on higher plans, so a 1,000-video knowledge base is $2 to $4 of transcripts. Videos without captions, private or failed videos aren't charged. Through x402 it's a flat $0.002 per transcript. Embeddings and the LLM are whatever you already use.

Questions or a video that doesn't work? Open an issue on the Actor page or email us and I'll look at it.