RAG-powered learning platform that transforms raw video courses into a searchable, interconnected knowledge graph with real-time AI Q&A and timestamp-cited answers.
Video is the dominant medium for online learning — but it's fundamentally opaque to search. When a 10-hour Udemy course covers a concept you need, the only way to find it is scrubbing a progress bar and hoping you recognise the frame. There's no Ctrl+F for knowledge buried in audio.
I wanted to query a course like a conversation — ask a natural-language question and get a precise answer with the exact video timestamp attached. Not a list of chapters, not a vague "around 3 hours in" — the exact moment, with the surrounding context quoted verbatim.
MindMesh ingests video via yt-dlp, transcribes audio with Whisper, semantically chunks the transcript, embeds it into a vector store, and wraps everything in a LangChain RAG chain that returns answers with timestamp references.
Every chunk retains its start and end timestamp from the Whisper output. When the retriever surfaces a chunk, the answer includes a direct clickable timestamp reference — so users can jump straight to the source moment in the video.
Whisper accuracy dropped noticeably on low-bitrate audio with background music. I added a preprocessing step using FFmpeg to normalise, denoise, and resample to 16kHz mono before transcription.
Hard chunk boundaries caused sentences split across two chunks to both fail retrieval. A 15% token overlap on either side of each boundary eliminated these misses entirely.
I spent a week optimising chunk sizes before building a single UI screen. Prototyping the UX first revealed that users cared more about answer formatting and timestamps than raw retrieval precision.
Loading a 2-hour course from raw audio on every startup took 90 seconds. Serialising the FAISS index to disk after first ingestion cut startup time to under 2 seconds on every subsequent load.
Browse the source code to see the full Whisper → FAISS → LangChain RAG pipeline, or reach out to discuss education AI applications.