build.unfiltered

From the reel · 7 November 2026

Earshot: set it up so Claude can read sound

Claude cannot hear. Earshot turns sound into text that Claude can read. This is the setup guide from the SOUND reel, and what we found when we tested it.

Claude reads text and sees pictures. It cannot take in audio. So when Claude edits a video, it cannot hear the sound effects it puts in.

Earshot does not give Claude ears. It turns sound into text: what a sound is, how loud it is and when it starts. Claude reads that text. Earshot also has a bank of 252 sound effects, all made in code, and timing rules measured on professional launch films.

Earshot is free and open source (MIT licence). It runs on your computer, on the CPU. It needs no GPU and no account. This guide uses version 0.4.0, checked on 6 November 2026.

What you need

ItemWhy
Python 3.10 or laterEarshot is a Python program.
ffmpeg and ffprobeEarshot uses them to read audio and video files. They must be on your PATH.
About 1.1 GB of free diskEarshot downloads two open models the first time you use it.
Claude CodeOnly for step 3. You can use Earshot without it, from the terminal.

Step 1. Install ffmpeg

  1. On a Mac, open Terminal and type brew install ffmpeg. This needs Homebrew.
  2. On Linux, use your package manager, for example sudo apt install ffmpeg.
  3. Type ffmpeg -version. If you see a version number, ffmpeg is installed.

Step 2. Install Earshot

  1. Get the code: git clone https://github.com/Leonxlnx/earshot && cd earshot
  2. Make a separate Python environment: python3 -m venv .venv
  3. Install PyTorch for the CPU: .venv/bin/python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
  4. Install Earshot with the MCP server: .venv/bin/python -m pip install -e ".[mcp,llm,dev]"
  5. Download the models and test the install: .venv/bin/python -m earshot doctor --load. This step downloads about 1.1 GB. Do it once.
  6. On Windows, use .venv/Scripts/python in place of .venv/bin/python.

Step 3. Connect Earshot to Claude Code

  1. Find the full path of your Earshot folder. In that folder, type pwd.
  2. Type this command. Replace /path/to/earshot with that path: claude mcp add earshot --scope user -e EARSHOT_THREADS=3 -- /path/to/earshot/.venv/bin/python -m earshot mcp
  3. Start Claude Code again. Type /mcp. Make sure that earshot is in the list.
  4. Optional: copy the two skills from the skills/ folder of the repo into ~/.claude/skills/. They tell Claude which Earshot command answers which question.

Step 4. Use it

In Claude Code, ask in plain words. For example: "Use earshot to make a timeline of reel.mp4 and tell me where each sound effect starts." From the terminal, use the commands below. Put .venv/bin/python -m in front of each one if earshot is not on your PATH.

The commands we used most

CommandWhat it gives you
earshot timeline reel.mp4A list of the sounds in the file, with the time, the level and a name for each.
earshot critique reel.mp4A score against 7 professional launch films, with fixes.
earshot sfx list --kind whooshThe whooshes in the bank of 252, with their ids.
earshot sfx render whoosh_fast/crisp -o w.wav --dur 0.8 --apex 0.32One sound from the bank, cut to your length, with its loudest point where you want it.
earshot kit init --preset launch -o kit.jsonYour own sound kit: one sound for each job (pop, whoosh, hit, tick). Use it again on every reel.
earshot kit render kit.json --out kitMakes the kit's sound files and measures where each one starts and peaks.

What we found when we tested it

We tested Earshot on a real reel: 78 seconds, 57 sound effects, all made in code and each tied to a word. Then we made a second version with Earshot's sounds and timing. Same picture, same voice, same music.

TestWhat happened
Timeline of the finished reelIt named 20 sounds. 9 of them were the voice. It called every section "inspirational corporate background music".
Timeline of the effects aloneIt named the pops ("bubble pop") and the thuds ("kick drum"). It did not name the whooshes, the snaps or the ticks.
Version B, with the bank and the launch timingThe effects sat 2 dB clearer over the voice (median). Effects more than 15 dB under the voice went from 18 to 12.
Critique of both versionsBoth got 20 of 23, with the same three tips. The score did not see the change.

When it does not work

ProblemWhat to do
"ffmpeg not found"Install ffmpeg (step 1). Then open a new terminal.
The install or the models failType .venv/bin/python -m earshot doctor --load. It says what is missing.
Claude Code does not show earshotCheck the path in the claude mcp add command. It must point to .venv/bin/python inside the Earshot folder.
The first call times outStart Claude Code with MCP_TIMEOUT=60000 (step 3).
Each terminal command takes 20 to 40 secondsThat is normal: each command loads the models again. For many calls, use the MCP server, where the models stay loaded. sfx, kit and measure load no models.

Sources

All posts · build.unfiltered