Score a film with an AI agent

Published October 2026

An agent can watch a cut, write a cue sheet, and compose and orchestrate a cue. It cannot make an orchestra play, do drop-frame timecode math reliably by hand, or hand an editor a session. Those are the steps Doseedo's MCP tools cover: doseedo_tempo_map turns hit points into a tempo map that lands each one on a downbeat, doseedo_render_score plays every part on its instrument model and returns a stem per part, a mix and a receipt saying whether each part played its notes, doseedo_generate_sound makes the hits that aren't notes, and doseedo_session_from_stems builds a Logic Pro or Ableton Live project. ffmpeg does the last mile. This is the whole loop, with real numbers.

Who does what

claude mcp add --transport http --scope user doseedo https://api.doseedo.com/mcp

The server publishes this sequence itself: doseedo_recipes {name:"score-a-film"} returns it to any agent, step by step.

1. Spot the picture

The agent works from the cut. Three commands give it what a music editor would mark up:

# cuts: every scene change with its time
ffmpeg -i film.mp4 -vf "select=gt(scene\,0.3),showinfo" -f null - 2>&1 | grep pts_time

# a frame every two seconds, for the agent to look at (mood, action, light)
mkdir -p frames && ffmpeg -i film.mp4 -vf fps=1/2,scale=640:-1 frames/%04d.jpg

# where the dialogue is: spans of sound vs. silence on the dialogue track
ffmpeg -i dialogue.wav -af silencedetect=noise=-35dB:d=0.5 -f null - 2>&1 | grep silence_

Without a clean dialogue track, doseedo_separate_stems on the production audio returns a vocals stem to run the last command on. The result is a cue sheet, the document every later step reads:

2. Land the hits: doseedo_tempo_map

A composer scoring to picture bends the tempo so a bar line falls on each hit: between two hits Δt seconds apart, a whole number of bars at bpm = bars × beats per bar × 60 / Δt, picked to stay nearest the cue's tempo. The tool does exactly that, in timecode, at the film's frame rate, including drop-frame (;) at 29.97 and 59.94:

doseedo_tempo_map {
  "bpm": 96,
  "fps": 24,
  "start_timecode": "01:02:10:00",
  "hits": [
    { "timecode": "01:02:17:18", "label": "door slam" },
    { "timecode": "01:02:27:12", "label": "she turns" },
    { "timecode": "01:02:40:12", "label": "title card" }
  ]
}

Its answer for that cue sheet, trimmed:

{
  "tempo_map": [
    { "beat": 0,  "bpm": 92.9032 },
    { "beat": 12, "bpm": 98.4615 },
    { "beat": 28, "bpm": 92.3077 },
    { "beat": 48, "bpm": 96 }
  ],
  "hits": [
    { "label": "door slam",  "time_s": 7.75, "beat": 12, "bar": 4,  "beat_in_bar": 1 },
    { "label": "she turns",  "time_s": 17.5, "beat": 28, "bar": 8,  "beat_in_bar": 1 },
    { "label": "title card", "time_s": 30.5, "beat": 48, "bar": 13, "beat_in_bar": 1 }
  ],
  "bpm_range": [ 84.48, 107.52 ],
  "max_tempo_deviation_bpm": 3.69,
  "timing_error_ms": 0.006,
  "cue_start": { "seconds": 3730, "timecode": "01:02:10:00" }
}

Every hit is on beat 1 of a bar, and the tempo never moves more than 3.7 bpm from 96. The default range is ±12%, and bpm_range widens or narrows it. When no whole number of bars fits between two hits, the hit lands on a beat instead and notes says so. When nothing fits at all, the tool refuses and says to widen the range or play the hit off the grid. It is pure math: free, instant, no key.

3. Write the parts on that grid

Now the agent composes, in beats from bar 1 on the grid it was just given: the accent, the stinger or the change goes on hits[].beat. doseedo_instruments lists what can play, each with its range in concert pitch (a horn is 41–77, a cello 36–81) and its exact call. The parts can be note rows the agent writes, a multi-track MIDI file from pretty_midi or music21, or a MusicXML export from Dorico, Sibelius, Finale or MuseScore, in which case dynamics, hairpins, staccato and accents, pizz. and con sord. are performed and transposing parts are read at concert pitch.

4. Play every part: doseedo_render_score

doseedo_render_score {
  "bpm": 96,
  "tempo_map": [ { "beat": 0, "bpm": 92.9032 }, { "beat": 12, "bpm": 98.4615 },
                 { "beat": 28, "bpm": 92.3077 }, { "beat": 48, "bpm": 96 } ],
  "parts": [
    { "name": "Timpani", "instrument": "vibraphone", "timbre_preset": "timpani:preset1",
      "notes": [ { "pitch": 43, "start_beats": 12, "duration_beats": 2, "velocity": 118 },
                 { "pitch": 38, "start_beats": 48, "duration_beats": 4, "velocity": 124 } ] },
    { "name": "Horn",     "instrument": "french_horn",    "notes": [ … ] },
    { "name": "Violin I", "instrument": "violin",         "notes": [ … ] },
    { "name": "Cello",    "instrument": "cello",          "notes": [ … ] },
    { "name": "Piano",    "instrument": "acoustic_piano", "notes": [ … ] }
  ]
}

Before anything renders, the submit reply's prepared block says which instrument plays each part and why (resolved_by: the part's own call, an override, the written name, or the General MIDI program), so a wrong match is caught early. For this call:

"prepared": {
  "parts": [
    { "name": "Timpani", "instrument": "vibraphone", "timbre_preset": "timpani:preset1",
      "catalog_id": "timpani", "resolved_by": "part", "notes": 2, "range": [ 38, 43 ] },
    { "name": "Horn", "instrument": "french_horn", "catalog_id": "french_horn",
      "resolved_by": "part", "notes": 2, "range": [ 55, 60 ] },
    …
  ],
  "duration_s": 42.5, "bpm": 96, "time_signature": "4/4", "tempo_changes": 4
}

The render takes one to four minutes. The result has a stem per part and a mix, as pre-signed links (24 hours, plain curl -OJ):

"parts": [
  { "index": 0, "name": "Timpani", "instrument": "vibraphone", "url": "https://…",
    "fit": { "verdict": "pass", "coverage": …, "med_offset_ms": … }, "timbre": { … }, "level": { … } },
  …
],
"mix_url": "https://…"

• The fit receipt. coverage is the share of a part's written attacks that were heard and med_offset_ms is its timing. A part that fails is re-rendered once automatically. If one still doesn't pass, check names it: listen, then re-render that part alone with doseedo_render_midi, or fix the bar with revise_take_url.
• Limits per call. Up to 8 stems (a chord on a one-line instrument is split into voices, each a stem). At most one World or custom instrument (the timpani here). No synth or electronic one-shot parts, which go through doseedo_render_midi. Up to 600 seconds. For a bigger orchestra, render it in calls by section (strings, then brass and winds) with the same tempo map. Every call starts at bar 1, so the stems line up.
• Cost. 10 credits per 30-second window for each group of up to six parts. This 42.5-second cue with five parts is two windows: 20 credits. doseedo_quote prices it before it runs.

5. The hits that are not notes: doseedo_generate_sound

doseedo_generate_sound { "prompt": "heavy wooden door slamming shut in a stone hallway", "kind": "hit" }

About 20 seconds later it is a 2-second WAV, 10 credits (the sound-effects guide has the real reply). It goes on the picture at the hit's time: 7.75 seconds into this cue.

6. The session: doseedo_session_from_stems

doseedo_session_from_stems { "from_job": "<the render_score job_id>", "target": "logic" }

from_job reads everything from the score render: each part's audio on its own track from bar 1, the written parts as MIDI tracks beside them, and the tempo map. In 30 to 90 seconds it returns a zipped .logicx (or .als with target:"ableton") that a composer or music editor opens and keeps working in. The session build costs no credits. For a cue rendered in several calls, pass stems: [{name, audio_url}, …] from all of them (up to 24), the midi_tracks and the same tempo_map.

7. On the picture: duck, normalize, mux

The last mile is ffmpeg on your machine. This places the cue and the door slam, ducks the music under the dialogue, normalizes to −16 LUFS for the web and muxes it onto the video. If the film file starts at timecode 01:00:00:00, cue 1M1 (01:02:10:00) is 130 seconds in, and the door slam is 7.75 seconds after that:

ffmpeg -i film.mp4 -i mix.wav -i door_slam.wav -filter_complex "
  [1:a]adelay=130000:all=1[m];
  [2:a]adelay=137750:all=1[sfx];
  [0:a]asplit=2[dlg][key];
  [m][key]sidechaincompress=threshold=0.03:ratio=8:attack=20:release=400[duck];
  [dlg][duck][sfx]amix=inputs=3:duration=first:normalize=0,
    loudnorm=I=-16:TP=-1.5:LRA=11[a]" \
  -map 0:v -map "[a]" -c:v copy -c:a aac -b:a 192k film_scored.mp4

• Ducking keys the music off the film's own audio. On a test with dialogue at about −24 dBFS RMS, the music came down 9.4 dB under the dialogue and back up after it. Raise threshold if it ducks under room tone.
• Loudness. −16 LUFS is a common web target. Broadcast delivery is −23 LUFS (EBU R128) or −24 (ATSC A/85). Single-pass loudnorm is approximate (the test above landed at −18.3), so for delivery run it twice: once with print_format=json to measure, then with the measured values.
• Stems for the mix stage. A re-recording mixer wants the stems or the session from step 6, not a mixdown.

Honest limits

• The agent writes the music. Doseedo plays it, times it and delivers it. The themes, the harmony and the dramatic choices are the agent's, or yours.
• Timing is a performance. Measured on production renders on October 22, 2026, note onsets land within about ±30 ms of the score, with a median of about 5 ms: about a frame at 24 fps. Tight for a cue, not sample-accurate.
• Read the receipts and listen. A fit verdict is a measurement by a second model, and a pass is not a creative judgement.
• Instrumental only. No sung or spoken lines.

Related

• Render MIDI with real instruments
• Sound effects and foley over MCP
• A custom AI instrument from a recording
• Separate the dialogue from a mixed track
• Logic Pro MCP server