A custom AI instrument, trained from a recording

Published October 2026

The instrument catalog covers the orchestra, the band and a shelf of World instruments, and it will never have everything: your grandfather's banjo, a prepared piano, a folk instrument nobody sampled, the player in your band. doseedo_train_instrument takes 30 seconds to 3 minutes of a recording of one instrument and trains a private model of it in about a minute or two. From then on doseedo_render_midi plays any notes you write in that sound, at your tempo, with takes to choose from. Two lighter options sit beside it: a recording's tone applied to one render (timbre transfer), and a recorded line played back by another instrument.

Three ways to bring your own sound

All three are on the same MCP server, with the same key, as every other tool. Connect it once:

claude mcp add --transport http --scope user doseedo https://api.doseedo.com/mcp

Train it: doseedo_train_instrument

Upload the recording with doseedo_get_upload_url (a presigned PUT that returns an audio_key), or pass any https URL, including a separated stem from doseedo_separate_stems. Then name the closest catalog instrument: it picks the instrument model the new one is trained on. A shamisen trains as acoustic_guitar, a whistle as flute, a hand drum as percussion with drum_mode:true.

doseedo_train_instrument {
  "audio_key": "ingest/…/harp_the_swan.wav",
  "instrument": "acoustic_guitar",
  "name": "Concert harp"
}

• The recording. One instrument, solo, as clean and dry as you have it. Up to 8 files of the same instrument (audio_keys / audio_urls). The first 3 minutes are used and at least 8 seconds are required. A model learns the notes it hears, so record the register you plan to write in.
• The notes are transcribed for you. To skip that, pass notes as one list per file, in seconds. That helps on material a transcriber struggles with.
• Training. 300 steps for up to 60 seconds of audio and 400 above, adjustable from 50 to 800 with steps. It runs as an async job: doseedo_wait_task until it is done.

This is a real result from October 22, 2026, trained on 21.9 seconds of a solo harp recording, trimmed to the fields an agent uses:

{
  "status": "completed",
  "kind": "train_instrument",
  "lora_id": "37bf54234a326620",
  "instrument": "acoustic_guitar",
  "seconds_of_audio": 21.91,
  "windows": [
    { "source": "harp_the_swan.wav", "start_s": 0,      "len_s": 10.96, "notes": 73 },
    { "source": "harp_the_swan.wav", "start_s": 10.955, "len_s": 10.96, "notes": 53 }
  ],
  "steps": 300,
  "loss": { "first": 0.5976, "last": 0.4273 },
  "timings": { "total_s": 62.04, "train_s": 57.27 },
  "use": {
    "tool": "doseedo_render_midi",
    "args": { "lora_id": "37bf54234a326620", "instrument": "acoustic_guitar", "lora_strength": 1 }
  }
}

Sixty-two seconds from submit to a usable instrument. The use block is the render call, ready to copy.

Play it: render_midi with lora_id

doseedo_render_midi {
  "instrument": "acoustic_guitar",
  "lora_id": "37bf54234a326620",
  "bpm": 66,
  "takes": 3,
  "notes": [ { "pitch": 67, "start_beats": 0, "duration_beats": 1, "velocity": 70 }, … ]
}

• Pass the instrument it trained as. The adapter belongs to that instrument model. result.lora.applied on the render confirms it played.
• Too strong? A small training set can overshoot. lora_strength goes from 0 to 2 (default 1), and about 0.6 is the usual fix.
• It is yours. Only the account that trained it can render with it. Find it again later with doseedo_list_jobs {kind:"train_instrument"}.
• Everything else still applies: takes, the per-take check against your notes, tempo_map, revise_take_url. In doseedo_render_score a part takes a lora_id too, but only one custom or World instrument per call. The server applies an adapter to the whole batch, so two different ones in one call would play neither.

No training: a recording's tone, or a line re-played

Timbre transfer. timbre_audio_url on doseedo_render_midi is a short recording (up to 24 MB, the first 30 seconds are used) whose sound the render takes on. The notes stay yours and the tone comes from the recording. Use it for a one-off: a reference violin sound, a guitar tone you like, a synth you can't install.

A recorded line, another instrument. doseedo_cover_stem takes a recording of a line (a stem, a solo, humming, a whistled tune, a guitar riff) and plays it back on the instrument you name, straight from the audio. It is the fastest way from "I can sing it" to "here it is on a cello".

What it costs

Training is a flat 20 credits per run, refunded if it fails. Rendering with the trained instrument afterwards is priced like any render: 10 credits per 30-second window, with up to four takes in one call at no extra cost. doseedo_quote prices both before anything runs. It needs no sign-in, and for a training run its whole answer is "flat 20 credits per training run (refunded if it fails)".

Honest limits

• One instrument per model. Training expects one instrument playing alone. For an instrument inside a mix, separate it first (stem separation) and train on the stem.
• It inherits the base model's habits. The adapter is trained on the closest catalog instrument, so choose the one that plays the most like it: plucked with plucked, bowed with bowed, blown with blown.
• Takes are performances. As with any render, check the takes against your notes and listen before you ship.

Related

• Render MIDI with real instruments
• Score a film with an AI agent
• Separate the stem to train on
• Audio to MIDI: transcription over MCP
• The catalog instruments, with demos