Tutorial — AI Completions

This app supports AI-driven completions and comments on sheet music. The core feature uses an instruction-following model (Magenta LLM) to generate suggestions, annotations, and musical completions based on prompts you provide.

Core: Comment-driven Generation

The primary workflow is: add a comment anywhere on the score (a measure, beat, or individual note), write an instruction in the comment, and ask the model to generate music or annotations that follow that instruction.

How It Works

In the Studio, place a comment on a note, measure, or range using the comment tool.
Write a concise instruction in the comment box. Example: "Continue with a more dynamic melody for 8 bars, keep it playable by intermediate pianists."
Open the AI Completions action for that comment (context menu or Completions panel) and choose "Generate".
The Magenta LLM receives the selection context + your instruction and returns suggested musical material or textual annotations.
Review results in the preview pane; accept to apply generated notes/annotations to the score or insert as an editable suggestion.

Example Comment Prompts

"Continue with more dynamic melody from here for 8 bars, add subtle syncopation."
"Expand this motif into a 4-bar call-and-response passage in a jazz idiom."
"Add an accompanying arpeggiated left hand supporting this line, moderate difficulty."
"Rewrite this phrase to emphasize the off-beats and add a brief drum pattern suggestion."
"Add performance instruction: light staccato on beats 2 and 4 and a crescendo over the next 4 bars."

Button Reference

Common buttons in the Studio UI and what they do (icon shown first):

Cursor / Select: Switch to selection tool to pick notes or ranges.
Undo: Revert the last change.
Redo: Re-apply an undone change.
Play: Start playback from the current position.
Pause: Pause playback.
Add Comment Here: Place a comment at the current cursor/selection (use for AI-driven instructions).
Note buttons: Insert notes of specific durations.
Rest buttons: Insert rests of corresponding durations.
Add Track: Create a new instrument/track in the score.
Mute (M): Mute the selected track.
Solo (S): Solo the selected track (mute others).
AI Transform: Open AI transform actions for the current selection.
LLM + Magenta Continue: Send the current selection/comment to the Magenta LLM to continue or generate material.
Prompt Lab: Open the prompt testing and tuning UI for AI requests (no icon).
Export MIDI: Export the current score as a MIDI file.
Save Project JSON: Save the project state to JSON.
File input: Import a file (`.json`, `.mid`, `.midi`, `.gp3/.gp4/.gp5`).
Insert Active: Insert the currently selected tool into the score at the specified position.
Zoom: Adjust sheet zoom level.
Theme: Toggle or select sheet visual theme/colors.

Model Behavior & Settings

The Magenta LLM uses the current score context (nearby measures, key, tempo) plus your comment to generate outputs. Explicit constraints improve accuracy.
Adjust temperature / creativity in the AI panel: lower for conservative edits, higher for novel variations.
Output formats: the model can return structured JSON with notes and metadata or plain-text suggestions depending on the requested format in the comment (e.g., "Return as JSON: notes, durations").

Apply & Edit Generated Material

Generated notes appear as a reversible edit suggestion; you can accept, modify, or reject them.
When accepted, edits are inserted into the score and saved as part of the revision history (if enabled).

Best Practices

Be specific: include length (bars), style (jazz/pop/classical), instrument, and difficulty.
Limit requests to small sections for faster and more accurate results.
When you want multiple variants, ask the model: "Generate 2 alternatives."

Privacy & Service Notes

AI requests may be sent to a remote Magenta LLM service—check your app's settings for the model endpoint.
Large requests may take longer and consume more tokens; the panel exposes token/timeout settings.

Need This Wired Up?

If you want, I can implement the comment → AI pipeline UI (context menu, panel, and apply flow) or wire the front-end to a local Magenta endpoint. Which would you prefer?

Back to Studio