Custom chat UI
Run the assistant from your own chat interface.
You can run the assistant from your own chat window instead of the editor's panel. Your window sends the messages, and the changes still appear on the editor's canvas.
Sending messages
Hide the built-in panel, and pass onChatEvent. The first event is ready, and it carries api, the handle of the live canvas. Pass the handle to runChatTurn for each message:
import { useRef, useState } from 'react';
import {
VideoEditor,
runChatTurn,
withChatEvents,
type ChatEvent,
type ChatUiMessage,
type EditorVerbApi,
} from '@layermetry/media-editor/react';
const MEDIA_API_URL = 'https://media.example.com';
const SESSION_ID = 'campaign-42';
function textOf(message: ChatUiMessage): string {
return message.parts.map((part) => (part.type === 'text' ? part.text : '')).join('');
}
export default function App() {
const api = useRef<EditorVerbApi | null>(null);
const [messages, setMessages] = useState<ChatUiMessage[]>([]);
const [draft, setDraft] = useState('');
function handleChatEvent(event: ChatEvent) {
// The first event hands you the live canvas.
if (event.type === 'ready') api.current = event.api;
if (event.type === 'receipt') {
console.log(`${event.toolName}: ${event.result.ok ? 'done' : event.result.error}`);
}
}
async function send() {
if (!api.current || draft.trim() === '') return;
const turnId = `turn-${Date.now()}`;
const userMessage: ChatUiMessage = { id: turnId, role: 'user', parts: [{ type: 'text', text: draft }] };
setDraft('');
const history = await runChatTurn(messages, userMessage, {
endpoint: `${MEDIA_API_URL}/video-editor-chat`,
sessionId: SESSION_ID,
toolContracts: api.current.toolContracts,
executeTool: withChatEvents(api.current.executeTool, { sessionId: SESSION_ID, turnId, emit: handleChatEvent }),
onHistoryChange: setMessages,
});
setMessages(history);
}
return (
<div style={{ position: 'fixed', inset: 0, display: 'flex' }}>
<div style={{ flex: 1, minWidth: 0 }}>
<VideoEditor
licenseKey="YOUR_LICENSE_KEY"
onClose={() => console.log('The editor was closed')}
mediaApiUrl={MEDIA_API_URL}
// Hide the built-in panel. Your own chat replaces it.
ui={{ features: { 'ai.chat': false } }}
onChatEvent={handleChatEvent}
/>
</div>
<aside style={{ width: 320, padding: 16, borderLeft: '1px solid #ddd', overflowY: 'auto' }}>
{messages.map((message) => (
<p key={message.id}>
<b>{message.role}:</b> {textOf(message)}
</p>
))}
<input value={draft} onChange={(event) => setDraft(event.target.value)} placeholder="Ask for a change" />
<button onClick={send}>Send</button>
</aside>
</div>
);
}runChatTurn(history, userMessage, options) sends the conversation to the chat service, runs the actions that the service asks for on the canvas, and returns the whole conversation after the turn:
| Option | Description |
|---|---|
endpoint | The chat service: {mediaApiUrl}/video-editor-chat. |
toolContracts | The actions that the service may ask for: api.toolContracts. |
executeTool | Runs one action: api.executeTool, wrapped with withChatEvents to report events. |
sessionId | The name of the conversation. Pass it on every turn. |
templateMetadata | Your brand details, the same object as the prop. Pass it on every turn. |
activeBrandId | The id of the brand kit to send when templateMetadata.brands is a list. Without it, or when the kit is not listed, the editor uses templateMetadata.activeBrandId, then the first kit. |
onHistoryChange | Called with all messages each time the conversation changes. |
onAssistantUpdate | Called with the assistant message while it grows. |
signal | An AbortSignal that stops the turn. |
Pass sessionId and templateMetadata on every turn. When templateMetadata.brands is a list, the first round of each turn sends one flat kit. See Brand kits. The chat service keeps what it learns about your brand under the conversation's name. Without them, each turn starts with no brand and no memory.
A message is { id, role, parts }. A user message has one part: { type: 'text', text }.
Streaming responses
The chat service streams its answer. runChatTurn builds the assistant message as the parts arrive, and calls onAssistantUpdate and onHistoryChange, so your window can show the words while they come.
onChatEvent receives five kinds of event:
type | When | Fields |
|---|---|---|
ready | The canvas can be edited. | sessionId, api |
message | The conversation changed. | turnId, message, messages |
toolCall | An action is about to run. | turnId, toolName, args |
receipt | An action has run. | turnId, toolName, result |
done | The turn ended. | turnId, messages, and error if it failed. A refused turn also has errorCode, and topUpUrl when the server sent one. |
Turns typed in the editor's own panel arrive at onChatEvent by themselves, as message, toolCall, receipt, and done events. Turns that you run arrive there only through withChatEvents, as in the example, and only as toolCall and receipt events: a toolCall before each action, and a receipt when the action returns a result. The editor never sends message or done for a turn that you run. To follow such a turn, use onHistoryChange and onAssistantUpdate, and store the list that onHistoryChange gives you. A turn that fails or is stopped does not return, but the actions it already ran are in that list.
Store messages, not the events. A receipt carries everything that one action changed, and can be large.
Tool results
Each action returns a result, which receipt events carry. ok says whether the action ran. On failure, error is a sentence that says why, and the chat service passes it to the model, so the model can try again.
{ "ok": true, "receipt": { "summary": "Added 1 text item." } }On success, receipt describes the change: a one-line summary, and the items and fields that changed, with their values before and after. A turn stops after 8 rounds of actions.
To run actions yourself, without the chat service, call api.execute(name, args). See Editor API.