---
name: ferret-media
description: Use for sound, video, recording and web pages in a Ferret world: AudioService with four buses, AudioPlayer, AmbientZones, VideoScreen (HLS or MP4 on a plane), HtmlScreen (a page as a texture), MediaPlayer, recording, frame-exact capture to WebM/PNG (captureFrames) and cutaways (SectionView).
---

# Media: audio, video, HTML surfaces, capture, section views

## When to use
Music, effects, ambient sound by zone, a screen in the world that plays a stream or shows a page, a recording of the scene as a video file; a frame-exact clip or picture series; a cutaway.

## The model
`AudioService` has a master gain and four buses (music, sfx, ambient, voice). `AmbientZones` maps zones to ambient sounds. `VideoScreen` plays HLS or MP4 onto a plane (`loadHls` for HLS); `HtmlScreen` draws a web page onto a plane as a texture. Every media component implements `MediaPlayer`. `SceneRecorder` records the engine's canvas (`captureStream` + `MediaRecorder`) to one file: MP4 where the browser writes it, else WebM with its Duration put in; `start`, `pause`, `resume`, `stop()` -> `{ blob, mime, durationMs, bytes, audio }`. The inspector has `record.start`, `record.stop`, `record.state`. `captureFrames` moves the world exactly 1/fps per frame (nothing dropped, loops close) and writes WebM (WebCodecs) or PNGs; inspector `capture`. `SectionView` (package `view`) cuts meshes by 1 to 6 planes on the GPU with hatched caps (`all`/`any`).

## Steps
1. Add `AudioService` to the world; define sounds (`SoundDef`) and play with `PlayOptions` on a bus.
2. Ambient: map zone names to sounds in `AmbientZones` (needs `ZoneService`, see `ferret-interaction-zones`).
3. Video or page screens: add the component to an entity with a plane mesh.
4. Browsers block audio until a user gesture: start it after the first input.
5. Recording: `world.addService(new SceneRecorder())`, then `recorder.start()` and `await recorder.stop()`. The sound joins only when the audio is already unlocked at `start()` (`result.audio`).
6. Clip: `captureFrames(engine, scene, world, { seconds, fps, width, height, supersample, beforeFrame })`. Cutaway: `new SectionView(scene, { targets, planes, mode, frame })`, toggle `active`.

## Check afterwards
Each bus can be muted separately; leaving a zone fades its ambient sound; the video plane shows frames (not black); a recording plays back and shows the scene; a capture has `seconds * fps` frames; a section shows hatched caps.

## What usually goes wrong
- Audio never starts: no user gesture yet.
- HLS on a browser without native support: pass the hls.js loader as the API describes.
- A recording is empty or has no sound: software rendering and encoding (the simulators, a headless browser) deliver the first data about a second after the start, so record for a few seconds; a locked audio context delivers nothing, so a recording started before the first user action has no sound, and the browser may not play an MP4 with AAC back (check the file from outside). Playwright's WebKit has no `captureStream`.
- HTML pages that forbid framing or fetching: `HtmlScreen` draws a fetched page, not a live frame.
- Capture: no codec for the size -> `format: 'png'`; code timed by `performance.now()` runs at wall-clock speed. Section: no caps on open meshes; shadows come from whole meshes.

## Files

- `reference/packages.md` — Characters, animation graphs, NPC navigation, interactables, zones and transformations.
- `reference/api-media.md` — every export of `@ferret/media` with signature and doc comment
- `reference/api-view.md` — every export of `@ferret/view` with signature and doc comment