FERRETGame engine over Babylon.js
Ferret

VR (WebXR)

A Ferret game is played in a headset through WebXR (Meta Quest Browser first). The game does not change to get VR: the same levels, behaviours, actions and UI serve the screen and the headset. The game opts in with one option and decides, as for every other input, what each controller button does.

On OS Beroun nothing is loaded: the @ferret/xr package is fetched by import() only when xr is set and the browser has navigator.xr.

Turning it on#

const game = createGame({ canvas, input: { actions, bindings }, xr: { enabled: true } });

XrService (a world service named xr) then offers supported, enter(), exit(), active, state (off, starting, visible, blurred) and the events xr:enter, xr:exit, xr:state. A session must start from a click, so a page needs a button:

import { addEnterVrButton } from './_shared/vr';   // the samples' button; shown only where a headset session can start
await addEnterVrButton(game);

The showcase and the viewer also start with ?xr in the address. Features (hand-tracking, layers, local-floor, bounded-floor) are asked for as optional; xr.status reports what the session got, and a game never assumes one.

blurred (the headset's system menu is up, or the headset is off the head) pauses the world the way Platform.onSuspend does, and resumes it on visible.

If the game page sits in an iframe, the frame needs allow="xr-spatial-tracking".

Head and body#

In a session the view is the headset. The character's capsule stays the player: XrRig puts its origin at the capsule's feet, and room-scale steps move the capsule through CharacterMovement, so a player cannot walk through a wall. If the head goes into geometry the view fades to black until it comes out; the rig never pushes the view back.

  • Standing play uses the real head height; seated play (a setting) raises the rig so the eyes are at the character's eye height. The world scale is 1:1.
  • "Forward" follows the head's yaw, or the left controller's (a setting).
  • First person only in a session: the character's body is hidden and hands are shown (controller models, or tracked hands). With xr: { body: 'arms', bodyRig } the character's arms follow the controllers by two-bone IK.
  • Camera shake becomes a controller pulse (xr.pulse(hand, strength, ms)), a scripted camera move becomes a blink (fade, move, fade), a cutscene camera plays on a virtual screen in front of the player. A game can replace each.

Input: the xr kind#

Three new binding types, next to key, button and axisKeys:

const bindings = {
  xr: {
    move:     [{ xrStick: 'left' }],
    turn:     [{ xrStick: 'right' }],
    jump:     [{ xrButton: 'a', label: 'A' }],
    use:      [{ xrButton: 'trigger', label: 'Trigger' }, { xrPinch: 'right', label: 'Pinch' }],
    teleport: [{ xrButton: 'b', label: 'B' }],
    menu:     [{ xrButton: 'y', hand: 'left', label: 'Y' }, { xrButton: 'menu' }],
  },
};

xrButton is trigger, squeeze, thumbstick, a, b, x, y or menu; with no hand it means the player's dominant hand (a setting). xrPinch is hand tracking: thumb to index finger. input.kind becomes 'xr' on xr:enter and returns to the previous kind on xr:exit; prompt(action) gives the controller's label ("Trigger", "A"). Ferret ships no mapping of its own: the samples' examples/_shared/vr.ts is one example, each game picks its own. The same rule as on OS Beroun holds for the console kind; the headset is a different device with its own buttons.

input.pose('head' | 'left' | 'right') gives position, rotation, velocities and tracked for the current fixed step.

Moving and turning#

The game's move action works as it does today, relative to the forward above. Turning reads the game's turn action (change the names with xr: { actions: { turn, teleport } }):

  • snap turn (30, 45 or 90 degrees, default 45) or smooth turn (speed setting). Pitch from sticks is ignored: the head owns pitch.
  • teleport (setting, off by default): hold the teleport action, an arc shows the target, release to go; the target is checked against the NavMesh and the capsule, and the move is behind a short fade.
  • a vignette while moving or turning smoothly (default on), and a fade for every jump of the view: travel, teleport, respawn.

The settings (XrSettings: turn, snapAngle, turnSpeed, vignette, seated, forward, dominantHand, teleport) are saved in the browser under the game's storageKey and appear as an "In VR" page of the pause menu. A game may limit them with limits (for example teleport: false), but never removes the snap turn and the vignette: they are the comfort floor.

UI in the world#

The model of UIService (menu, HUD lines, prompt, toasts, loading) is drawn by XrUi on panels in the world, so game UI built on UIService needs no change. The menu opens 1.5 m in front of the head and then stays in the world; items are chosen by the controller's laser and the trigger, or by the stick with A and B. The HUD and the prompt sit on a small panel that follows the head lazily. A game's transition gets ctx.xr and may show its interlude world in the headset.

Panels are drawn with Canvas 2D. A game that builds its own DOM elements does not get them in the headset: put the text through UIService.

Pointing, touching, grabbing#

Interactor gets two focus sources besides the view cone: the hand's ray (laser) and touch (the hand inside the target's bounding box). Interactable does not change: interact(actor) is called by whatever focused it, and the prompt shows the matching button.

Grabbable (in @ferret/interaction) is a component on an entity with a RigidBody:

entity.add(Grabbable, { maxMass: 20, hands: 'both', twoHanded: true, snapPoints: [{ position: [0, 0, 0.12] }] });

A hand takes it with squeeze. The object follows the hand by a velocity drive, so heavy objects lag and a wall in the way stops them; on release it keeps the hand's velocity of the last 100 ms (a throw). On a screen the same component is held in front of the camera by the use action (screenHold), so a game gets grabbing on both. Events: grab:take, grab:release.

Multiplayer#

An entity with XrRig replicates its head and hand poses through xrPoseField; remote players show them with XrRemoteAvatar. Screen players replicate nothing new.

Rendering and budgets#

Tier xr of QUALITY: one shadow cascade with a 1024 map, MSAA 4, no FXAA and no post-processing, draw distance 200 m, textures up to 2048. Fixed foveation is on; multiview is reported by xr.status, not forced; the framebuffer scale follows the frame time (0.8 to 1.2) and changes only behind a fade. The frame rate is the highest the headset offers up to xr.frameRate (default 90). Streaming uses smaller radii.

Targets, measured on the headset: frame time under 13.9 ms on Quest 2 (72 Hz) and 11.1 ms on Quest 3 (90 Hz) for 99% of frames; under 150 draw calls per eye; under 750 000 triangles in view; page memory under 1.5 GB on Quest 2; a travel keeps frames coming; entering VR in under 2 s. Load glTF, shapes and NavMeshes behind a fade, never in free play.

Installing on the headset#

A game site with a pwa.json next to its template gets a web app manifest and a service worker (tools/pwa): the game installs from Quest Browser, starts from the library, and plays cached levels offline.

Commands#

The same through the inspector and the viewer's MCP: xr.status, xr.enter, xr.exit, xr.settings (pass a partial object to change the settings).

Checking#

Without a headset: xr.status on a screen reports unsupported; the unit tests drive XrService with a fake XrDriver; Playwright specs use an emulated headset. On a Quest: comfort first (snap turn, vignette, fades), then the budgets above.