🥽 WebXR & VR

Immersive VR, AR passthrough, and Cardboard VR for phones — all with one‑line setup

📋 Overview

The XR module adds immersive Virtual Reality, Augmented Reality, and Google Cardboard support to any Nova64 cart. It integrates directly with the existing camera and renderer systems, so your 3D scenes work in VR with zero refactoring.

💡 Three modes, one API

WebXR VR — Full immersive VR on Quest, Vision Pro, Index, or any WebXR-compatible headset. Includes controller models, hand tracking, and a world-space HUD.
WebXR AR — Augmented reality passthrough with hit-test support for placing objects on real surfaces.
Cardboard VR — Stereoscopic split-screen with gyroscope head tracking for phones without WebXR. Works with any Google Cardboard-style viewer.

⚠️ Important notes

Post-processing is auto-disabled during VR/AR sessions (bloom, vignette, etc. don't support multi-view stereo rendering). They are restored when you exit VR.
2D HUD overlay is rendered on a world-space billboard in VR that follows your gaze at a comfortable 2m distance. Your print() and draw() calls work in VR.
HTTPS required — WebXR only works over HTTPS or localhost.
Call enableVR() or enableAR() in your cart's init() function. The user clicks the "Enter VR" / "Start AR" button when ready.

🥽 VR Functions

enableVR(options?)

Enables WebXR VR mode. Shows an "Enter VR" button on screen. Sets up the camera rig, two controllers with grip models, hand tracking meshes, and a world-space HUD billboard.

options object - Optional configuration

options.referenceSpace string - 'local-floor' (default), 'local', or 'bounded-floor'

Example:

export function init() {
  // Enable VR with default settings (standing/room-scale)
  enableVR();

  // Build your 3D scene normally
  createCube(2, 0x00ccff, [0, 1, -3], { material: 'holographic' });
  setAmbientLight(0x404060, 0.6);
  createPointLight(0xff6600, 1.5, 20, [3, 4, 2]);
  setFog(0x0a0a1a, 5, 25);
}

enableAR(options?)

Enables WebXR AR mode with passthrough video. Shows a "Start AR" button. Automatically requests hit-test and optionally hand-tracking and dom-overlay features.

options object - Optional configuration

options.referenceSpace string - 'local' (default)

options.sessionInit object - Extra XRSessionInit properties (requiredFeatures, optionalFeatures)

Example:

export function init() {
  enableAR();
  // Objects will appear in the real world through passthrough
  createSphere(0.2, 0xff00ff, [0, 0, -1], { material: 'holographic' });
}

enableCardboardVR()

Enables stereoscopic split-screen rendering with gyroscope head tracking for phones without WebXR. Uses StereoEffect (code-split, only loaded when called) and DeviceOrientationEvent. Automatically enters fullscreen and locks to landscape orientation.

📱 iOS Permission

On iOS 13+, the browser requires explicit user permission for device orientation. enableCardboardVR() handles this automatically by calling DeviceOrientationEvent.requestPermission().

Example:

export async function init() {
  // Cardboard mode for any phone with a gyroscope
  await enableCardboardVR();

  createGradientSkybox(0x0a0a2e, 0x1a1a4e);
  createCube(3, 0x00ffcc, [0, 1, -5], { material: 'metallic' });
}

disableXR()

Cleanly exits any active XR or Cardboard session. Removes the VR/AR button, cleans up controllers, hand models, the VR HUD, the camera rig, and restores the camera to the scene root. Safe to call even if no session is active.

Example:

// Exit VR on game over
if (health <= 0) {
  disableXR();
}

isXRActive()

Returns true while any XR mode is presenting (VR, AR, or Cardboard).

Returns: boolean

Example:

export function update(dt) {
  if (isXRActive()) {
    // VR-specific input handling
    const ctrls = getXRControllers();
    // ...
  } else {
    // Desktop keyboard input
    if (key('KeyW')) player.z -= 5 * dt;
  }
}

isXRSupported()

Returns true if the browser has the navigator.xr API available. Does not check for specific session types.

Returns: boolean

getXRSession()

Returns the raw XRSession object for advanced usage, or null if not in a session.

Returns: XRSession | null

getXRMode()

Returns the current XR mode string.

Returns: string | null - 'vr', 'ar', 'cardboard', or null

🎮 Controller & Hand Tracking

getXRControllers()

Returns an array of controller state objects (typically 2 — one per hand). Each object includes world-space position, quaternion rotation, button states, analog axes, and grip position.

Returns: Array<ControllerState>

ControllerState Object

Property	Type	Description
`index`	number	0 = left, 1 = right
`position`	{x, y, z}	World-space position in meters
`rotation`	{x, y, z, w}	Quaternion rotation
`buttons`	Array	Array of `{ pressed, touched, value }`
`axes`	Array	Thumbstick/touchpad [x, y] values (-1 to 1)
`grip.position`	{x, y, z}	Grip (handle) world-space position

Standard Button Mapping

Index	Quest / Touch	Index / Vive
0	Trigger	Trigger
1	Grip / Squeeze	Grip
2	—	Trackpad
3	Thumbstick click	—
4	A / X button	—
5	B / Y button	—

Example — spawn objects with trigger:

export function update(dt) {
  if (!isXRActive()) return;

  const ctrls = getXRControllers();
  for (const ctrl of ctrls) {
    // Trigger pressed (button 0)
    if (ctrl.buttons[0]?.pressed) {
      createSphere(0.1, 0x00ff88, [
        ctrl.position.x,
        ctrl.position.y,
        ctrl.position.z,
      ]);
    }

    // Thumbstick for locomotion
    const [stickX, stickY] = ctrl.axes;
    if (stickY) setCameraRigPosition(
      rig.x, rig.y, rig.z - stickY * 3 * dt
    );
  }
}

getXRHands()

Returns hand-tracking joint data for devices that support it (Quest, Vision Pro). Each hand has 25 joint positions following the WebXR Hand Input spec.

Returns: Array<HandState> - Only hands that are tracked are included

HandState Object

Property	Type	Description
`index`	number	0 = left, 1 = right
`joints`	object	Map of joint name → `{x, y, z}` world positions

🖐️ Joint Names

Joints follow the WebXR Hand Input spec: wrist, thumb-metacarpal, thumb-phalanx-proximal, thumb-phalanx-distal, thumb-tip, index-finger-metacarpal, … index-finger-tip, middle-finger-tip, ring-finger-tip, pinky-finger-tip, etc. (25 joints total per hand.)

Example — move a cursor to index finger tip:

const hands = getXRHands();
if (hands.length > 0) {
  const tip = hands[0].joints['index-finger-tip'];
  setPosition(cursor, tip.x, tip.y, tip.z);
}

setCameraRigPosition(x, y, z)

Moves the VR player rig to a new world-space position. The camera (headset) position is relative to this rig origin. Use this for teleportation or smooth locomotion.

x, y, z number - World-space coordinates

Example — teleport to a position:

// Teleport player to a new location
setCameraRigPosition(10, 0, -5);

setXRReferenceSpace(type)

Changes the WebXR reference space type after initialization.

type string - 'local', 'local-floor', or 'bounded-floor'

🎮 Complete VR Cart Example

This example shows a complete VR cart with controller interaction, dual-mode input (VR + keyboard), and a HUD overlay.

examples/my-vr-game/code.js

let cubes = [];
let player = { x: 0, y: 0, z: 0 };

export function init() {
  // One line to enable VR — shows "Enter VR" button
  enableVR({ referenceSpace: 'local-floor' });

  // Build scene (works both on-screen and in VR)
  setCameraPosition(0, 1.6, 5);
  setAmbientLight(0x404060, 0.6);
  createPointLight(0x00ccff, 2, 20, [0, 5, 0]);
  setFog(0x0a0a1a, 5, 25);
  createGradientSkybox(0x0a0a2e, 0x1a1a4e);

  // Floor
  const floor = createPlane(20, 20, 0x334455, [0, 0, 0]);
  rotateMesh(floor, -Math.PI / 2, 0, 0);

  // Scatter some objects
  for (let i = 0; i < 10; i++) {
    const c = createCube(0.5, Math.random() * 0xffffff,
      [(Math.random() - 0.5) * 8, 0.5, (Math.random() - 0.5) * 8],
      { material: 'holographic' });
    cubes.push(c);
  }
}

export function update(dt) {
  // Rotate all cubes
  cubes.forEach(c => rotateMesh(c, 0, dt, 0));

  if (isXRActive()) {
    // VR: spawn cubes with trigger
    for (const ctrl of getXRControllers()) {
      if (ctrl.buttons[0]?.pressed) {
        cubes.push(createCube(0.15, 0x00ff88, [
          ctrl.position.x, ctrl.position.y, ctrl.position.z
        ]));
      }
    }
  } else {
    // Desktop: keyboard controls
    if (key('KeyW')) player.z -= 5 * dt;
    if (key('KeyS')) player.z += 5 * dt;
    setCameraPosition(player.x, 1.6, player.z + 5);
  }
}

export function draw() {
  // HUD works in both VR and desktop
  print('My VR Game', 10, 10, 0x00ccff);
  print(`Cubes: ${cubes.length}`, 10, 30, 0xffffff);
}