10. Replay

Replay records the layout of the screen as a wireframe, so a session can be watched without capturing pixels.

10.1 Frames

• MUST record a wireframe: view kinds, frames (position and size), visibility and state, and touches. MUST NOT capture pixels or screenshots.
• A frame is a keyframe (the viewport and every node) or a delta (the nodes added, updated and removed since the previous frame, by node id). Frames follow schemas/v0.1/replay-segment.schema.json:
  • a node has an id, stable for the same view within the session, a parent, a kind (container, text, input, button, image, switch, slider, scroll, keyboard, webview, map, video, hidden), a rect [x, y, width, height] in points, and optionally its scroll, its style (colors, border, corner radius), its text (already masked, 10.3) and its state;
  • a root’s rect is in the screen’s coordinates; any other node’s is in its parent’s content coordinates, before the parent’s scroll (its content offset [x, y], absent when zero) is taken away. On screen, a node is at its parent’s position plus its own rect minus the parent’s scroll. Scrolling then changes one scroll, not the rect of every node that moves;
  • nodes are listed parents first, siblings in drawing order;
  • a player applies one delta in this order: its update entries first (a changed parent moves the node, with the nodes under it), then its remove entries (each id with the nodes under it at that point), then its add entries (an id that exists is replaced). So one delta can move a child away from a parent it then removes, and can remove an id and add it back as a new node. A recorder MUST write deltas that give the next frame’s nodes when applied in this order, as replay.apply_delta does (appendix B.2);
  • touches are { t, x, y, phase }, with phase one of down, move, up, cancel.
• MUST start every session’s replay, and every return to the foreground, with a keyframe. SHOULD also write a keyframe on each screen change, at least every 30 seconds, and when a delta would be larger than half a keyframe.
• MUST number frames with seq, an integer that increases within the session, and time them with t, Unix nanoseconds as a decimal string.
• SHOULD capture at most twice a second, and only after something changed (a touch, a screen change, the keyboard, a scroll), so a still screen costs nothing.
• SHOULD stay under 60 KB a minute after compression on a typical screen, and MUST NOT send more than 300 KB a minute: past it, the SDK sends keyframes only, at most one every 5 seconds, for the rest of that minute.

10.2 Segments

Frames travel in segments, not one log record per frame, so the log store holds one small record per segment and the frames live in object storage.

• SHOULD group about 10 seconds of frames in a segment: a JSON object with v (1), session_id and frames, compressed with gzip.
• MUST upload a segment as an attachment of kind replay.segment (1.7), then send a log record with event name apsio.replay.segment and these attributes:

Attribute	Type	Value
apsio.attachment.id	string	The attachment that holds the segment
apsio.replay.segment.first_sequence	int	The seq of its first frame
apsio.replay.segment.last_sequence	int	The seq of its last frame
apsio.replay.segment.end_time	int, Unix ns	The time of its last frame
apsio.replay.segment.keyframe	boolean	Whether it holds at least one keyframe
apsio.replay.segment.bytes	int	The attachment’s size, compressed

The record’s timeUnixNano is the time of the segment’s first frame, and it carries session.id like any record of the session.

• SHOULD close the open segment when the app moves to the background or terminates, so a crash loses at most the last segment.
• MUST record replay only when all of these hold:
  • the app turned replay on (an SDK MAY leave it off until the app does);
  • the policy’s capture.replay is not false (section 12);
  • the session is kept for replay: the decision of 9.1 with the rate session_rate × replay_rate, so a session with replay is always a kept session.
• MUST stop recording and drop unsent segments when consent is withdrawn or the kill switch turns on, like the rest of the queue (4.1).

10.3 Masking defaults

Masking happens on the device, before a frame leaves memory, and fails closed:

• MUST mask all text by default, not only text inputs, as replay.text_mask does (appendix B.2): every extended grapheme cluster (Unicode UAX #29) becomes x, except clusters made only of White_Space characters, which stay. The length, spaces and line breaks of the text are kept; its characters are not.
• MUST always mask secure text fields, and fields for passwords, one-time codes and card numbers; no setting can unmask them.
• MUST render images as boxes.
• MUST hide touches that fall on anything not shown: masked text (masked by the default or by a setting), an image, a secure field, the keyboard, a hidden box. Taps on a custom PIN pad made of plain buttons would otherwise give the PIN away. An app that wants taps on its buttons visible unmasks them (for example by class).
• MUST record a touch only when every node under its point, at any depth and in any window, shows its content: one masked or hidden node under it is enough to drop the touch, whatever is drawn above. A shown overlay (a transparent bar, a scrim, a container) over masked text does not reveal the tap. A touch over no recorded node is not recorded either.
• MUST render as a hidden box anything the SDK cannot read: web views, maps, video, camera previews, and view types whose content is not reachable (for example SwiftUI internals).
• MAY let the app opt down to masking text inputs only.

10.4 Overrides and precedence

An app overrides masking with mask, unmask and hide settings, set globally (at start), per view class, or per view instance (section 13). Hide removes the view’s content entirely; mask replaces its text; unmask shows it.

Precedence, the same on every platform:

1. Secure inputs are always masked.
2. Nothing is revealed inside a hidden ancestor.
3. Between settings at the same level, hide beats mask, and mask beats unmask.
4. The nearest explicit setting wins: instance beats class, and class beats global.

A view’s accessibility identifier or tag holding the word apsio-mask, apsio-unmask or apsio-hide counts as an instance setting. A word is a maximal run of ASCII letters, digits and hyphens: any other character, white space, _, ., ,, / or : among them, separates words. A word MUST equal the token exactly, case included: row apsio-mask, checkout.coupon.apsio-hide, my_apsio-hide, card_number_apsio-mask and price_apsio-mask,total carry a setting; apsio-unmasked-total, my-apsio-hide (one word), apsio-unmask2 and Apsio-Hide do not. Cross-platform frameworks use these tokens.

A class setting, of the app or of the policy (10.5), names a class by its run-time name, and a name that matches no class would fail open: the views it was meant to mask or hide would be shown. So:

• A class named as a string MUST keep that name in the app’s build. On Android, R8 renames the classes of a minified app unless the app keeps their names (-keepnames); on Apple platforms, a Swift class is named with its module (Checkout.CardField), as NSClassFromString resolves it. An API that takes the class itself (a Class, a type) needs neither.
• An SDK MUST resolve every name of the app’s and the policy’s hide and mask lists against the app’s classes when a session starts recording replay and when it applies a new policy. When one does not resolve, the SDK MUST record the rest of that session in strict mode: every text and input masked and every image hidden, whatever any setting unmasks, and every view whose content the SDK knows only as its own drawing hidden too. A renamed class and a class the app does not ship look the same; both fail closed. The check runs again for each session.
• In strict mode the SDK MUST log a warning on the device and send, once per session, a log record with event name apsio.replay.strict_mode, severity WARN, and the attribute apsio.replay.unresolved_classes: the names that did not resolve, so the app’s owner can fix the build or the policy.
• A name in an unmask list that does not resolve unmasks nothing, which already fails closed; it does not start strict mode.

replay.mask (appendix B.2) states the rules exactly, over a neutral tree each SDK maps its views onto. For each node, from the root down:

• the node’s own setting is its instance setting (the API call and the tokens), or, when it has none, its class setting (the classes it belongs to, against the app’s hide, mask and unmask lists); within a level, hide beats mask beats unmask;
• the policy’s floor is the strongest of the policy’s hide_classes and mask_classes the node or an ancestor belongs to (10.5): the own setting and the effective setting below are never weaker than it, whatever the app set at any level;
• inside a hidden ancestor, or with its own setting hide, or of kind webview, map, video or keyboard, the node is hidden, and so is everything inside it;
• otherwise its effective setting is its own setting, or the nearest ancestor’s;
• a secure input or an image is masked; a text or an input is masked when the effective setting is mask, shown when it is unmask, and otherwise follows the global text level (all masks text and inputs, inputs masks inputs only); a container is masked when the effective setting is mask;
• in strict mode (a name of the app’s or the policy’s hide or mask list does not resolve), a text or an input that is not hidden is masked, and an image is hidden;
• touches show only when touches are on and the node is shown; a touch is recorded only when every node under it has touches shown (10.3).

10.5 Remote tightening

The policy MAY tighten masking. It MUST NOT loosen it, and an SDK MUST ignore a policy that tries:

• masking.text: "all" masks all text even where the app chose inputs, and makes the SDK ignore every unmask setting; without masking.text the app’s settings stand, and an SDK MUST NOT read an absent masking.text as "all";
• masking.mask_classes and masking.hide_classes win over every setting of the app: a node of one of those classes, and everything inside it, is masked or hidden even where an instance setting, a token or a class of the app unmasks it. A policy is served per app key, and an app is one platform, so its class lists name that platform’s classes: a backend MUST NOT serve one policy with class lists to apps of different platforms. A name that does not resolve in the app starts strict mode (10.4);
• masking.text: "inputs" does nothing when the app masks all text;
• sampling.replay_rate can lower the share of sessions with replay, never raise it above the session rate;
• capture.replay: false stops replay.