In-depth: Reactivity

guides/release/in-depth-topics/reactivity/index.md

@@ -0,0 +1,35 @@
+# The Glimmer Reactivity System
+
+## Table of Contents
+
+1. [Tag Composition](./tag-composition.md): The formal composition semantics of Glimmer's tag-based
+   validation system.
+2. [The Fundamental Laws of Reactivity](./laws.md): A definition of Glimmer's reliable and
+   consistent reactive programming model, and the rules that reactive abstractions must
+   satisfy in order to safely support this model.
+3. [System Phases](./system-phases.md): A description of the phases of the Glimmer execution model:
+   _action_, _render_, and _idle_, and how the exeuction model supported batched _UI_ updates while
+   maintaining a _coherent_ data model.
+4. [Reactive Abstractions](./reactive-abstractions.md): A description of the implementation of
+   a number of reactive abstractions, and how they satisfy the laws of reactivity.
+
+### Pseudocode
+
+This directory also contains pseudocode for the foundation of a reactive system that satisfies these
+requirements, and uses them to demonstrate the implementation of the reactive abstractions.
+
+- [`tags.ts`](./pseudocode/tags.ts): A simple implementation of the tag-based validation system,
+  including an interface for a runtime that supports tag consumptions and tracking frames.
+- [`primitives.ts`](./pseudocode/primitives.ts): Implementation of:
+  - `Snapshot`, which captures a value at a specific revision with its tag validator.
+  - `PrimitiveCell` and `PrimitiveCache`, which implement a primitive root storage and a primitive
+    cached computation, both of which support law-abiding snapshots.
+- [`composition.ts`](./pseudocode/composition.ts): Implementations of the higher-level reactive
+  constructs described in [Reactive Abstractions](./reactive-abstractions.md) in terms of the
+  reactive primitives.
+
+> [!TIP]
+>
+> While these are significantly simplified versions of the production primitives that ship with
+> Ember and Glimmer, they serve as clear illustrations of how to implement reactive abstractions
+> that satisfy the reactive laws.

guides/release/in-depth-topics/reactivity/laws.md

@@ -0,0 +1,81 @@
+# The Fundamental Laws of Reactivity
+
+## ♾ The Fundamental Axiom of Reactivity
+
+> ### "A reactive abstraction must provide both the current value and a means to detect invalidation without recomputation."
+
+From the perspective of a Glimmer user, this axiom enables writing reactive code using standard
+JavaScript functions and getters that automatically reflect the current state of UI inputs.
+
+**Glimmer users write UI code as straightforward rendering functions**, yet the system behaves _as
+if_ these functions re-execute completely whenever any reactive value changes.
+
+> [!IMPORTANT]
+>
+> When root state is mutated, all reactive abstractions reflect those changes immediately, even when
+> implemented with caching. Glimmer's reactive values are _always coherent_ — changes are never
+> batched in ways that would allow inconsistencies between computed values and their underlying root
+> state.
+
+## Definitions
+
+- **Root Reactive State**: An atomic reactive value that can be updated directly. It is represented
+  by a single [value tag](./concepts.md#value-tag). You can create a single piece of root state
+  explicitly using the `cell` API, but containers from `tracked-builtins` and the storage created by
+  the `@tracked` decorator are also root reactive state.
+- **Formula**: A reactive computation that depends on a number of reactive values. A formula's
+  revision is the most recent revision of any of the members used during the last computation (as a
+  [combined tag](./concepts.md#combined-tag)). A
+  formula will _always_ recompute its output if the revision of any of its members is advanced.
+- **Snapshot**: A _snapshot_ of a reactive abstraction is its _current value_ at a specific
+  revision. The snapshot <a id="invalidate"></a> _invalidates_ when the abstraction's tag has a more
+  recent revision. _A reactive abstraction is said to _invalidate_ when any previous snapshots would
+  become invalid._
+
+## The Fundamental Laws of Reactivity
+
+In order to satisfy the _Fundamental Axiom of Reactivity_, all reactive abstractions must adhere to these six laws:
+
+1. **Dependency Tracking**: A reactive abstraction **must** [invalidate](#invalidate) when any
+   reactive values used in its _last computation_ have changed. _The revision of the tag associated
+   with the reactive abstraction <u>must</u> advance to match the revision of its most recently
+   updated member._
+
+2. **Value Coherence**: A reactive abstraction **must never** return a cached _value_ from a
+   revision older than its current revision. _After a root state update, any dependent reactive
+   abstractions must recompute their value when next snapshotted._
+
+3. **Transactional Consistency**: During a single rendering transaction, a reactive abstraction
+   **must** return the same value and revision for all snapshots taken within that transaction.
+
+4. **Snapshot Immutability**: The act of snapshotting a reactive abstraction **must not**
+   advance the reactive timeline. _Recursive snapshotting (akin to functional composition) naturally
+   involves tag consumption, yet remains consistent with this requirement as immutability applies
+   recursively to each snapshot operation._
+
+5. **Defined Granularity**: A reactive abstraction **must** define a contract specifying its
+   _invalidation granularity_, and **must not** invalidate more frequently than this contract
+   permits. When a reactive abstraction allows value mutations, it **must** specify its equivalence
+   comparison method. When a new value is equivalent to the previous value, the abstraction **must
+   not** invalidate.
+
+All reactive abstractions—including built-in mechanisms like `@tracked` and `createCache`, existing
+libraries such as `tracked-toolbox` and `tracked-builtins`, and new primitives like `cell`—must
+satisfy these six laws to maintain the Fundamental Axiom of Reactivity when these abstractions are
+composed together.
+
+> [!TIP]
+> 
+> In practice, the effectiveness of reactive composition is bounded by the **Defined Granularity** and **Specified Equivalence** of the underlying abstractions.
+> 
+> For instance, if a [`cell`](#cell) implementation defines granularity at the level of JSON serialization equality, then all higher-level abstractions built upon it will inherit this same granularity constraint.
+> 
+> The laws do not mandate comparing every value in every _computation_, nor do they require a
+> uniform approach to equivalence based solely on reference equality. Each abstraction defines its
+> own appropriate granularity and equivalence parameters.
+>
+> For developers building reactive abstractions, carefully selecting granularity and equivalence
+> specifications that align with user mental models is crucial—users will experience the system
+> through these decisions, expecting UI updates that accurately reflect meaningful changes in their
+> application state.
+> 

guides/release/in-depth-topics/reactivity/pseudocode/composition.ts

@@ -0,0 +1,123 @@
+import { PrimitiveCache, PrimitiveCell, type Status } from './primitives';
+import { runtime, MutableTag, type Tag } from './tags';
+
+export class LocalCopy<T> {
+  #upstream: PrimitiveCache<T>;
+  #local: PrimitiveCell<T>;
+
+  constructor(compute: () => T) {
+    this.#upstream = new PrimitiveCache(compute);
+    this.#local = new PrimitiveCell();
+  }
+
+  /**
+   * Safely return the value of the upstream computation or the local cell, whichever is more
+   * recent. This satisfies the laws of reactivity transitively through `mostRecent`.
+   */
+  read(): T {
+    return mostRecent(this.#upstream.snapshot(), this.#local.unsafeSnapshot()).value;
+  }
+
+  /**
+   * Safely write a value to the local cell during the "action" phase.
+   */
+  write(value: T): void {
+    this.#local.write(value);
+  }
+}
+
+/**
+ * Safely returns the most recent status from the given statuses. If there are multiple status with
+ * the same, latest revision, the first such status in the list will be returned.
+ *
+ * This satisfies the transactionality law because we consume all tags in all cases, which means
+ * that:
+ *
+ * > The value of the most recent status cannot change after the `MostRecent` was computed in the
+ * > same rendering transaction, because a change to any of the specified statuses would trigger a
+ * > backtracking assertion.
+ *
+ * The granularity of `mostRecent` is: the call to `mostRecent` will invalidate when the tags of any
+ * of the statuses passed to it invalidate. This is as granular as possible because a change to any
+ * of the tags would, by definition, make it the most recent.
+ */
+function mostRecent<S extends [Status<unknown>, ...Status<unknown>[]]>(...statuses: S): S[number] {
+  const [first, ...rest] = statuses;
+  runtime.consume(first.tag);
+
+  return rest.reduce((latest, status) => {
+    runtime.consume(latest.tag);
+    return status.tag.revision > latest.tag.revision ? status : latest;
+  }, first);
+}
+
+export function tracked<V, This extends object>(
+  _value: ClassAccessorDecoratorTarget<This, V>,
+  context: ClassAccessorDecoratorContext<This, V>
+): ClassAccessorDecoratorResult<This, V> {
+  // When the field is initialized, initialize a mutable tag to represent the root storage.
+  context.addInitializer(function (this: This) {
+    MutableTag.init(this, context.name);
+  });
+
+  return {
+    get(this: This): V {
+      // When the field is accessed, consume the tag to track the read, and return the underlying
+      // value stored in the field.
+      const tag = MutableTag.get(this, context.name);
+      tag.consume();
+      return context.access.get(this);
+    },
+
+    set(this: This, value: V): void {
+      // When the field is written, update the tag to track the write, and update the underlying
+      // value stored in the field.
+      const tag = MutableTag.get(this, context.name);
+      context.access.set(this, value);
+      tag.update();
+    },
+  };
+}
+
+const COMPUTE = new WeakMap<Cache<unknown>, () => unknown>();
+
+declare const FN: unique symbol;
+type FN = typeof FN;
+type Cache<T> = {
+  [FN]: () => T;
+};
+
+export function createCache<T>(fn: () => T): Cache<T> {
+  const cache = {} as Cache<T>;
+  let last = undefined as { value: T; tag: Tag; revision: number } | undefined;
+
+  COMPUTE.set(cache, () => {
+    if (last && last.revision >= last.tag.revision) {
+      runtime.consume(last.tag);
+      return last.value;
+    }
+
+    runtime.begin();
+    try {
+      const result = fn();
+      const tag = runtime.commit();
+      last = { value: result, tag, revision: runtime.current() };
+      runtime.consume(tag);
+      return result;
+    } catch {
+      last = undefined;
+    }
+  });
+
+  return cache;
+}
+
+export function getCache<T>(cache: Cache<T>): T {
+  const fn = COMPUTE.get(cache);
+
+  if (!fn) {
+    throw new Error('You must only call `getCache` with the return value of `createCache`');
+  }
+
+  return fn() as T;
+}

guides/release/in-depth-topics/reactivity/pseudocode/primitives.ts

@@ -0,0 +1,105 @@
+import { type Tag, MutableTag, runtime } from './tags';
+
+export class PrimitiveCell<T> {
+  readonly #tag: MutableTag = MutableTag.init(this, 'value');
+  #value: T;
+
+  /**
+   * Unsafely read the value of the cell. This is unsafe because it exposes the raw value of the tag
+   * and the last value of the cell, but relies on the caller to ensure that the tag is consumed if
+   * the abstraction needs to invalidate when the cell changes.
+   *
+   * Callers of `unsafeSnapshot` must satisfy the transactionality law by consuming the tag whenever a
+   * change to the value would result in a change to the computed value of the abstraction.
+   */
+  unsafeSnapshot(): Snapshot<T> {
+    return Snapshot.of({ value: this.#value, tag: this.#tag });
+  }
+
+  write(value: T): void {
+    this.#tag.update();
+    this.#value = value;
+  }
+}
+export type Status<T> = { value: T; tag: Tag };
+type Last<T> = { value: T; tag: Tag; revision: number };
+
+export class Snapshot<T> {
+  static of<T>(status: Status<T>): Snapshot<T> {
+    return new Snapshot({ value: status.value, tag: status.tag });
+  }
+  readonly #value: T;
+  readonly #tag: Tag;
+  readonly #revision: number;
+
+  private constructor({ value, tag }: Status<T>) {
+    this.#value = value;
+    this.#tag = tag;
+    this.#revision = tag.revision;
+  }
+
+  get tag(): Tag {
+    return this.#tag;
+  }
+
+  get value(): T {
+    return this.#value;
+  }
+}
+
+export class PrimitiveCache<T> {
+  readonly #compute: () => T;
+  #last: Last<T>;
+
+  constructor(compute: () => T) {
+    this.#compute = compute;
+
+    // A `PrimitiveCache` must always be initialized with a value. If all of the primitives used
+    // inside of a `PrimitiveCache` are compliant with the Fundamental Laws of Reactivity, then
+    // initializing a cache will never change the revision counter.
+    this.read();
+  }
+
+  /**
+   * Unsafely read the status of the cache. This is unsafe because it exposes the raw value of the
+   * tag and the last value of the cache, but relies on the caller to ensure that the tag is
+   * consumed if the abstraction needs to invalidate when the cache changes.
+   *
+   * Callers of `unsafeSnapshot` must satisfy the transactionality law by consuming the tag whenever a
+   * change to the value would result in a change to the computed value of the abstraction.
+   */
+  snapshot(): Snapshot<T> {
+    return Snapshot.of(this.#last);
+  }
+
+  /**
+   * Safely read the value of the cache. This satisfies the transactionality law because:
+   *
+   * 1. If the cache is valid, then it will return the last value of the cache. This is guaranteed
+   *    to be the same value for all reads in the same rendering transaction because any mutations
+   *    to any _members_ of the last tag will trigger a backtracking assertion.
+   * 2. If the cache is invalid, then the previous value of the cache is thrown away and the
+   *    computation is run again. Any subsequent reads from the cache will return the same value
+   *    because of (1).
+   */
+  read(): T {
+    if (this.#last && this.#last.revision >= this.#last.tag.revision) {
+      runtime.consume(this.#last.tag);
+      return this.#last.value;
+    }
+
+    runtime.begin();
+    try {
+      const result = this.#compute();
+      const tag = runtime.commit();
+      this.#last = { value: result, tag, revision: runtime.current() };
+      runtime.consume(tag);
+      return result;
+    } catch (e) {
+      // This is possible, but not currently modelled at all. The approach used by the error
+      // recovery branch that was not merged is: tags are permitted to capture errors, and
+      // value abstractions expose those errors in their safe read() abstractions.
+      throw e;
+    }
+  }
+}