chore(serdes): rename serialize(d=>r), new api

.changeset/nasty-planes-jam.md

@@ -2,8 +2,8 @@
 '@qwik.dev/core': minor
 ---
 
-FEAT: `useSerialized$(fn)` and `createSerialized$(fn)` allow serializing custom objects. You must provide a
+FEAT: `useSerializer$(fn)` and `createSerializer$(fn)` allow serializing custom objects. You must provide a
 function that converts the custom object to a serializable one via the `[SerializerSymbol]`
-property, and then provide `use|createSerialized$(fn)` with the function that creates the custom object
+property, and then provide `use|createSerializer$(fn)` with the function that creates the custom object
 from the serialized data. This will lazily create the value when needed. Note that the serializer
 function may return a Promise, which will be awaited. The deserializer must not return a Promise.

packages/docs/src/routes/api/qwik/api.json

@@ -222,18 +222,18 @@
       "mdFile": "core.createcontextid.md"
     },
     {
-      "name": "createSerialized$",
-      "id": "createserialized_",
+      "name": "createSerializer$",
+      "id": "createserializer_",
       "hierarchy": [
         {
-          "name": "createSerialized$",
-          "id": "createserialized_"
+          "name": "createSerializer$",
+          "id": "createserializer_"
         }
       ],
       "kind": "Function",
-      "content": "Create a signal that holds a custom serializable value. See [useSerialized$](#useserialized_) for more details.\n\n\n```typescript\ncreateSerialized$: <T extends CustomSerializable<any, S>, S = T extends {\n    [SerializerSymbol]: (obj: any) => infer U;\n} ? U : unknown>(qrl: (data: S | undefined) => T) => T extends Promise<any> ? never : SerializedSignal<T>\n```\n\n\n<table><thead><tr><th>\n\nParameter\n\n\n</th><th>\n\nType\n\n\n</th><th>\n\nDescription\n\n\n</th></tr></thead>\n<tbody><tr><td>\n\nqrl\n\n\n</td><td>\n\n(data: S \\| undefined) =&gt; T\n\n\n</td><td>\n\n\n</td></tr>\n</tbody></table>\n**Returns:**\n\nT extends Promise&lt;any&gt; ? never : SerializedSignal&lt;T&gt;",
+      "content": "Create a signal that holds a custom serializable value. See [useSerializer$](#useserializer_) for more details.\n\n\n```typescript\ncreateSerializer$: <T, S>(arg: SerializerArg<T, S>) => T extends Promise<any> ? never : SerializerSignal<T>\n```\n\n\n<table><thead><tr><th>\n\nParameter\n\n\n</th><th>\n\nType\n\n\n</th><th>\n\nDescription\n\n\n</th></tr></thead>\n<tbody><tr><td>\n\narg\n\n\n</td><td>\n\nSerializerArg&lt;T, S&gt;\n\n\n</td><td>\n\n\n</td></tr>\n</tbody></table>\n**Returns:**\n\nT extends Promise&lt;any&gt; ? never : SerializerSignal&lt;T&gt;",
       "editUrl": "https://github.com/QwikDev/qwik/tree/main/packages/qwik/src/core/signal/signal.public.ts",
-      "mdFile": "core.createserialized_.md"
+      "mdFile": "core.createserializer_.md"
     },
     {
       "name": "createSignal",
@@ -2046,18 +2046,18 @@
       "mdFile": "core.useresource_.md"
     },
     {
-      "name": "useSerialized$",
-      "id": "useserialized_",
+      "name": "useSerializer$",
+      "id": "useserializer_",
       "hierarchy": [
         {
-          "name": "useSerialized$",
-          "id": "useserialized_"
+          "name": "useSerializer$",
+          "id": "useserializer_"
         }
       ],
       "kind": "Variable",
-      "content": "Creates a signal which holds a custom serializable value. It requires that the value implements the `CustomSerializable` type, which means having a function under the `[SerializeSymbol]` property that returns a serializable value when called.\n\nThe `fn` you pass is called with the result of the serialization (in the browser, only when the value is needed), or `undefined` when not yet initialized. If you refer to other signals, `fn` will be called when those change just like computed signals, and then the argument will be the previous output, not the serialized result.\n\nThis is useful when using third party libraries that use custom objects that are not serializable.\n\nNote that the `fn` is called lazily, so it won't impact container resume.\n\n\n```typescript\nuseSerialized$: typeof createSerialized$\n```\n\n\n\n```tsx\nclass MyCustomSerializable {\n  constructor(public n: number) {}\n  inc() {\n    this.n++;\n  }\n  [SerializeSymbol]() {\n    return this.n;\n  }\n}\nconst Cmp = component$(() => {\n  const custom = useSerialized$<MyCustomSerializable, number>(\n    (prev) =>\n      new MyCustomSerializable(prev instanceof MyCustomSerializable ? prev : (prev ?? 3))\n  );\n  return <div onClick$={() => custom.value.inc()}>{custom.value.n}</div>;\n});\n```",
+      "content": "Creates a signal which holds a custom serializable value. It requires that the value implements the `CustomSerializable` type, which means having a function under the `[SerializeSymbol]` property that returns a serializable value when called.\n\nThe `fn` you pass is called with the result of the serialization (in the browser, only when the value is needed), or `undefined` when not yet initialized. If you refer to other signals, `fn` will be called when those change just like computed signals, and then the argument will be the previous output, not the serialized result.\n\nThis is useful when using third party libraries that use custom objects that are not serializable.\n\nNote that the `fn` is called lazily, so it won't impact container resume.\n\n\n```typescript\nuseSerializer$: typeof createSerializer$\n```\n\n\n\n```tsx\nclass MyCustomSerializable {\n  constructor(public n: number) {}\n  inc() {\n    this.n++;\n  }\n}\nconst Cmp = component$(() => {\n  const custom = useSerializer$({\n    deserialize: (data) => new MyCustomSerializable(data),\n    serialize: (data) => data.n,\n    initial: 2,\n  });\n  return <div onClick$={() => custom.value.inc()}>{custom.value.n}</div>;\n});\n```\n\n\nWhen using a Signal as the data to create the object, you may not need `serialize`<!-- -->. Furthermore, when the signal is updated, the serializer will be updated as well, and the current object will be passed as the second argument.\n\n```tsx\nconst Cmp = component$(() => {\n  const n = useSignal(2);\n  const custom = useSerializer$((_data, current) => {\n    if (current) {\n      current.n = n.value;\n      return current;\n    }\n    return new MyCustomSerializable(n.value);\n});\n  return <div onClick$={() => n.value++}>{custom.value.n}</div>;\n});\n```\n(note that in this example, the `{custom.value.n}` is not reactive, so the div text will not update)",
       "editUrl": "https://github.com/QwikDev/qwik/tree/main/packages/qwik/src/core/use/use-serialized.ts",
-      "mdFile": "core.useserialized_.md"
+      "mdFile": "core.useserializer_.md"
     },
     {
       "name": "useServerData",

packages/docs/src/routes/api/qwik/index.md

@@ -723,14 +723,12 @@ The name of the context.
 
 [Edit this section](https://github.com/QwikDev/qwik/tree/main/packages/qwik/src/core/use/use-context.ts)
 
-## createSerialized$
+## createSerializer$
 
-Create a signal that holds a custom serializable value. See [useSerialized$](#useserialized_) for more details.
+Create a signal that holds a custom serializable value. See [useSerializer$](#useserializer_) for more details.
 
 ```typescript
-createSerialized$: <T extends CustomSerializable<any, S>, S = T extends {
-    [SerializerSymbol]: (obj: any) => infer U;
-} ? U : unknown>(qrl: (data: S | undefined) => T) => T extends Promise<any> ? never : SerializedSignal<T>
+createSerializer$: <T, S>(arg: SerializerArg<T, S>) => T extends Promise<any> ? never : SerializerSignal<T>
 ```
 
 <table><thead><tr><th>
@@ -748,19 +746,19 @@ Description
 </th></tr></thead>
 <tbody><tr><td>
 
-qrl
+arg
 
 </td><td>
 
-(data: S \| undefined) =&gt; T
+SerializerArg&lt;T, S&gt;
 
 </td><td>
 
 </td></tr>
 </tbody></table>
 **Returns:**
 
-T extends Promise&lt;any&gt; ? never : SerializedSignal&lt;T&gt;
+T extends Promise&lt;any&gt; ? never : SerializerSignal&lt;T&gt;
 
 [Edit this section](https://github.com/QwikDev/qwik/tree/main/packages/qwik/src/core/signal/signal.public.ts)
 
@@ -4946,7 +4944,7 @@ _(Optional)_
 
 [Edit this section](https://github.com/QwikDev/qwik/tree/main/packages/qwik/src/core/use/use-resource-dollar.ts)
 
-## useSerialized$
+## useSerializer$
 
 Creates a signal which holds a custom serializable value. It requires that the value implements the `CustomSerializable` type, which means having a function under the `[SerializeSymbol]` property that returns a serializable value when called.
 
@@ -4957,7 +4955,7 @@ This is useful when using third party libraries that use custom objects that are
 Note that the `fn` is called lazily, so it won't impact container resume.
 
 ```typescript
-useSerialized$: typeof createSerialized$;
+useSerializer$: typeof createSerializer$;
 ```
 
 ```tsx
@@ -4966,21 +4964,35 @@ class MyCustomSerializable {
   inc() {
     this.n++;
   }
-  [SerializeSymbol]() {
-    return this.n;
-  }
 }
 const Cmp = component$(() => {
-  const custom = useSerialized$<MyCustomSerializable, number>(
-    (prev) =>
-      new MyCustomSerializable(
-        prev instanceof MyCustomSerializable ? prev : (prev ?? 3),
-      ),
-  );
+  const custom = useSerializer$({
+    deserialize: (data) => new MyCustomSerializable(data),
+    serialize: (data) => data.n,
+    initial: 2,
+  });
   return <div onClick$={() => custom.value.inc()}>{custom.value.n}</div>;
 });
 ```
 
+When using a Signal as the data to create the object, you may not need `serialize`. Furthermore, when the signal is updated, the serializer will be updated as well, and the current object will be passed as the second argument.
+
+```tsx
+const Cmp = component$(() => {
+  const n = useSignal(2);
+  const custom = useSerializer$((_data, current) => {
+    if (current) {
+      current.n = n.value;
+      return current;
+    }
+    return new MyCustomSerializable(n.value);
+  });
+  return <div onClick$={() => n.value++}>{custom.value.n}</div>;
+});
+```
+
+(note that in this example, the `{custom.value.n}` is not reactive, so the div text will not update)
+
 [Edit this section](https://github.com/QwikDev/qwik/tree/main/packages/qwik/src/core/use/use-serialized.ts)
 
 ## useServerData

packages/qwik/public.d.ts

@@ -6,7 +6,7 @@ export {
   ComputedSignal,
   ContextId,
   createComputed$,
-  createSerialized$,
+  createSerializer$,
   createContextId,
   createSignal,
   CSSProperties,
@@ -61,7 +61,7 @@ export {
   useOnDocument,
   useOnWindow,
   useResource$,
-  useSerialized$,
+  useSerializer$,
   useServerData,
   useSignal,
   useStore,

packages/qwik/src/core/api.md

@@ -120,20 +120,21 @@ export const createComputedQrl: <T>(qrl: QRL<() => T>) => ComputedSignal_2<T>;
 // @public
 export const createContextId: <STATE = unknown>(name: string) => ContextId<STATE>;
 
-// Warning: (ae-forgotten-export) The symbol "CustomSerializable" needs to be exported by the entry point index.d.ts
-// Warning: (ae-forgotten-export) The symbol "SerializedSignal" needs to be exported by the entry point index.d.ts
+// Warning: (ae-forgotten-export) The symbol "SerializerArg" needs to be exported by the entry point index.d.ts
+// Warning: (ae-forgotten-export) The symbol "SerializerSignal" needs to be exported by the entry point index.d.ts
 //
 // @public
-export const createSerialized$: <T extends CustomSerializable<any, S>, S = T extends {
-    [SerializerSymbol]: (obj: any) => infer U;
-} ? U : unknown>(qrl: (data: S | undefined) => T) => T extends Promise<any> ? never : SerializedSignal<T>;
+export const createSerializer$: <T, S>(arg: SerializerArg<T, S>) => T extends Promise<any> ? never : SerializerSignal<T>;
 
-// Warning: (ae-forgotten-export) The symbol "ConstructorFn" needs to be exported by the entry point index.d.ts
-// Warning: (ae-forgotten-export) The symbol "SerializedSignal_2" needs to be exported by the entry point index.d.ts
-// Warning: (ae-internal-missing-underscore) The name "createSerializedQrl" should be prefixed with an underscore because the declaration is marked as @internal
+// Warning: (ae-forgotten-export) The symbol "SerializerSignal_2" needs to be exported by the entry point index.d.ts
+// Warning: (ae-internal-missing-underscore) The name "createSerializerQrl" should be prefixed with an underscore because the declaration is marked as @internal
 //
 // @internal (undocumented)
-export const createSerializedQrl: <T extends CustomSerializable<T, S>, S>(qrl: QRL<ConstructorFn<T, S>>) => SerializedSignal_2<T>;
+export const createSerializerQrl: <T, S>(arg: QRL<{
+    serialize: (data: S | undefined) => T;
+    deserialize: (data: T) => S;
+    initial?: S;
+}>) => SerializerSignal_2<T, S>;
 
 // @public
 export const createSignal: {
@@ -1109,12 +1110,12 @@ export const useResource$: <T>(generatorFn: ResourceFn<T>, opts?: ResourceOption
 export const useResourceQrl: <T>(qrl: QRL<ResourceFn<T>>, opts?: ResourceOptions) => ResourceReturn<T>;
 
 // @public
-export const useSerialized$: typeof createSerialized$;
+export const useSerializer$: typeof createSerializer$;
 
-// Warning: (ae-internal-missing-underscore) The name "useSerializedQrl" should be prefixed with an underscore because the declaration is marked as @internal
+// Warning: (ae-internal-missing-underscore) The name "useSerializerQrl" should be prefixed with an underscore because the declaration is marked as @internal
 //
 // @internal (undocumented)
-export const useSerializedQrl: <F extends ConstructorFn<any, any>>(qrl: QRL<F>) => ReadonlySignal<unknown>;
+export const useSerializerQrl: <T, S>(qrl: QRL<SerializerArg<T, S>>) => ReadonlySignal<unknown>;
 
 // @public (undocumented)
 export function useServerData<T>(key: string): T | undefined;

packages/qwik/src/core/index.ts

@@ -110,7 +110,7 @@ export type { ContextId } from './use/use-context';
 export type { UseStoreOptions } from './use/use-store.public';
 export type { ComputedFn } from './use/use-computed';
 export { useComputedQrl } from './use/use-computed';
-export { useSerializedQrl, useSerialized$ } from './use/use-serialized';
+export { useSerializerQrl, useSerializer$ } from './use/use-serialized';
 export type { OnVisibleTaskOptions, VisibleTaskStrategy } from './use/use-visible-task';
 export { useVisibleTaskQrl } from './use/use-visible-task';
 export type { EagernessOptions, TaskCtx, TaskFn, Tracker, UseTaskOptions } from './use/use-task';
@@ -138,8 +138,8 @@ export {
   createSignal,
   createComputedQrl,
   createComputed$,
-  createSerializedQrl,
-  createSerialized$,
+  createSerializerQrl,
+  createSerializer$,
 } from './signal/signal.public';
 export { EffectPropData as _EffectData } from './signal/signal';
 

packages/qwik/src/core/shared/serialization.md

@@ -37,3 +37,17 @@ To avoid blocking the main thread on wake, we lazily restore the roots, with cac
 The serialized text is first parsed to get an array of encoded root data.
 
 Then, a proxy gets the raw data and returns an array that deserializes properties on demand and caches them. Objects are also lazily restored.
+
+## Out-of-order streaming
+
+when we allow out-of-order streaming but we want interactivity while the page is still streaming, we can't scan the state once for cycles/Promises/references, because later state might refer to earlier state.
+
+Therefore we must make the serializer single-pass. To do this, we could store each object as a root, but that will make references take more bytes. Instead, we could implement sub-paths for references, and when we encounter a reference, we output that instead.
+
+For later references it would be best to have a root reference, so we'd probably add a root for the sub-reference and later we can refer to that.
+
+We'll still need to keep track of Promises. We can do this by waiting, but that halts streaming. Instead, we could write a forward reference id, and when the promise is resolved, we store the result as the next root item. At the end of the stream we can emit a mapping from forward references to root index.
+
+Then later, when we send out-of-order state, it will append to the existing state on the client.
+
+This single-pass approach might just be better in general, because several bugs were found due to differences between the first and second pass code, and it allows streaming the data while the promises are being resolved.