RFC consistent view update & tree swapping (#8)

* Implement the ability to swap the router tree atomically * Field alignement * Fix comment * Final implementation of the new Tree API * Improve docs * Add tests for Lookup, Has and Reverse public API * Add Benchmark for Lookup function. * Use an Option interface instead of func(*Router) * docs: Enhance explanation of Fox's dynamic routing capabilities and performance focus --------- Co-authored-by: tigerwill90 <[email protected]>
tigerwill90 · Mar 27, 2023 · f8fb803 · f8fb803
1 parent eea87e7
commit f8fb803
Show file tree

Hide file tree

Showing 12 changed files with 1,371 additions and 919 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1,3 +1,5 @@
 .idea
 .prof
-.test
+.test
+go.work
+go.work.sum
diff --git a/README.md b/README.md
@@ -11,6 +11,9 @@ reads** while allowing **concurrent writes**.
 The router tree is optimized for high-concurrency and high performance reads, and low-concurrency write. Fox has a small memory footprint, and 
 in many case, it does not do a single heap allocation while handling request.
 
+Fox supports various use cases, but it is especially designed for applications that require changes at runtime to their 
+routing structure based on user input, configuration changes, or other runtime events.
+
 ## Disclaimer
 The current api is not yet stabilize. Breaking changes may occur before `v1.0.0` and will be noted on the release note.
 
@@ -23,7 +26,7 @@ name. Due to Fox design, wildcard route are cheap and scale really well.
 
 **Detect panic:** You can register a fallback handler that is fire in case of panics occurring during handling an HTTP request.
 
-**Get the current route:** You can easily retrieve the route for the current matched request. This actually makes it easier to integrate
+**Get the current route:** You can easily retrieve the route of the matched request. This actually makes it easier to integrate
 observability middleware like open telemetry (disable by default).
 
 **Only explicit matches:** Inspired from [httprouter](https://github.com/julienschmidt/httprouter), a request can only match
@@ -116,7 +119,7 @@ Pattern /users/uuid_:id
 /users/uuid             no match
 ```
 
-#### Catch all parameter
+### Catch all parameter
 Catch-all parameters can be used to match everything at the end of a route. The placeholder start with `*` followed by a name.
 ```
 Pattern /src/*filepath
@@ -140,9 +143,33 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request, params fox.P
 }
 ```
 
-### Adding, updating and removing route
-In this example, the handler for `route/:action` allow to dynamically register, update and remove handler for the given route and method.
-Due to Fox design, those actions are perfectly safe and may be executed concurrently. 
+## Concurrency
+Fox implements a [Concurrent Radix Tree](https://github.com/npgall/concurrent-trees/blob/master/documentation/TreeDesign.md) that supports **lock-free** 
+reads while allowing **concurrent writes**, by calculating the changes which would be made to the tree were it mutable, and assembling those changes 
+into a **patch**, which is then applied to the tree in a **single atomic operation**.
+
+For example, here we are inserting the new key `toast` into to the tree which require an existing node to be split:
+
+<p align="center" width="100%">
+    <img width="100%" src="assets/tree-apply-patch.png">
+</p>
+
+When traversing the tree during a patch, reading threads will either see the **old version** or the **new version** of the (sub-)tree, but both version are 
+consistent view of the tree.
+
+#### Other key points
+
+- Routing requests is lock-free (reading thread never block, even while writes are ongoing)
+- The router always see a consistent version of the tree while routing request
+- Reading threads do not block writing threads (adding, updating or removing a handler can be done concurrently)
+- Writing threads block each other but never block reading threads
+
+As such threads that route requests should never encounter latency due to ongoing writes or other concurrent readers.
+
+### Managing routes a runtime
+#### Routing mutation
+In this example, the handler for `routes/:action` allow to dynamically register, update and remove handler for the
+given route and method. Thanks to Fox's design, those actions are perfectly safe and may be executed concurrently.
 
 ```go
 package main
@@ -214,35 +241,115 @@ func Must(err error) {
 }
 ```
 
-## Concurrency
-Fox implements a [Concurrent Radix Tree](https://github.com/npgall/concurrent-trees/blob/master/documentation/TreeDesign.md) that supports **lock-free** 
-reads while allowing **concurrent writes**, by calculating the changes which would be made to the tree were it mutable, and assembling those changes 
-into a **patch**, which is then applied to the tree in a **single atomic operation**.
+#### Tree swapping
+Fox also enables you to replace the entire tree in a single atomic operation using the `Store` and `Swap` methods.
+Note that router's options apply automatically on the new tree.
+````go
+package main
 
-For example, here we are inserting the new key `toast` into to the tree which require an existing node to be split:
+import (
+	"fox-by-example/db"
+	"github.com/tigerwill90/fox"
+	"html/template"
+	"io"
+	"log"
+	"net/http"
+	"strings"
+	"time"
+)
 
-<p align="center" width="100%">
-    <img width="100%" src="assets/tree-apply-patch.png">
-</p>
+type HtmlRenderer struct {
+	Template template.HTML
+}
 
-When traversing the tree during a patch, reading threads will either see the **old version** or the **new version** of the (sub-)tree, but both version are 
-consistent view of the tree.
+func (h *HtmlRenderer) ServeHTTP(w http.ResponseWriter, r *http.Request, params fox.Params) {
+	log.Printf("matched route: %s", params.Get(fox.RouteKey))
+	w.Header().Set("Content-Type", "text/html; charset=utf-8")
+	_, _ = io.Copy(w, strings.NewReader(string(h.Template)))
+}
 
-#### Other key points
+func main() {
+	r := fox.New(fox.WithSaveMatchedRoute(true))
 
-- Routing requests is lock-free (reading thread never block, even while writes are ongoing)
-- The router always see a consistent version of the tree while routing request
-- Reading threads do not block writing threads (adding, updating or removing a handler can be done concurrently)
-- Writing threads block each other but never block reading threads
+	routes := db.GetRoutes()
+	for _, rte := range routes {
+		Must(r.Handler(rte.Method, rte.Path, &HtmlRenderer{Template: rte.Template}))
+	}
 
-As such threads that route requests should never encounter latency due to ongoing writes or other concurrent readers.
+	go Reload(r)
+
+	log.Fatalln(http.ListenAndServe(":8080", r))
+}
+
+func Reload(r *fox.Router) {
+	for range time.Tick(10 * time.Second) {
+		routes := db.GetRoutes()
+		tree := r.NewTree()
+		for _, rte := range routes {
+			if err := tree.Handler(rte.Method, rte.Path, &HtmlRenderer{Template: rte.Template}); err != nil {
+				log.Printf("error reloading route: %s\n", err)
+				continue
+			}
+		}
+		// Replace the currently in-use routing tree with the new provided.
+		r.Use(tree)
+		log.Println("route reloaded")
+	}
+}
+
+func Must(err error) {
+	if err != nil {
+		panic(err)
+	}
+}
+````
+
+#### Advanced usage: consistent view updates
+In certain situations, it's necessary to maintain a consistent view of the tree while performing updates.
+The `Tree` API allow you to take control of the internal `sync.Mutex` to prevent concurrent writes from
+other threads. **Remember that all write operation should be run serially.**
+
+In the following example, the `Upsert` function needs to perform a lookup on the tree to check if a handler
+is already registered for the provided method and path. By locking the `Tree`, this operation ensures
+atomicity, as it prevents other threads from modifying the tree between the lookup and the write operation.
+Note that all read operation on the tree remain lock-free.
+````go
+func Upsert(t *fox.Tree, method, path string, handler fox.Handler) error {
+    t.Lock()
+    defer t.Unlock()
+    if fox.Has(t, method, path) {
+        return t.Update(method, path, handler)
+    }
+    return t.Handler(method, path, handler)
+}
+````
+
+#### Concurrent safety and proper usage of Tree APIs
+Some important consideration to keep in mind when using `Tree` API. Each instance as its own `sync.Mutex` and `sync.Pool` 
+that may be used to serialize write and reduce memory allocation. Since the router tree may be swapped at any
+given time, you **MUST always copy the pointer locally** to avoid inadvertently releasing Params to the wrong pool 
+or worst, causing a deadlock by locking/unlocking the wrong `Tree`.
+
+````go
+// Good
+t := r.Tree()
+t.Lock()
+defer t.Unlock()
+
+// Dramatically bad, may cause deadlock:
+r.Tree().Lock()
+defer r.Tree().Unlock()
+```` 
+
+This principle also applies to the `fox.Lookup` function, which requires releasing the `fox.Params` slice by calling `params.Free(tree)`.
+Always ensure that the `Tree` pointer passed as a parameter to `params.Free` is the same as the one passed to the `fox.Lookup` function.
 
 ## Working with http.Handler
 Fox itself implements the `http.Handler` interface which make easy to chain any compatible middleware before the router. Moreover, the router
 provides convenient `fox.WrapF` and `fox.WrapH` adapter to be use with `http.Handler`. Named and catch all parameters are forwarded via the
 request context
 ```go
-_ = r.Handler(http.MethodGet, "/users/:id", fox.WrapF(func(w http.ResponseWriter, r *http.Request) {
+_ = r.Tree().Handler(http.MethodGet, "/users/:id", fox.WrapF(func(w http.ResponseWriter, r *http.Request) {
     params := fox.ParamsFromContext(r.Context())
     _, _ = fmt.Fprintf(w, "user id: %s\n", params.Get("id"))
 }))

diff --git a/iter.go b/iter.go
@@ -5,7 +5,7 @@ import (
 )
 
 type Iterator struct {
-	r       *Router
+	tree    *Tree
 	method  string
 	current *node
 	stacks  []stack
@@ -17,14 +17,14 @@ type Iterator struct {
 // An Iterator is safe to use when the router is serving request, when routing updates are ongoing or
 // in parallel with other Iterators. Note that changes that happen while iterating over routes may not be reflected
 // by the Iterator. This api is EXPERIMENTAL and is likely to change in future release.
-func (fox *Router) NewIterator() *Iterator {
+func NewIterator(t *Tree) *Iterator {
 	return &Iterator{
-		r: fox,
+		tree: t,
 	}
 }
 
 func (it *Iterator) methods() map[string]*node {
-	nds := *it.r.trees.Load()
+	nds := it.tree.load()
 	m := make(map[string]*node, len(nds))
 	for i := range nds {
 		if len(nds[i].children) > 0 {
@@ -40,7 +40,7 @@ func (it *Iterator) SeekPrefix(key string) {
 	nds := it.methods()
 	keys := make([]string, 0, len(nds))
 	for method, n := range nds {
-		result := it.r.search(n, key)
+		result := it.tree.search(n, key)
 		if result.isExactMatch() || result.isKeyMidEdge() {
 			nds[method] = result.matched
 			keys = append(keys, method)
@@ -82,7 +82,7 @@ func (it *Iterator) SeekMethodPrefix(method, key string) {
 	stacks := make([]stack, 0, 1)
 	n, ok := nds[method]
 	if ok {
-		result := it.r.search(n, key)
+		result := it.tree.search(n, key)
 		if result.isExactMatch() || result.isKeyMidEdge() {
 			stacks = append(stacks, stack{
 				edges:  []*node{result.matched},
@@ -194,8 +194,8 @@ type rawIterator struct {
 }
 
 type stack struct {
-	edges  []*node
 	method string
+	edges  []*node
 }
 
 func (it *rawIterator) fullPath() string {

diff --git a/iter_test.go b/iter_test.go
@@ -11,16 +11,16 @@ import (
 var routesCases = []string{"/fox/router", "/foo/bar/:baz", "/foo/bar/:baz/:name", "/john/doe/*args", "/john/doe"}
 
 func TestIterator_Rewind(t *testing.T) {
-	r := New()
+	tree := New().Tree()
 	for _, rte := range routesCases {
-		require.NoError(t, r.Handler(http.MethodGet, rte, emptyHandler))
-		require.NoError(t, r.Handler(http.MethodPost, rte, emptyHandler))
-		require.NoError(t, r.Handler(http.MethodHead, rte, emptyHandler))
+		require.NoError(t, tree.Handler(http.MethodGet, rte, emptyHandler))
+		require.NoError(t, tree.Handler(http.MethodPost, rte, emptyHandler))
+		require.NoError(t, tree.Handler(http.MethodHead, rte, emptyHandler))
 	}
 
 	results := make(map[string][]string)
 
-	it := r.NewIterator()
+	it := NewIterator(tree)
 	for it.Rewind(); it.Valid(); it.Next() {
 		assert.NotNil(t, it.Handler())
 		results[it.Method()] = append(results[it.method], it.Path())
@@ -32,16 +32,16 @@ func TestIterator_Rewind(t *testing.T) {
 }
 
 func TestIterator_SeekMethod(t *testing.T) {
-	r := New()
+	tree := New().Tree()
 	for _, rte := range routesCases {
-		require.NoError(t, r.Handler(http.MethodGet, rte, emptyHandler))
-		require.NoError(t, r.Handler(http.MethodPost, rte, emptyHandler))
-		require.NoError(t, r.Handler(http.MethodHead, rte, emptyHandler))
+		require.NoError(t, tree.Handler(http.MethodGet, rte, emptyHandler))
+		require.NoError(t, tree.Handler(http.MethodPost, rte, emptyHandler))
+		require.NoError(t, tree.Handler(http.MethodHead, rte, emptyHandler))
 	}
 
 	results := make(map[string][]string)
 
-	it := r.NewIterator()
+	it := NewIterator(tree)
 	for it.SeekMethod(http.MethodHead); it.Valid(); it.Next() {
 		assert.NotNil(t, it.Handler())
 		results[it.Method()] = append(results[it.method], it.Path())
@@ -52,17 +52,17 @@ func TestIterator_SeekMethod(t *testing.T) {
 }
 
 func TestIterator_SeekPrefix(t *testing.T) {
-	r := New()
+	tree := New().Tree()
 	for _, rte := range routesCases {
-		require.NoError(t, r.Handler(http.MethodGet, rte, emptyHandler))
-		require.NoError(t, r.Handler(http.MethodPost, rte, emptyHandler))
-		require.NoError(t, r.Handler(http.MethodHead, rte, emptyHandler))
+		require.NoError(t, tree.Handler(http.MethodGet, rte, emptyHandler))
+		require.NoError(t, tree.Handler(http.MethodPost, rte, emptyHandler))
+		require.NoError(t, tree.Handler(http.MethodHead, rte, emptyHandler))
 	}
 
 	want := []string{"/foo/bar/:baz", "/foo/bar/:baz/:name"}
 	results := make(map[string][]string)
 
-	it := r.NewIterator()
+	it := NewIterator(tree)
 	for it.SeekPrefix("/foo"); it.Valid(); it.Next() {
 		assert.NotNil(t, it.Handler())
 		results[it.Method()] = append(results[it.method], it.Path())
@@ -74,17 +74,17 @@ func TestIterator_SeekPrefix(t *testing.T) {
 }
 
 func TestIterator_SeekMethodPrefix(t *testing.T) {
-	r := New()
+	tree := New().Tree()
 	for _, rte := range routesCases {
-		require.NoError(t, r.Handler(http.MethodGet, rte, emptyHandler))
-		require.NoError(t, r.Handler(http.MethodPost, rte, emptyHandler))
-		require.NoError(t, r.Handler(http.MethodHead, rte, emptyHandler))
+		require.NoError(t, tree.Handler(http.MethodGet, rte, emptyHandler))
+		require.NoError(t, tree.Handler(http.MethodPost, rte, emptyHandler))
+		require.NoError(t, tree.Handler(http.MethodHead, rte, emptyHandler))
 	}
 
 	want := []string{"/foo/bar/:baz", "/foo/bar/:baz/:name"}
 	results := make(map[string][]string)
 
-	it := r.NewIterator()
+	it := NewIterator(tree)
 	for it.SeekMethodPrefix(http.MethodHead, "/foo"); it.Valid(); it.Next() {
 		results[it.Method()] = append(results[it.method], it.Path())
 	}
@@ -93,9 +93,9 @@ func TestIterator_SeekMethodPrefix(t *testing.T) {
 	assert.ElementsMatch(t, want, results[http.MethodHead])
 }
 
-func ExampleRouter_NewIterator() {
+func ExampleNewIterator() {
 	r := New()
-	it := r.NewIterator()
+	it := NewIterator(r.Tree())
 
 	// Iterate over all routes
 	for it.Rewind(); it.Valid(); it.Next() {