Skip to content

Commit

Permalink
Allow the Monitor to purge unused queries
Browse files Browse the repository at this point in the history
This is done by running a query against the monitor's queryindex and collecting
all queries referenced, and then swapping this collection out for the previous
cache.

To be threadsafe, updates to the queryindex are now done while holding the
readlock of a ReadWriteLock, and the cache holds the writelock firstly during
the creation of an update log, and then when the update log is copied to the
new cache and the caches are swapped.
  • Loading branch information
romseygeek committed Aug 12, 2014
1 parent 28a3502 commit 9de359f
Show file tree
Hide file tree
Showing 10 changed files with 415 additions and 177 deletions.
68 changes: 68 additions & 0 deletions PURGE-NOTES
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
indexer thread purge thread

clone tlog
add docs <-- dropped!

collect

commit

clear tlog

swap tlog

-----------------------------------------

* add docs + commit is guarded by a read lock
* purge thread creates purge log
* purge log creation guarded by write lock
* if purge log exists, indexer adds its queries to it
* purge thread adds purge log contents after collectiong



add docs |_
add to purge log | <-- no purge log, so dropped!

-| create purge log

collect

commit
_| copy purge log
| swap cache

So I need to make adding docs to the cache and committing protected by the
readlock.

* update _parses_ the docs, but doesn't add them to the cache
* commit() now takes a list of CacheEntry objects

add & parse docs
-| create purge log

collect

_| copy purge log
| swap cache

add to cache |
add to purge log |-
commit |


* problem now is: what happens if another thread commits before the cache is
populated? We can get NPEs from the cache
- silently drop? We don't guarantee that the doc is there until update()
returns, so I think this is OK.

add & parse docs
delete docs
commit
match in index
retrieve from cache -> NPE!


add to cache |
add to purge log |-
commit |
5 changes: 5 additions & 0 deletions src/main/java/uk/co/flax/luwak/MatchError.java
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,9 @@ public MatchError(String queryId, Exception error) {
this.queryId = queryId;
this.error = error;
}

@Override
public String toString() {
return queryId + ":" + error.getMessage();
}
}
Loading

0 comments on commit 9de359f

Please sign in to comment.