Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TM.anomaly: fix score for likelihood anomaly modes #666

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

TM.anomaly: fix score for likelihood anomaly modes #666

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7fb6912
TM.anomaly: improve doc for anomaly modes
breznak Sep 16, 2019
9c1a2cf
Merge branch 'master_community' into anomaly_likelihood_docs
breznak Sep 18, 2019
9e0245f
TM.anomaly: fix scores for likelihoods
breznak Sep 18, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions src/htm/algorithms/TemporalMemory.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -506,7 +506,8 @@ void TemporalMemory::compute(const SDR &activeColumns,
const Real raw = computeRawAnomalyScore(
activeColumns,
cellsToColumns( getPredictiveCells() ));
tmAnomaly_.anomaly_ = tmAnomaly_.anomalyLikelihood_.anomalyProbability(raw);
tmAnomaly_.anomaly_ = 1.0f - tmAnomaly_.anomalyLikelihood_.anomalyProbability(raw); // AnomalyLikelihood returs a likelihood of the score (given past scores),
// and we want to detect unlikely scores, aka big changes -> 1.0-likelihood represents "anomaly"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, I'm not sure this is correct at all.
Revisitiing anomaly likelihood in original Numenta's py code:

def anomalyProbability(self, value, anomalyScore, timestamp=None):
"""
Compute the probability that the current value plus anomaly score represents
an anomaly given the historical distribution of anomaly scores. The closer
the number is to 1, the higher the chance it is an anomaly.
...

And both codes (Numenta's py, and ours cpp) internally toggle likelihoods to anomaly before returning
likelihood = 1.0 - likelihoods[0], so doing another 1-anomalyProbability() switches back to likelihoods[0]

So, according to docs and logic of the code, the TM.anomaly should represent anomalyProbability() in AnomalyLikelihood.

@Thanh-Binh can you describe in detail your experiment and where you see errors? We must replicate it here, ideally even simpler, synthetic test for likelihood: anomaly scores high 1.0, 0.9, 0.98, 1.0, ... -> anomalyProbability() should drop. Same for low score: 0.0, 0.1, 0.05, 0.0,.. -> anProb() drops to 0.
For a sudden change (high anomalies -> low, and vice versa) the anProb() should be high.

} break;

case ANMode::LOGLIKELIHOOD: {
Expand All @@ -515,7 +516,7 @@ void TemporalMemory::compute(const SDR &activeColumns,
cellsToColumns( getPredictiveCells() ));
const Real like = tmAnomaly_.anomalyLikelihood_.anomalyProbability(raw);
const Real log = tmAnomaly_.anomalyLikelihood_.computeLogLikelihood(like);
tmAnomaly_.anomaly_ = log;
tmAnomaly_.anomaly_ = 1.0f - log;
} break;
// TODO: Update mean & standard deviation of anomaly here.
};
Expand Down
19 changes: 17 additions & 2 deletions src/htm/algorithms/TemporalMemory.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -688,8 +688,23 @@ class TemporalMemory : public Serializable
* anomaly score computed for the current inputs
* (auto-updates after each call to TM::compute())
*
* @return a float value from computeRawAnomalyScore()
* from Anomaly.hpp
* Note, if you use manual learning methods for TM (activateDendrites, etc.) the scores
* will not be in sync (works only if `TM.compute` is used).
*
* Options `ANMode` {DISABLED, RAW, LIKELIHOOD, LOGLIKELIHOOD} passed during constructor
* of this class affect the computation of anomaly here.
* - "DISABLED": always returns 0.5f, no computation is performed (use if you don't care for anomaly
* and need maximum speed).
* - "RAW": return value from `computeRawAnomalyScore()` from `Anomaly.hpp`, default.
* - "LIKELIHOOD": Compute the probability that the current anomaly score represents
* an anomaly given the historical distribution of anomaly scores. The closer
* the number is to 1, the higher the chance it is an anomaly.
* Computed from `anomalyProbability()` in `AnomalyLikelihood.hpp`.
* - "LOGLIKELIHOOD": Compute a log scale representation of the likelihood value. This mode performs
* `computeLogLikelihood()` from `AnomalyLikelihood.hpp`
*
* @return a float value based on `ANMode` argument used in constructor of the TM class.
* By default return value from `computeRawAnomalyScore()` from `Anomaly.hpp`
*/
const Real &anomaly = tmAnomaly_.anomaly_; //this is position dependant, the struct anomaly_tm must be defined before this use,
// otherwise this surprisingly compiles, but a call to `tmAnomaly_.anomaly` segfaults!
Expand Down