Skip to content

Commit cdf8cde

Browse files
authored
Prune resolved errors (#69)
This pull request adds a new `ErrorTracker.Plugins.Pruner` which automatically prunes resolved errors. This plugin can be used automatically or manually. ℹ️ I've also added a new priv/repo/seeds.exs script that populates the db with some fake errors and occurrences. May be useful for testing the dashboard, pruning and other features easily. ## Automatic use Register the plugin under the ErrorTracker configuration. The pruner will run in the background every 30 minutes and remove up to 1000 resolved errors older than 5 minutes (this options can be overriden) ```elixir config :error_tracker, repo: MyApp.Repo, otp_app: :my_app, plugins: [ErrorTracker.Plugins.Pruner] ``` ## Manual use If you want to have more fine-grained control over when the pruner runs you can use the `ErrorTracker.Plugins.Pruner.prune_errors/1` function. In this case you must pass the `:limit` and `:max_age` options explicitly. You may want to call this function from an Oban job to leverage its cron-like capabilities. ```elixir defmodule MyApp.ErrorPruner do use Oban.Job def perform(%Job{}) do ErrorTracker.Plugins.Pruner.prune_errors(limit: 10_000, max_age: :timer.minutes(60)) end end ``` Closes #64
1 parent 90f6edc commit cdf8cde

File tree

10 files changed

+232
-4
lines changed

10 files changed

+232
-4
lines changed

guides/Getting Started.md

+9-1
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ Open the generated migration and call the `up` and `down` functions on `ErrorTra
5656
defmodule MyApp.Repo.Migrations.AddErrorTracker do
5757
use Ecto.Migration
5858

59-
def up, do: ErrorTracker.Migration.up(version: 2)
59+
def up, do: ErrorTracker.Migration.up(version: 3)
6060

6161
# We specify `version: 1` in `down`, to ensure we remove all migrations.
6262
def down, do: ErrorTracker.Migration.down(version: 1)
@@ -141,3 +141,11 @@ We currently do not support notifications out of the box.
141141
However, we provideo some detailed Telemetry events that you may use to implement your own notifications following your custom rules and notification channels.
142142

143143
If you want to take a look at the events you can attach to, take a look at `ErrorTracker.Telemetry` module documentation.
144+
145+
## Pruning resolved errors
146+
147+
By default errors are kept in the database indefinitely. This is not ideal for production
148+
environments where you may want to prune old errors that have been resolved.
149+
150+
The `ErrorTracker.Plugins.Pruner` module provides automatic pruning functionality with a configurable
151+
interval and error age.

lib/error_tracker/application.ex

+1-1
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ defmodule ErrorTracker.Application do
44
use Application
55

66
def start(_type, _args) do
7-
children = []
7+
children = Application.get_env(:error_tracker, :plugins, [])
88

99
attach_handlers()
1010

lib/error_tracker/migration/postgres.ex

+1-1
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ defmodule ErrorTracker.Migration.Postgres do
77
alias ErrorTracker.Migration.SQLMigrator
88

99
@initial_version 1
10-
@current_version 2
10+
@current_version 3
1111
@default_prefix "public"
1212

1313
@impl ErrorTracker.Migration
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
defmodule ErrorTracker.Migration.Postgres.V03 do
2+
@moduledoc false
3+
4+
use Ecto.Migration
5+
6+
def up(%{prefix: prefix}) do
7+
create_if_not_exists index(:error_tracker_errors, [:last_occurrence_at], prefix: prefix)
8+
end
9+
10+
def down(%{prefix: prefix}) do
11+
drop_if_exists index(:error_tracker_errors, [:last_occurrence_at], prefix: prefix)
12+
end
13+
end

lib/error_tracker/migration/sqlite.ex

+1-1
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ defmodule ErrorTracker.Migration.SQLite do
77
alias ErrorTracker.Migration.SQLMigrator
88

99
@initial_version 2
10-
@current_version 2
10+
@current_version 3
1111

1212
@impl ErrorTracker.Migration
1313
def up(opts) do
+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
defmodule ErrorTracker.Migration.SQLite.V03 do
2+
@moduledoc false
3+
4+
use Ecto.Migration
5+
6+
def up(_opts) do
7+
create_if_not_exists index(:error_tracker_errors, [:last_occurrence_at])
8+
end
9+
10+
def down(_opts) do
11+
drop_if_exists index(:error_tracker_errors, [:last_occurrence_at])
12+
end
13+
end

lib/error_tracker/plugins/pruner.ex

+142
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
defmodule ErrorTracker.Plugins.Pruner do
2+
@moduledoc """
3+
Periodically delete resolved errors based on their age.
4+
5+
Pruning allows you to keep your database size under control by removing old errors that are not
6+
needed anymore.
7+
8+
## Using the pruner
9+
10+
To enable the pruner you must register the plugin in the ErrorTracker configuration. This will use
11+
the default options, which is to prune errors resolved after 5 minutes.
12+
13+
config :error_tracker,
14+
plugins: [ErrorTracker.Plugins.Pruner]
15+
16+
You can override the default options by passing them as an argument when registering the plugin.
17+
18+
config :error_tracker,
19+
plugins: [{ErrorTracker.Plugins.Pruner, max_age: :timer.minutes(30)}]
20+
21+
## Options
22+
23+
- `:limit` - the maximum number of errors to prune on each execution. Occurrences are removed
24+
along the errors. The default is 200 to prevent timeouts and unnecesary database load.
25+
26+
- `:max_age` - the number of milliseconds after a resolved error may be pruned. The default is 24
27+
hours.
28+
29+
- `:interval` - the interval in milliseconds between pruning runs. The default is 30 minutes.
30+
31+
You may find the `:timer` module functions useful to pass readable values to the `:max_age` and
32+
`:interval` options.
33+
34+
## Manual pruning
35+
36+
In certain cases you may prefer to run the pruner manually. This can be done by calling the
37+
`prune_errors/2` function from your application code. This function supports the `:limit` and
38+
`:max_age` options as described above.
39+
40+
For example, you may call this function from an Oban worker so you can leverage Oban's cron
41+
capabilities and have a more granular control over when pruning is run.
42+
43+
defmodule MyApp.ErrorPruner do
44+
use Oban.Job
45+
46+
def perform(%Job{}) do
47+
ErrorTracker.Plugins.Pruner.prune_errors(limit: 10_000, max_age: :timer.minutes(60))
48+
end
49+
end
50+
"""
51+
use GenServer
52+
53+
import Ecto.Query
54+
55+
alias ErrorTracker.Error
56+
alias ErrorTracker.Occurrence
57+
alias ErrorTracker.Repo
58+
59+
@doc """
60+
Prunes resolved errors.
61+
62+
You do not need to use this function if you activate the Pruner plugin. This function is exposed
63+
only for advanced use cases and Oban integration.
64+
65+
## Options
66+
67+
- `:limit` - the maximum number of errors to prune on each execution. Occurrences are removed
68+
along the errors. The default is 200 to prevent timeouts and unnecesary database load.
69+
70+
- `:max_age` - the number of milliseconds after a resolved error may be pruned. The default is 24
71+
hours. You may find the `:timer` module functions useful to pass readable values to this option.
72+
"""
73+
@spec prune_errors(keyword()) :: {:ok, list(Error.t())}
74+
def prune_errors(opts \\ []) do
75+
limit = opts[:limit] || raise ":limit option is required"
76+
max_age = opts[:max_age] || raise ":max_age option is required"
77+
time = DateTime.add(DateTime.utc_now(), -max_age, :millisecond)
78+
79+
errors =
80+
Repo.all(
81+
from error in Error,
82+
select: [:id, :kind, :source_line, :source_function],
83+
where: error.status == :resolved,
84+
where: error.last_occurrence_at < ^time,
85+
limit: ^limit
86+
)
87+
88+
if Enum.any?(errors) do
89+
_pruned_occurrences_count =
90+
errors
91+
|> Ecto.assoc(:occurrences)
92+
|> prune_occurrences()
93+
|> Enum.sum()
94+
95+
Repo.delete_all(from error in Error, where: error.id in ^Enum.map(errors, & &1.id))
96+
end
97+
98+
{:ok, errors}
99+
end
100+
101+
defp prune_occurrences(occurrences_query) do
102+
Stream.unfold(occurrences_query, fn occurrences_query ->
103+
occurrences_ids =
104+
Repo.all(from occurrence in occurrences_query, select: occurrence.id, limit: 1000)
105+
106+
case Repo.delete_all(from o in Occurrence, where: o.id in ^occurrences_ids) do
107+
{0, _} -> nil
108+
{deleted, _} -> {deleted, occurrences_query}
109+
end
110+
end)
111+
end
112+
113+
def start_link(state \\ []) do
114+
GenServer.start_link(__MODULE__, state, name: __MODULE__)
115+
end
116+
117+
@impl GenServer
118+
@doc false
119+
def init(state \\ []) do
120+
state = %{
121+
limit: state[:limit] || 200,
122+
max_age: state[:max_age] || :timer.hours(24),
123+
interval: state[:interval] || :timer.minutes(30)
124+
}
125+
126+
{:ok, schedule_prune(state)}
127+
end
128+
129+
@impl GenServer
130+
@doc false
131+
def handle_info(:prune, state) do
132+
{:ok, _pruned} = prune_errors(state)
133+
134+
{:noreply, schedule_prune(state)}
135+
end
136+
137+
defp schedule_prune(state = %{interval: interval}) do
138+
Process.send_after(self(), :prune, interval)
139+
140+
state
141+
end
142+
end

lib/error_tracker/repo.ex

+4
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,10 @@ defmodule ErrorTracker.Repo do
2525
dispatch(:all, [queryable], opts)
2626
end
2727

28+
def delete_all(queryable, opts \\ []) do
29+
dispatch(:delete_all, [queryable], opts)
30+
end
31+
2832
def aggregate(queryable, aggregate, opts \\ []) do
2933
dispatch(:aggregate, [queryable, aggregate], opts)
3034
end

mix.exs

+3
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,9 @@ defmodule ErrorTracker.MixProject do
6565
ErrorTracker.Integrations.Phoenix,
6666
ErrorTracker.Integrations.Plug
6767
],
68+
Plugins: [
69+
ErrorTracker.Plugins.Pruner
70+
],
6871
Schemas: [
6972
ErrorTracker.Error,
7073
ErrorTracker.Occurrence,

priv/repo/seeds.exs

+45
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
adapter =
2+
case Application.get_env(:error_tracker, :ecto_adapter) do
3+
:postgres -> Ecto.Adapters.Postgres
4+
:sqlite3 -> Ecto.Adapters.SQLite3
5+
end
6+
7+
defmodule ErrorTrackerDev.Repo do
8+
use Ecto.Repo, otp_app: :error_tracker, adapter: adapter
9+
end
10+
11+
ErrorTrackerDev.Repo.start_link()
12+
13+
ErrorTrackerDev.Repo.delete_all(ErrorTracker.Error)
14+
15+
errors =
16+
for i <- 1..100 do
17+
%{
18+
kind: "Error #{i}",
19+
reason: "Reason #{i}",
20+
source_line: "line",
21+
source_function: "function",
22+
status: :unresolved,
23+
fingerprint: "#{i}",
24+
last_occurrence_at: DateTime.utc_now(),
25+
inserted_at: DateTime.utc_now(),
26+
updated_at: DateTime.utc_now()
27+
}
28+
end
29+
30+
{_, errors} = dbg(ErrorTrackerDev.Repo.insert_all(ErrorTracker.Error, errors, returning: [:id]))
31+
32+
for error <- errors do
33+
occurrences =
34+
for _i <- 1..200 do
35+
%{
36+
context: %{},
37+
reason: "REASON",
38+
stacktrace: %ErrorTracker.Stacktrace{},
39+
error_id: error.id,
40+
inserted_at: DateTime.utc_now()
41+
}
42+
end
43+
44+
ErrorTrackerDev.Repo.insert_all(ErrorTracker.Occurrence, occurrences)
45+
end

0 commit comments

Comments
 (0)