Skip to content

Commit

Permalink
Add task to create all indexes (#892)
Browse files Browse the repository at this point in the history
* Add task to create all indexes

* Update changelog and readme

* Address review comments

* Control verbosity of the index creation
  • Loading branch information
Danil Nurgaliev authored Aug 29, 2023
1 parent 94d8e11 commit 541b696
Show file tree
Hide file tree
Showing 5 changed files with 82 additions and 0 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

### New Features

* [#888](https://github.com/toptal/chewy/pull/892): Rake task to create missing indexes ([@konalegi](https://github.com/konalegi))

### Changes

### Bugs Fixed
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1197,6 +1197,10 @@ Right now the approach is that if some data had been updated, but index definiti

Also, there is always full reset alternative with `rake chewy:reset`.

#### `chewy:create_missing_indexes`

This rake task creates newly defined indexes in ElasticSearch and skips existing ones. Useful for production-like environments.

#### Parallelizing rake tasks

Every task described above has its own parallel version. Every parallel rake task takes the number for processes for execution as the first argument and the rest of the arguments are exactly the same as for the non-parallel task version.
Expand Down
18 changes: 18 additions & 0 deletions lib/chewy/rake_helper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,24 @@ def delete_by_query_options_from_env(env)
end
end

def create_missing_indexes!(output: $stdout, env: ENV)
subscribed_task_stats(output) do
Chewy.eager_load!
all_indexes = Chewy::Index.descendants
all_indexes -= [Chewy::Stash::Journal] unless Chewy.configuration[:journal]
all_indexes.each do |index|
if index.exists?
output.puts "#{index.name} already exists, skipping" if env['VERBOSE']
next
end

index.create!

output.puts "#{index.name} index successfully created"
end
end
end

def normalize_indexes(*identifiers)
identifiers.flatten(1).map { |identifier| normalize_index(identifier) }
end
Expand Down
5 changes: 5 additions & 0 deletions lib/tasks/chewy.rake
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,11 @@ namespace :chewy do
Chewy::RakeHelper.update_mapping(name: args[:index_name])
end

desc 'Creates missing indexes'
task create_missing_indexes: :environment do
Chewy::RakeHelper.create_missing_indexes!
end

namespace :parallel do
desc 'Parallel version of `rake chewy:reset`'
task reset: :environment do |_task, args|
Expand Down
53 changes: 53 additions & 0 deletions spec/chewy/rake_helper_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -470,6 +470,59 @@
end
end

describe '.create_missing_indexes!' do
before do
[CountriesIndex, Chewy::Stash::Specification].map(&:create!)

# To avoid flaky issues when previous specs were run
expect(Chewy::Index).to receive(:descendants).and_return(
[
UsersIndex,
CountriesIndex,
CitiesIndex,
Chewy::Stash::Specification,
Chewy::Stash::Journal
]
)
end

specify do
output = StringIO.new
described_class.create_missing_indexes!(output: output)
expect(CitiesIndex.exists?).to be_truthy
expect(UsersIndex.exists?).to be_truthy
expect(Chewy::Stash::Journal.exists?).to be_falsey
expect(output.string).to match(Regexp.new(<<-OUTPUT, Regexp::MULTILINE))
UsersIndex index successfully created
CitiesIndex index successfully created
Total: \\d+s\\Z
OUTPUT
end

context 'when verbose' do
specify do
output = StringIO.new
described_class.create_missing_indexes!(output: output, env: {'VERBOSE' => '1'})
expect(output.string).to match(Regexp.new(<<-OUTPUT, Regexp::MULTILINE))
UsersIndex index successfully created
CountriesIndex already exists, skipping
CitiesIndex index successfully created
Chewy::Stash::Specification already exists, skipping
Total: \\d+s\\Z
OUTPUT
end
end

context 'when journaling is enabled' do
before { Chewy.config.settings[:journal] = true }
after { Chewy.config.settings.delete(:journal) }
specify do
described_class.create_missing_indexes!(output: StringIO.new)
expect(Chewy::Stash::Journal.exists?).to be_truthy
end
end
end

describe '.journal_create' do
specify do
output = StringIO.new
Expand Down

0 comments on commit 541b696

Please sign in to comment.