Elasticsearch migration from 5.6 to 6.8

[jphilou]

ElasticSearch Upgrade from v5.6 to V6.8 for docker env. like dev one.

The docker-compose file has been updated in order to facilitate and automotive plugins installation and prepare the environment for migration.
The ElasticSearch 5.6 is still build and run but would not be the main ES for the service. It could be decommissioned after migration.
docker-compose file contains as well :

• api building with integration of the requirements.txt for python pip installation (component updates)
• ES 5.6 listening on port 9201 instead of 9200 : > elasticsearch26
• redis no change
• postgresql no change
• ES 6.8 listening on port 9200 and 9300 (elasticsearch6.Dockerfile) > elasticsearch
• Logstash 6.8 for data migration purpose. (logstash6.Dockerfile)

migration steps

1. build the environment : docker-compose build
2. Launch the environment : docker-compose up -d
3. Create the index and its mapping : ./scripts/esUpdateMappings56.sh
4. Data Migration : docker-compose start logstash6 It will stop by itself when migration would be achieved
5. postgresql shema up to date: docker-compose exec api .build/venv/bin/alembic upgrade head

post migration

if all the process has been passed without issues. you can comment or remove in the docker-compose.yml services :

• logstash
• elasticsearch56

[codacy-production]

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation	Diff coverage
✅ -0.02%	✅ 92.27%

Coverage variation details

	Coverable lines	Covered lines	Coverage
Common ancestor commit (24e010e)	19853	18407	92.72%
Head commit (ce8c934)	19934 (+81)	18479 (+72)	92.70% (-0.02%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details

	Coverable lines	Covered lines	Diff coverage
Pull request (#1813)	194	179	92.27%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

---

_{🚀 Don’t miss a bit, follow what’s new on Codacy.}

_{Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more}