Launch AWS ElasticSearch Service per tenant #2
Conversation
tieujason330
commented
Dec 7, 2021
tieujason330
commented
- based off Elasticsearch integration awslabs/aws-saas-boost#114
- Launch AWS ElasticSearch 7.10 Service per tenant
- env variables used
- ES_CLUSTER_ENDPOINT
- ES_USERNAME
- ES_PASSWORD
| Type: Number | ||
| Default: '2' | ||
| MinValue: '1' | ||
| MaxValue: '20' |
There was a problem hiding this comment.
@tieujason330 I assume this is auto-scaled based on demand?
There was a problem hiding this comment.
no this is a fixed amount of nodes being used, there's no auto-scaling option for aws elasticsearch. or we'd need to handle it ourselves
|
Please monitor the service cost in our AWS bill over the next couple of weeks to get a sense of operation costs. Thank you. |
jason's account accumulated a nice steady cost in the last week: https://docs.google.com/spreadsheets/d/18tj-covBTYCc7R2v6fS0M9JIaWwtnqP622iEL_O64Xw/edit#gid=1188663573 |
|
Okay, not too bad. $36 per tenant for OpenSearch per day. We'll optimize when we have too. |
| @@ -692,8 +815,24 @@ Resources: | |||
| Value: !GetAtt rds.Outputs.RdsEndpoint | |||
| - Name: DB_NAME | |||
| Value: !Ref RDSDatabase | |||
| - Name: ES_CLUSTER_ENDPOINT | |||
There was a problem hiding this comment.
is it also not possible to add these new env vars directly under Environment as opposed to in both the if and else sections of the if statement on the ProvisionRDS condition?
also wouldn't it be more accurate to add these under their own "if ProvisionES" block and then join them with the others? or is that just really clunky to define in CFT
There was a problem hiding this comment.
i wasn't able to add multiple if conditions here, keep getting syntax errors
resources/tenant-onboarding.yaml
Outdated
| @@ -215,6 +331,8 @@ Conditions: | |||
| HasDomainName: !Not [!Equals [!Ref DomainName, '']] | |||
| ProvisionEFS: !Equals [!Ref UseEFS, 'true'] | |||
| ProvisionRDS: !Equals [!Ref UseRDS, 'true'] | |||
| ProvisionES: !Equals [!Ref UseES, 'true'] | |||
| ESEnforceHTTPS: !Equals [ !Ref "EnforceHTTPS", "true" ] | |||
resources/tenant-onboarding.yaml
Outdated
| Condition: ProvisionES | ||
| Properties: | ||
| TemplateURL: !Sub https://${SaaSBoostBucket}.s3.amazonaws.com/tenant-onboarding-es.yaml | ||
| TimeoutInMinutes: 45 |
There was a problem hiding this comment.
45 minutes for timeout?! is this maybe accounting for the fact that we could relaunch a tenant with existing data that would take a long time to load? i noticed in the referenced PR that this was 30, which is still a fairly large value
There was a problem hiding this comment.
elasticsearch takes a while to launch, usually around 15+ minutes, i think mainly because the domain endpoint gets created and it waits til the endpoint is accessible. i can set it back to 30, i think i set to 45 when i initially tested
remove
tieujason330
force-pushed
the
add-elasticsearch
branch
from
09d6ca2 to
71de183
Compare
|
unfortunately based on that PR, it sounds like the PR creator is just going to leave it open for the community to use since there's too much work to be done to make it production-ready/merged into main. i'll happily redo my changes if it ever goes into main though since that'll probably mean it'll include much more customization options in the admin ui. |
* elasticsearch cft * set 30 timeout remove
* elasticsearch cft * set 30 timeout remove
* elasticsearch cft * set 30 timeout remove
* elasticsearch cft * set 30 timeout remove
