Clarify hardware profile and instance configuration related docs for ECH #2039

kunisen · 2025-07-04T13:35:03Z

Background

There are a bunch of hardware profile and instance configuration related ECH docs that are not clear enough and have caused some confusions to users and support so far.
After syncing with @yuvielastic @maggieghamry @stefnestor and @jakommo, we decided to make some doc updates to clarify.

Details in this internal ticket (https://github.com/elastic/support-tech-lead/issues/1586). Below is a quick recap of what we would like to change.

For reviewers

Requested @yuvielastic your kind review from PM's perspective
Requested @claudia-correia your kind review from engineering's perspective
Requested @maggieghamry your kind review from support's perspective
Requested @eedugon or @shainaraskas your kind review from docs team's perspective

Recap of what to change

:: [1]

https://www.elastic.co/docs/deploy-manage/deploy/elastic-cloud/ec-change-hardware-profile

Change title to Manage hardware profiles
Add that version <= 7.10 can't do hardware profile migration

:: [2]

https://www.elastic.co/docs/deploy-manage/deploy/elastic-cloud/change-hardware

Update the doc from mentioning "Change hardware" to "Customize instance configuration" and make this update all through this doc page. <= It's technically more proper to say "instance configuration" but not just "hardware"
Add "Hardware profile is also referenced as deployment templates (DTs) in ECH to make it more logically connectable"
Add an "In addition" paragraph to explain some details, that has confused users and support so far. We have already confirmed with Yuvi from technical perspective.

:: [3]

https://www.elastic.co/docs/deploy-manage/production-guidance/optimize-performance/search-speed

Reference regarding "hardware profile" is not correct. It should be the real hardware profile one as mentioned in [1].

:: [4]

https://www.elastic.co/docs/deploy-manage/production-guidance/optimize-performance/indexing-speed

Reference regarding "hardware profile" is not correct. It should be the real hardware profile one as mentioned in [1].

:: [5]

Move https://www.elastic.co/docs/deploy-manage/deploy/elastic-cloud/change-hardware - "customize IC" doc under https://www.elastic.co/docs/deploy-manage/deploy/elastic-cloud/ec-change-hardware-profile "manage hardware profile" to make it more logically clear.

🔍 Preview links for changed docs

cc @maggieghamry @stefnestor @jakommo

Clarify hardware profile and instance configuration related docs

github-actions · 2025-07-04T13:40:01Z

🔍 Preview links for changed docs

…cs-content into kunisen-docpr-stl1586

yuvielastic · 2025-07-06T21:15:07Z

Thanks Kuni for the updates.

I wanted to suggest few changes within this page under List of Hardware profiles section. Following are the changes, can we please incoporate these changes as well?

Existing under CPU optimized (ARM): This profile is similar to CPU optimized profile but is powered by AWS Graviton2 instances.

Modification: This profile is similar to CPU optimized profile but powered by ARM instances. Currently, we offer ARM instances on AWS.

Existing under General purpose (ARM): This profile is similar to the General purpose profile but is powered by AWS Graviton2 instances.

Modification: This profile is similar to General purpose profile but powered by ARM instances. Currently, we offer ARM instances on AWS.

Existing under Vector search optimized (ARM): This profile is suited for Vector search, Generative AI and Semantic search optimized workloads.

Modification: This profile is suited for Vector search, Generative AI and Semantic search optimized workloads powered by ARM instances. Currently, we offer ARM instances on AWS.

To add: Vector search optimized
This profile is suited for Vector search, Generative AI and Semantic search optimized workloads. You can find the exact storage, memory, and vCPU allotment on the hardware details page for each cloud provider.

Ideal use case

Optimized for applications that leverage Vector Search and/or Generative AI. Also the optimal choice for utilizing ELSER for semantic search applications. Broadly suitable for all semantic search, text embedding, image search, and other Vector Search use cases.

Rest LGTM

per #2039 (comment)

kunisen · 2025-07-07T05:16:09Z

Thank you @yuvielastic I made the change accordingly:
2749011

Could you please kindly confirm if it looks good and make an approval explicitly accordingly? 🙏

FWIW, the diff may be a bit tricky to understand about vector search hardware profile

We originally had Vector search optimized (RAM) which was not dedicated for ARM but for non-ARM / generic description.
We added Vector search optimized and rewrote previous Vector search optimized (ARM) to make it dedicated for ARM

From github diff perspective, it shows as "we newly added ARM section" but it's a 2-step updates as explained above.

yuvielastic

LGTM

yuvielastic · 2025-07-06T21:05:59Z

deploy-manage/deploy/elastic-cloud/ec-change-hardware-profile.md

@@ -24,6 +26,11 @@ The {{ecloud}} console indicates when a new version of a hardware profile is ava

 ## Change the hardware profile using the {{ecloud}} console [ec_change_the_hardware_profile_using_the_elastic_cloud_console]

+::::{note}
+Deployment with Elastic stack version prior to 7.10 does not support hardware profile change {{ecloud}} console and API. If you want to make change on hardware profile, upgrading to version 7.10 and onwards is required.


Just for my understanding: What's causing this behavior? Is this something that's fixable easily?

Just for my understanding: What's causing this behavior? Is this something that's fixable easily?

@yuvielastic it's a product issue that we only support hardware profile migration based on node_roles, where previously in cloud deployment, they use node types and we only introduced node_roles from Elasticsearch 7.10.

You can review this KB (external link) for more details: https://support.elastic.co/knowledge/2040b616 (you can also view internal link and I can share it with you internally if you need more history context)

Also, @gigerdo and I had a sync recently and he suggested that we should use a more self-explanatory message - which is being handled in CP-11182 (internal ticket).

IMHO it's not an easy fix - and we probably want to encourage customer more on upgrading to 7.10+ (preferably the current latest they can upgrade to) instead of fixing this and supporting customers to do HW migration on old versions. (It may cause other issues correspondingly as we didn't handle it very smoothly in earlier versions during node type to node roles migration, and it might be hard to cover all the test cases when we add an additional logic to hardware migration APIs - the most difficult part I can expect is the plan fails in the middle that having a mixed of node roles and types, then it may become technically hard to tackle.)

Thanks Kuni for clarifying and agreed that we should encourage customers to upgrade to 7.10+ especially as that version is super old. Also, good to know that we will be updating the message to be more self-explanatory (separately in CP-11182).

claudia-correia

LGTM — Thanks for making these changes, I believe the docs are more clear and consistent now! 🙇🏻‍♀️
Just left a small comment about how to use the API in order to get deprecated ICs/DTs.

claudia-correia · 2025-07-07T09:56:19Z

deploy-manage/deploy/elastic-cloud/change-hardware.md

+
+In addtion, you can refer to below information about how these terminologies are referenced. 
+* _Deprecated_ is also referenced as _legacy_. 
+* When using APIs, `hide_deprecated=true` is equivalent as `metadata=legacy:true`.


I don't think this is true. Using hide_deprecated=true would be equivalent to using metadata=legacy:false (if this was available) as it should show all of the ICs/DTs that are not deprecated/legacy 🙂

In contrast, using metadata=legacy:true returns the deprecated ICs/DTs.

Not specifying neither the metadata or hide_deprecated query params will return all DTs (and respective ICs). In this case, we can use the approach mentioned on the next statement — checking the presence of the metadata.legacy field.

Suggested change

* When using APIs, `hide_deprecated=true` is equivalent as `metadata=legacy:true`.

* When using APIs, `hide_deprecated=true` is equivalent as `metadata=legacy:false`. Or in contrast, using `hide_deprecated=false` is equivalent as `metadata=legacy:true`.

Oh I am sorry for that, I must have written a typo here. 🤦 Thank you @claudia-correia for the very valuable check! 🙏

(I made a suggestion based on your comment and committed it)

Thanks for addressing my comment, Kuni 🙇🏻‍♀️ However, on a second thought, I think we should be a bit more explicit about this, as it may cause confusion to customers.

My statement above ("Using hide_deprecated=true would be equivalent to using metadata=legacy:false") is not entirely true, actually. Sorry about that. Let me try to make everything clearer:

We consider a DT to be deprecated/legacy whenever it contains a legacy: true metadata entry.

If the DT does not contain a legacy metadata entry OR it contains that entry set to false, then the DT is considered active/valid (i.e., not deprecated/legacy).

When using the metadata=legacy:true query parameter, this will return all legacy/deprecated DTs.

When using the hide_deprecated=true query parameter, this will return all active/valid DTs.

Using metadata=legacy:false won't actually return anything because usually we don't set the legacy field to false, we simply omit that field from the metadata. This is why using hide_deprecated=true is not actually equivalent to using metadata=legacy:false. Sorry for the confusion here.

With this being said, I think we should only mention the three possible scenarios in the docs:

Using metadata=legacy:true to get legacy/deprecated DTs.

Using hide_deprecated=true to get active/valid DTs.

Not using any of the query parameters above to get all DTs.

In this case, we can check the presence of legacy: true in the metadata entries within the API response, to verify if an IC/DT is deprecated or not.

I also wonder if it would be worth it to add to the docs that "The above API requests return a list of DTs and the respective ICs referenced within each DT".

deploy-manage/deploy/elastic-cloud/change-hardware.md

kunisen · 2025-07-07T10:12:05Z

Thank you Claudia for the very quick and helpful review! Much appreciated! 🙇

eedugon · 2025-07-07T14:31:44Z

deploy-manage/deploy/elastic-cloud/change-hardware.md

@@ -8,9 +8,11 @@ products:
  - id: cloud-hosted
 ---

-# Change hardware [ec-change-hardware-for-a-specific-resource]
+# Customize instance configuration [ec-change-instance-configuration]


Changing the anchor name (the part under [ and ]) could provoke existing links to this page (using the old anchor name) to not locate the section within the doc.

Changing the heading name is not a problem, of course.

I would only change the anchor names if I was sure it's an anchor that hasn't been used or referenced in the past.

kunisen added 3 commits July 4, 2025 22:18

Clarify hardware profile and instance configuration related docs

2f9e518

Clarify hardware profile and instance configuration related docs

Update ec-change-hardware-profile.md

9d7cb96

Update ec-change-hardware-profile.md

2c5fafc

kunisen requested review from eedugon, shainaraskas, claudia-correia and yuvielastic July 4, 2025 13:35

kunisen self-assigned this Jul 4, 2025

kunisen requested a review from a team as a code owner July 4, 2025 13:35

kunisen added documentation Improvements or additions to documentation supportability ability enable self-service or support of product labels Jul 4, 2025

github-actions bot deployed to docs-preview July 4, 2025 13:35 View deployment

Merge branch 'main' into kunisen-docpr-stl1586

f5afb2b

github-actions bot deployed to docs-preview July 4, 2025 13:38 View deployment

Merge branch 'kunisen-docpr-stl1586' of https://github.com/elastic/do…

b4ad1c5

…cs-content into kunisen-docpr-stl1586

github-actions bot deployed to docs-preview July 4, 2025 13:43 View deployment

kunisen requested a review from maggieghamry July 4, 2025 13:45

Update ec-change-hardware-profile.md

2749011

per #2039 (comment)

github-actions bot deployed to docs-preview July 7, 2025 05:14 View deployment

yuvielastic approved these changes Jul 7, 2025

View reviewed changes

claudia-correia reviewed Jul 7, 2025

View reviewed changes

Update deploy-manage/deploy/elastic-cloud/change-hardware.md

6f22fe1

github-actions bot had a problem deploying to docs-preview July 7, 2025 10:10 Failure

Update deploy-manage/deploy/elastic-cloud/change-hardware.md

ba8353d

github-actions bot deployed to docs-preview July 7, 2025 10:12 View deployment

Merge branch 'main' into kunisen-docpr-stl1586

17d4bd1

github-actions bot deployed to docs-preview July 7, 2025 12:27 View deployment

eedugon reviewed Jul 7, 2025

View reviewed changes

	* When using APIs, `hide_deprecated=true` is equivalent as `metadata=legacy:true`.
	* When using APIs, `hide_deprecated=true` is equivalent as `metadata=legacy:false`. Or in contrast, using `hide_deprecated=false` is equivalent as `metadata=legacy:true`.

Clarify hardware profile and instance configuration related docs for ECH #2039

Are you sure you want to change the base?

Clarify hardware profile and instance configuration related docs for ECH #2039

Conversation

kunisen commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

For reviewers

Recap of what to change

:: [1]

:: [2]

:: [3]

:: [4]

:: [5]

🔍 Preview links for changed docs

Uh oh!

github-actions bot commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Preview links for changed docs

Uh oh!

yuvielastic commented Jul 6, 2025

Uh oh!

kunisen commented Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yuvielastic left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kunisen Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

claudia-correia left a comment

Choose a reason for hiding this comment

Uh oh!

claudia-correia Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kunisen commented Jul 7, 2025

Uh oh!

eedugon Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kunisen commented Jul 4, 2025 •

edited

Loading

github-actions bot commented Jul 4, 2025 •

edited

Loading

kunisen commented Jul 7, 2025 •

edited

Loading

kunisen Jul 7, 2025 •

edited

Loading

claudia-correia Jul 7, 2025 •

edited

Loading

eedugon Jul 7, 2025 •

edited

Loading