Skip to content

Latest commit

 

History

History
700 lines (570 loc) · 19.6 KB

README.md

File metadata and controls

700 lines (570 loc) · 19.6 KB

ChromeOS HWID Database User Guide

[TOC]

This document is aimed at defining the standard way to update the HWID database.

Introduction to HWID Database

The role of a hardware ID (HWID) configuration file is to verify that every hardware component used in a Chrome device is qualified. The config file mainly lists the phase of the device image_id, the pattern of the encoded string pattern, the mapping of bit value to the component encoded_field, and the probe result of each component component. For convenience, we call the HWID config file HWID database.

Update Rule

The HWID database has two tasks: (1) verifies the probe result and generates the HWID string when building new devices at the factory, and (2) decodes the HWID string generated by every previous version of the database. To ensure the new database can successfully decode the old HWID string, there are some rules while updating the database.

1. Common Rule: Only Append, Do Not Modify or Remove Items

The first common rule is: do not remove anything. Because we not only need to decode current HWID, but also need to decode EVERY previous valid HWID. Only these items can be modified:

  • The probeable attribute of the component
  • The status of the component item
  • The rules items

2. Pattern: Do Not Modify the Set of encoded_field {#rule_encoded_field}

It's obvious that we are not allowed to remove the field in the pattern. Adding a field whose component class is new seems not break the previous HWID, but it actually causes ambiguity. After adding the component in the pattern, we would decode old HWID as index = 0. But actually we have no information about this component in old HWID. There are 3 possible situations:

  1. The old device doesn't have the component, so the component is not in the HWID database.
  2. The old device has the component, but it cannot be probed previously.
  3. The old device has the component and it can be probed, but user forgot to add into HWID database.

Most of the case is 1, the old device doesn't have the component. But we don't enforce index 0, which means there is no probed result (or no component) currently. So it is weird to allow this happen. The reasonable way is: the component set in each pattern cannot be modified. It can prevent this ambiguity from happening.

However, we might lean to relax this rule in the early build. The case (2) has a high chance of happening: every old device has the component, but it cannot be probed. Creating a new pattern for adding an extra component seems overkill. In this case we allow appending the pattern directly.

In summary, if we confirm that all the old device has the same kind of the component, then it is allowed to append to the current pattern. Otherwise, please create a new pattern when we need to modify the set of component class.

3. Encoded_fields: Do Not Modify Encoded Mapping

Because encoded_field is shared with all patterns, the existed items in the encoded_field cannot be modified. Otherwise the old HWID cannot lookup the correct mapping. Therefore, if we need to change the mapping of a component, we can only add new field into encoded_fields.

4. Components: Use deprecated/unsupported to Remove Component

Because the components items should be encoded in encoded_field in normal case, we cannot remove them. When we don't allow a component in the new unit, use deprecated status. If you don't even allow to match it, use unsupported status. Please refer Status of a Component Item for more details.

Database Builder

Originally it was tedious to convert the probe result to HWID database. Most of the time we copy and paste it from an existing project and modify the probe value. In addition, there are many implict rules and relations between each section. For example, the number of bits in the field of the pattern should be large enough to contain all items in the encoded field.

Now we have a tool called database builder to generate or update the HWID database. This tool is aimed at covering most of the common and tedious HWID database change, like adding or removing components. But the HWID database still needs to be manually updated in the following scenarios:

Scenarios that Cannot Use Database Builder

  • Multiple image_ids are active

    In the database builder, we assume that only the latest (maximum) image_id is active. So we recommend not to use the tool if the assumption is not true.

  • Modify the rules

    The rule section gives us much flexibility in assigning and verifying the value of the component. On the other hand, there is no universal diagram for the rule section. We still leave it to user update the rules manually.

  • Change the status of component items

    Because changing the status is quite easy. Just change it :)

Command of Database Builder

# Build the HWID database
$ hwid build-database \
    --project <project name> \
    [--material-file <HWID material file> ] \
    [--hwid-db-path <hwid db folder>] \
    [--image-id <IMAGE_ID>] \  # Name of the image_id, default is 'EVT'
    [--add-default-component COMP [COMP ...]] \  # Add the default item
    [--add-null-component COMP [COMP ...]] \  # Add the null item
    [--region REGION [REGION ...]] \  # Add supported regions
    [--chassis ID1 [ID2 ...]] \  # Add supported chassis

# Update the HWID database
$ hwid update-database \
    --project <project name> \
    [--hwid-db-path <hwid db folder>] \
    [--material-file <HWID material file> ] \
    [--output-database <output file>] \  # Write into different file
    [--image-id <IMAGE_ID>] \  # Name of the image_id
    [--add-default-component COMP [COMP ...]] \  # Add the default item
    [--add-null-component COMP [COMP ...]] \  # Add the null item
    [--region REGION [REGION ...]] \  # Add supported regions
    [--chassis ID1 [ID2 ...]] \  # Add supported chassis

# the --hwid-db-path argument is only required if the HWID database is not
# placed in /usr/local/factory/hwid/

Post-Processing

Though the database builder handle most of the work, there is still some manual post-processing.

  • Check the name of component items

    The builder will decide a name by the probe result, but it might be meaningless. Please change the name in encoded_fields and components if you don't like it.

  • Confirm the status of the component items

    All component items generated by the database builder are unqualified. After the SIE or TAM confirms the component is qualified at AVL, please change the status to supported.

Use Cases

Below, we will list some common use cases of HWID database change, and provide the recommendation way to do it.

1. Create A New HWID Database

At the beginning of the project, we need to create a new HWID database. The easiest way is using the database builder with a valid probe result. The steps of generating HWID database are:

  1. Confirm the categories of the components that should be encoded into HWID.
  2. Collect materials (probed results, for example) to build the HWID DB.
  3. Confirm the project name and the phase.
  4. (Optional) Check if there are missing components in the probe result. If so, add a default item with the argument "--add-default-component". Please refer Add a Default Item for more details.
  5. (Optional) Check if there are components that do not exist in some SKU. For example, one SKU has cellular and another SKU does not. If so, add a null item with the argument "--add-null-component". Please refer Add a Null Item for more details.

Here is an example to create a product named GOOGLE with the phase EVT, add a default item for battery component, and add a null item for cellular component.

# Get the probe result.
$ hwid collect-material --output-file /tmp/material.yaml

# Create the database at /usr/local/factory/hwid/GOOGLE
$ hwid build-database \
    --material-file /tmp/material.yaml \
    --project GOOGLE \
    --image-id EVT \
    --add-default-component battery \
    --add-null-component cellular

2. Add Second Source and Update Firmware

This is the most common update in HWID database. When we introduce second source or update firmware version, we need to add new supported component item into HWID database. Just append the new item in the component field and check if the encoding bit is enough or not. If we update the firmware version, we should mark the old one as "deprecated" to ensure all DUTs in the factory use the latest firmware.

The command for this scenario is:

# Get the probe result.
$ hwid collect-material --output-file /tmp/material.yaml

# Update the database at /usr/local/factory/hwid/GOOGLE
$ hwid update-database \
    --material-file /tmp/material.yaml \
    --project GOOGLE

3. Add a Default Item {#add_default_item}

When we bring up a device, the probing code for some hardware components might not be ready. For example, a device cannot probe cellular in EVT build, and can probe it later. Before the probing code is ready, please add a default item. After the probing code is ready, add the component item and deprecate the default item.

|||---|||

Before the probing code is ready

image_id:
  0: EVT

pattern:
- image_ids: [0]
  encoding_scheme: base8192
  fields:
  - cellular_field: 3

encoded_fields:
  cellular_field:
    0: {cellular: cellular_default}

components:
  cellular:
    items:
      cellular_default:
        status: unqualified
        value: NULL

After the probing code is ready

image_id:
  0: EVT

pattern:
- image_ids: [0]
  encoding_scheme: base8192
  fields:
  - cellular_field: 3

encoded_fields:
  cellular_field:
    0: {cellular: cellular_default}
    1: {cellular: aa}

components:
  cellular:
    items:
      cellular_default:
        status: unsupported
        value: NULL
      aa:
        value: {compact_str: aa}

|||---|||

The command for this scenario is:

# Add a default item with "--add-default-component" argument
$ hwid build-database \
    --project GOOGLE \
    --material-file /tmp/material.yaml \  # the probe result doesn't contain
                                          # cellulars.
    --add-default-component cellular

# Update the database by the probed result
$ hwid update-database \
    --project GOOGLE \
    --material-file /tmp/material.yaml  # the probe result contains the
                                        # cellulars "aa".

4. Add a Null Item {#add_null_item}

When we have a special SKU that does not have a component, please add a null item. For example, we have a SKU with cellular component and another SKU without the cellular component. Then the HWID database should be like:

|||---|||

Before adding the null item

image_id:
  0: EVT

pattern:
- image_ids: [0]
  encoding_scheme: base8192
  fields:
  - cellular_field: 3

encoded_fields:
  cellular_field:
    0: {cellular: aa}

components:
  cellular:
    items:
      aa:
        value: {compact_str: aa}

After adding the null item

image_id:
  0: EVT

pattern:
- image_ids: [0]
  encoding_scheme: base8192
  fields:
  - cellular_field: 3

encoded_fields:
  cellular_field:
    0: {cellular: aa}
    1: {cellular: NULL}

components:
  cellular:
    items:
      aa:
        value: {compact_str: aa}

|||---|||

The command for this scenario is:

# Add a default item with "--add-null-component" argument.
$ hwid update-database \
    --project GOOGLE \
    --add-null-component cellular

5. Deprecate a Component Item

Deprecating the component item is quite easy. Please manually add "status: deprecated" in the target component item. Then the new unit may not use this component item.

The example below is to deprecate the cellular component item aa.

|||---|||

Before deprecating the item

image_id:
  0: EVT

pattern:
- image_ids: [0]
  encoding_scheme: base8192
  fields:
  - cellular_field: 3

encoded_fields:
  cellular_field:
    0: {cellular: aa}
    1: {cellular: bb}

components:
  cellular:
    items:
      aa:
        value: {compact_str: aa}
      bb:
        value: {compact_str: bb}

After deprecating the item

image_id:
  0: EVT

pattern:
- image_ids: [0]
  encoding_scheme: base8192
  fields:
  - cellular_field: 3

encoded_fields:
  cellular_field:
    0: {cellular: aa}
    1: {cellular: bb}

components:
  cellular:
    items:
      aa:
        status: deprecated
        value: {compact_str: aa}
      bb:
        value: {compact_str: bb}

|||---|||

6. Add New Component in Device

When we add new components during the build, we need to create a new pattern and new image_id. Please refer Update Rule 2 for more details.

For example, we don't have cellular in EVT build, but add cellular in DVT build. To complicate the situation even more, in DVT build we might have one SKU that has cellular and another SKU that doesn't. In this case, we need to distinguish:

  1. No cellular in EVT build
  2. No cellular in DVT build
  3. Has cellular in DVT build

The difference between 1 and 2 is: in EVT build we don't check the probed result of the cellular component. The actual meaning should be "we don't have information about cellular" instead of "this device doesn't have cellular". To distinguish the case 1 and 2, we need to create a new pattern that adds the new component, while retaining the old pattern which lacks the component.

The HWID database for this example: |||---|||

Before adding the cellular component

image_id:
  0: EVT

pattern:
- image_ids: [0]
  encoding_scheme: base8192
  fields:
  - cpu_field: 3

encoded_fields:
  cpu_field:
    0: {cpu: cpu_aa}

components:
  cpu:
    items:
      cpu_aa:
        value: {compact_str: cpu_aa}

After adding the cellular component

image_id:
  0: EVT
  1: DVT

pattern:
- image_ids: [0]
  encoding_scheme: base8192
  fields:
  - cpu_field: 3

- image_ids: [1]
  encoding_scheme: base8192
  fields:
  - cpu_field: 3
  - cellular_field: 3

encoded_fields:
  cpu_field:
    0: {cpu: cpu_aa}
  cellular_field:
    0: {cellular: NULL}
    1: {cellular: aa}

components:
  cpu:
    items:
      cpu_aa:
        value: {compact_str: cpu_aa}
  cellular:
    items:
      aa:
        value: {compact_str: aa}

|||---|||

The command for this scenario is:

# Create the database at EVT build
$ hwid build-database \
    --project GOOGLE \
    --image-id EVT \
    --material-file /tmp/material.yaml  # the probe result contains doesn't
                                        # contain cellulars.


# Update the database at DVT build.
$ hwid update-database \
    --project GOOGLE \
    --image-id DVT \
    --add-null-component cellular \
    --material-file /tmp/material.yaml  # the probe result contains the
                                        # cellular "aa".

*** note Note: If all the old devices have the same kind of the component, then it is allowed to append the component in the current pattern.

|||---|||

Before adding the cellular component

image_id:
  0: EVT

pattern:
- image_ids: [0]
  encoding_scheme: base8192
  fields:
  - cpu_field: 3

encoded_fields:
  cpu_field:
    0: {cpu: cpu_aa}

components:
  cpu:
    items:
      cpu_aa:
        value: {compact_str: cpu_aa}

After adding the cellular component

image_id:
  0: EVT

pattern:
- image_ids: [0]
  encoding_scheme: base8192
  fields:
  - cpu_field: 3
  - cellular_field: 3

encoded_fields:
  cpu_field:
    0: {cpu: cpu_aa}
  cellular_field:
    0: {cellular: aa}

components:
  cpu:
    items:
      cpu_aa:
        value: {compact_str: cpu_aa}
  cellular:
    items:
      aa:
        value: {compact_str: aa}

|||---|||

The command for this scenario is:

# Update the database without assigning the image_id.
$ hwid update-database \
    --project GOOGLE \
    --material-file /tmp/material.yaml  # the probe result contains the
                                        # cellular "aa".

WARNING: Extra fields [cellular_field] without assigning a new image_id.
If the fields are added into the current pattern, the index of these fields will
be encoded to index 0 for all old HWID string. Enter "y" if you are sure all old
devices with old HWID string have the component with index 0. [y/N] y

Please choose "Y" for the question, then the database builder will append the component into the current pattern.

7. Convert Legacy Region Style to List Style

We use magic tag "!region_field" to generate the region encoded field. There are two styles: legacy and list style. In legacy style, each region has its own index mapping. For example, "us" maps to 29 and "gb" maps to 16. However, in list style, only the listed regions are encoded. In the example below, "us" maps to 1 and "gb" maps to 2.

encoded_fields:
  # Legacy style
  old_region_field: !region_field
  # List style
  new_region_field: !region_field ['us', 'gb']

Because the update rule, we cannot just replace the legacy style because the encoding mapping is different. If we want to convert legacy style to list style, we should create another encoded_field and remove the old one from the pattern. For example:

|||---|||

Before converting the style

image_id:
  0: EVT

pattern:
- image_ids: [0]
  encoding_scheme: base8192
  fields:
  - region_field: 8

encoded_fields:
  region_field: !region_field

components:
  region: !region_component

After converting the style

image_id:
  0: EVT
  1: DVT

pattern:
- image_ids: [0]
  encoding_scheme: base8192
  fields:
  - region_field: 8
- image_ids: [1]
  encoding_scheme: base8192
  fields:
  - new_region_field: 5

encoded_fields:
  region_field: !region_field
  new_region_field: !region_field ['us']

components:
  region: !region_component

|||---|||

The database builder will automatically convert the style if the old database uses the legacy style. *** note Note: a new image_id is required when converting.

Appendix

Unprobeable Component

When a component with the attribute probeable: False, it means one of these:

  • The component class is removed at the latest pattern.
  • The value is determined at the evaluation time. Set at rule.
  • Exception: If there is only one item in this component class, set the index to 0. It is only for backward compatibility. Please use "default" argument instead of using this.

Default Component Item

If a component item has the attribute default: True, it is a default component item. The feature of the default component item is:

  • It is always matched when generating HWID string, no matter the probe result exists or not.
  • If another component is also matched, then the default component item will be ignored.

Status of a Component Item {#component_status}

  • supported

    Default status. The component is currently used to build new units.

  • unqualified

    The component is now allowed to be used after PVT phase. The components added by the auto-generator should be unqualified. Then SIE or TAM should remove it after the component is confirmed.

  • deprecated

    The component is no longer being used to build new units, but is supported in RMA process.

  • unsupported

    The component is not allowed to be used to build new units, and is not supported in RMA process. The component cannot be matched. It is used for the wrong component or the equivalent component but with different probed result.

  • duplicate

    The component is a subset of another comopnent, so it should not be matched while encoding. For example:

  comp_1:
    values:
      key: !re 'PART_NUMBER_1-.*'
  comp_2:
    status: duplicate  # this is necessary, otherwise 'PART_NUMBER_1-2' will
                       # match comp_1 and comp_2, and cause an error.
    values:
      key: PART_NUMBER_1-2