Support 'custom' changes being added to exporter.sourceDbChanges

[nick4598]

No description provided.

[JulijaRamoskiene]

_sourceDbChanges? should be required. As API user I have no idea how to handle undefined source db changes. It is impossible to add custom changes and I am too worried to throw error (what if it is expected).
It should be transformers responsibility to handle case when source db change is undefined and act accordingly (throw / not to call addCustomChanges / initalize custom changes / something else).

[nick4598]

made sourceDbChanges required, and only call addCustomChanges if its defined.

[JulijaRamoskiene]

Continuing this discussion: https://github.com/iTwin/imodel-transformer/pull/172/files/e576fa6b45d47d3996366ff9a910a9d5369ba892#r1888402014

I realized where was my logical mistake was in previous discussion, these are fixed requirements of how model can be inserted via custom changes API:

• model id is added to insertedIds
• modeled element id is added to inserted ids (it is same as model id). This is needed as model can not be created without modeled element.
• model of modeled element is added to updated / inserted ids. This is needed so that modeled element is exported to target in exportModeledContents()

Turns out model of modeled element needs to be marked as changed (not parent model of inserted model as I thought previously). I got mixed up as those are the same in my test data (both equal to RepositoryModel).

Suggest to update current implementation to call addCustomElementChange instead of 'handleChange' with modelet element id:

    addCustomModelChange(changeType, ids) {
        for (const id of core_bentley_1.Id64.iterable(ids)) {
            this.handleChange(this.model, changeType, id);
            // Also add the model's modeledElement to the element changes. The modeledElement and model go hand in hand.
            addCustomElementChange(changeType, id);
        }
    }

This is test that I used for my investigation, which is failing with current implementation:

  it.only("should insert Model", async () => {
    const masterIModelName = "Master";
    const masterSeedFileName = path.join(outputDir, `${masterIModelName}.bim`);
    if (IModelJsFs.existsSync(masterSeedFileName))
      IModelJsFs.removeSync(masterSeedFileName);
    const masterSeedState = { 1: 1, 2: 1, 20: 1, 21: 1, 40: 1, 41: 2, 42: 3 };
    const masterSeedDb = SnapshotDb.createEmpty(masterSeedFileName, {
      rootSubject: { name: masterIModelName },
    });
    // eslint-disable-next-line deprecation/deprecation
    masterSeedDb.nativeDb.setITwinId(iTwinId); // workaround for "ContextId was not properly setup in the checkpoint" issue
    const schemaPathForMultiAspect =
      IModelTransformerTestUtils.getPathToSchemaWithMultiAspect();
    await masterSeedDb.importSchemas([schemaPathForMultiAspect]);

    populateTimelineSeed(masterSeedDb, masterSeedState);

    const masterSeed: TimelineIModelState = {
      // HACK: we know this will only be used for seeding via its path and performCheckpoint
      db: masterSeedDb as any as BriefcaseDb,
      id: "master-seed",
      state: masterSeedState,
    };

    let physicalModelIdInSource: Id64String | undefined;
    let modelUnderRepositoryModel: Id64String | undefined;
    const timeline: Timeline = [
      { master: { seed: masterSeed } }, // masterSeedState is above
      { branch1: { branch: "master" } },
      { master: { 100: 100, 101: 101 } },
      {
        master: {
          manualUpdate(db) {
            physicalModelIdInSource = PhysicalModel.insert(
              db,
              IModel.rootSubjectId,
              "MyPhysicalModel"
            );
            modelUnderRepositoryModel = DefinitionModel.insert(
              db,
              IModel.rootSubjectId,
              "MyModelUnderRepositoryModel"
            );
          },
        },
      },
      { branch1: { sync: ["master"] } }, // master->branch1 forward sync to pick up relationship change
      {
        branch1: {
          // delete model and element from branch so that we can attempt to add it back in as a custom 'Inserted' change
          manualUpdate(db) {
            const physicalPartitionIdInTarget =
              IModelTestUtils.queryByCodeValue(db, "MyPhysicalModel");
            expect(physicalPartitionIdInTarget).to.not.equal(Id64.invalid);
            db.models.deleteModel(physicalPartitionIdInTarget);
            db.elements.deleteElement(physicalPartitionIdInTarget);
            const modelUnderRepositoryModelId =
              IModelTestUtils.queryByCodeValue(
                db,
                "MyModelUnderRepositoryModel"
              );
            expect(modelUnderRepositoryModelId).to.not.equal(Id64.invalid);
            db.models.deleteModel(modelUnderRepositoryModelId);
            db.elements.deleteElement(modelUnderRepositoryModelId);
          },
        },
      },
      {
        assert({ branch1 }) {
          // Extra assert to make sure relationship and element are deleted in branch1
         
          // assume if element is gone, aspect is gone
          const physicalPartitionIdInTarget = IModelTestUtils.queryByCodeValue(
            branch1.db,
            "MyPhysicalModel"
          );
          expect(physicalPartitionIdInTarget).to.equal(Id64.invalid);
          const modelUnderRepositoryModelInTarget =
            IModelTestUtils.queryByCodeValue(
              branch1.db,
              "MyModelUnderRepositoryModel"
            );
          expect(modelUnderRepositoryModelInTarget).to.equal(Id64.invalid);
        },
      },
      {
        branch1: {
          sync: [
            "master",
            {
              init: {
                afterInitializeExporter: async (exporter) => {
                  exporter.sourceDbChanges?.addCustomModelChange(
                    "Inserted",
                    physicalModelIdInSource!
                  );
                  exporter.sourceDbChanges?.addCustomModelChange(
                    "Inserted",
                    modelUnderRepositoryModel!
                  );
                },
              },
            },
          ],
        },
      },
      {
        assert({ branch1 }) {
          const physicalPartitionIdInTarget = IModelTestUtils.queryByCodeValue(
            branch1.db,
            "MyPhysicalModel"
          );
          expect(physicalPartitionIdInTarget).to.not.equal(Id64.invalid);
          expect(branch1.db.elements.getElement(physicalPartitionIdInTarget)).to
            .not.be.undefined;
          expect(branch1.db.models.getModel(physicalPartitionIdInTarget)).to.not
            .be.undefined;
          const modelUnderRepositoryModelInTarget =
            IModelTestUtils.queryByCodeValue(
              branch1.db,
              "MyModelUnderRepositoryModel"
            );
          expect(modelUnderRepositoryModelInTarget).to.not.equal(Id64.invalid);
          expect(
            branch1.db.elements.getElement(modelUnderRepositoryModelInTarget)
          ).to.not.be.undefined;
          expect(branch1.db.models.getModel(modelUnderRepositoryModelInTarget))
            .to.not.be.undefined;
        },
      }
    ];
    const { tearDown } = await runTimeline(timeline, {
      iTwinId,
      accessToken,
    });
    await tearDown();
  });

[nick4598]

So does this mean the 'modelUnderRepositoryModel' is where we fail? expect(modelUnderRepositoryModelInTarget).to.not.equal(Id64.invalid);? because its modeled element has a model which is the repositoryModel, and we didn't add it to the updated? (Note: after rereading im usnure about tihs, but I hope my two assertions below are correct)

I didn't realize the modeled element could have a model that wasn't the model it was modelling... I started looking at the documentation for model:

Im guessing 'modeled element's model' is being refereenced by this line here? "The Element modeling the car-as-a-whole is also in a Model. What Element is that Model sub-modeling? BIS escapes from infinite regression by defining a special RepositoryModel that is not required to sub-model some other Element." docs

Is that right?

And to take it further the modeled element's model would be found right here on an element:

image taken from here

Is that also right?

[JulijaRamoskiene]

To improve performance, consider querying models of all elements at once in addModelToUpdated()

[JulijaRamoskiene]

This will not be needed if this is fixed: #229