Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: FEATURE: Rework CR CatchUp mechanism #4988

Closed
wants to merge 34 commits into from
Closed
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
3a2fc46
WIP: FEATURE: Rework CR CatchUp mechanism
bwaidelich Apr 11, 2024
99cd1e0
Remove `react/async` dependency
bwaidelich Apr 12, 2024
e8a0bdd
Remove `react/async` dependency 2/2
bwaidelich Apr 12, 2024
d2382ee
Cosmetic fix to satisfy linter
bwaidelich Apr 13, 2024
7dc35fc
Remove unused namespace imports
bwaidelich Apr 13, 2024
13e1238
Replace `CheckpointStorageInterface`
bwaidelich Apr 14, 2024
4c425cd
Replace `CheckpointStorageInterface`
bwaidelich Apr 14, 2024
b1d28f6
Merge branch 'feature/4746-rework-catchup-mechanism' of https://githu…
bwaidelich Apr 14, 2024
ad57a91
Remove `DbalClientInterface` and implementations
bwaidelich Apr 15, 2024
b817b1d
Update composer manifest
bwaidelich Apr 16, 2024
cb653fd
Tweak checkpoint error message
bwaidelich Apr 16, 2024
43da4c2
Re-add `neos/eventstore-doctrineadapter` dependency
bwaidelich Apr 16, 2024
bb1c498
Mostly cosmetic tweaks to satisfy linter
bwaidelich Apr 16, 2024
e62cb87
Make checkpoint table check a platform specific check
bwaidelich Apr 17, 2024
7cc13ed
Remove notion of "sync" from composer scripts and readme
bwaidelich Apr 17, 2024
fc6ec5c
Remove ProjectionCatchUpTriggerInterface and implementations
bwaidelich Apr 17, 2024
89eada5
Merge branch '9.0' into feature/4746-rework-catchup-mechanism
bwaidelich Apr 24, 2024
cc3a191
Merge branch '9.0' into feature/4746-rework-catchup-mechanism
bwaidelich Apr 25, 2024
7c8ae2c
Merge branch '9.0' into feature/4746-rework-catchup-mechanism
bwaidelich Apr 26, 2024
ed47eb6
Fix ContentStreamProjection checkpoint
bwaidelich Apr 26, 2024
592e78d
Merge branch '9.0' into feature/4746-rework-catchup-mechanism
bwaidelich Apr 30, 2024
4af1e3c
Merge branch '9.0' into feature/4746-rework-catchup-mechanism
bwaidelich May 1, 2024
4c68ca7
BUGFIX: Remove doctrine migrate and cr setup commands from behat scripts
bwaidelich May 1, 2024
7bfd941
Use dedicated connection for EventStore
bwaidelich May 1, 2024
e2e8af6
Re-use previously created event store instances
bwaidelich May 1, 2024
fb9b897
Merge branch '9.0' into feature/4746-rework-catchup-mechanism
bwaidelich May 6, 2024
082c5ca
Merge remote-tracking branch 'origin/9.0' into feature/4746-rework-ca…
mhsdesign May 17, 2024
adf6eef
TASK: Adjustments after merging 9.0 (beta10) to satisfy phpstan mostly
mhsdesign May 17, 2024
c6920ec
Merge remote-tracking branch 'origin/9.0' into feature/4746-rework-ca…
mhsdesign May 17, 2024
88abe17
Merge branch '9.0' into feature/4746-rework-catchup-mechanism
bwaidelich May 19, 2024
27f330f
Merge branch 'task/remove-dbal-client' into feature/4746-rework-catch…
bwaidelich May 19, 2024
b4fe6e9
Merge branch '9.0' into feature/4746-rework-catchup-mechanism
bwaidelich May 19, 2024
3abc8f0
Make ContentGraphTableNames::tableNamePrefix private again
bwaidelich May 19, 2024
3d6edc7
Merge branch '9.0' into feature/4746-rework-catchup-mechanism
bwaidelich May 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@ public function __construct()
self::bootstrapFlow();
$this->contentRepositoryRegistry = $this->getObject(ContentRepositoryRegistry::class);

$this->setupCRTestSuiteTrait();
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no longer needed, see

$this->setupDbalGraphAdapterIntegrityViolationTrait();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
namespace Neos\ContentGraph\DoctrineDbalAdapter\Tests\Behavior\Features\Bootstrap;

use Behat\Gherkin\Node\TableNode;
use Doctrine\DBAL\Connection;
use Doctrine\DBAL\DBALException;
use Doctrine\DBAL\Exception\InvalidArgumentException;
use Neos\ContentGraph\DoctrineDbalAdapter\DoctrineDbalContentGraphProjectionFactory;
Expand All @@ -26,7 +27,6 @@
use Neos\ContentRepository\Core\SharedModel\Node\NodeAggregateId;
use Neos\ContentRepository\Core\SharedModel\Workspace\ContentStreamId;
use Neos\ContentRepository\TestSuite\Behavior\Features\Bootstrap\CRTestSuiteRuntimeVariables;
use Neos\ContentRepositoryRegistry\DoctrineDbalClient\DoctrineDbalClient;
use Neos\Error\Messages\Error;
use Neos\Error\Messages\Result;
use PHPUnit\Framework\Assert;
Expand All @@ -40,7 +40,7 @@ trait ProjectionIntegrityViolationDetectionTrait
{
use CRTestSuiteRuntimeVariables;

private DoctrineDbalClient $dbalClient;
private Connection $dbal;

protected Result $lastIntegrityViolationDetectionResult;

Expand All @@ -61,7 +61,7 @@ protected function getTableNamePrefix(): string

public function setupDbalGraphAdapterIntegrityViolationTrait()
{
$this->dbalClient = $this->getObject(DoctrineDbalClient::class);
$this->dbal = $this->getObject(Connection::class);
}

/**
Expand All @@ -79,7 +79,7 @@ public function iRemoveTheFollowingSubtreeTag(TableNode $payloadTable): void
if (!$subtreeTags->contain($subtreeTagToRemove)) {
throw new \RuntimeException(sprintf('Failed to remove subtree tag "%s" because that tag is not set', $subtreeTagToRemove->value), 1708618267);
}
$this->dbalClient->getConnection()->update(
$this->dbal->update(
$this->getTableNamePrefix() . '_hierarchyrelation',
[
'subtreetags' => json_encode($subtreeTags->without($subtreeTagToRemove), JSON_THROW_ON_ERROR | JSON_FORCE_OBJECT),
Expand All @@ -96,7 +96,7 @@ public function iAddTheFollowingHierarchyRelation(TableNode $payloadTable): void
{
$dataset = $this->transformPayloadTableToDataset($payloadTable);
$record = $this->transformDatasetToHierarchyRelationRecord($dataset);
$this->dbalClient->getConnection()->insert(
$this->dbal->insert(
$this->getTableNamePrefix() . '_hierarchyrelation',
$record
);
Expand All @@ -113,7 +113,7 @@ public function iChangeTheFollowingHierarchyRelationsDimensionSpacePointHash(Tab
$record = $this->transformDatasetToHierarchyRelationRecord($dataset);
unset($record['position']);

$this->dbalClient->getConnection()->update(
$this->dbal->update(
$this->getTableNamePrefix() . '_hierarchyrelation',
[
'dimensionspacepointhash' => $dataset['newDimensionSpacePointHash']
Expand All @@ -133,7 +133,7 @@ public function iChangeTheFollowingHierarchyRelationsEdgeName(TableNode $payload
$record = $this->transformDatasetToHierarchyRelationRecord($dataset);
unset($record['position']);

$this->dbalClient->getConnection()->update(
$this->dbal->update(
$this->getTableNamePrefix() . '_hierarchyrelation',
[
'name' => $dataset['newName']
Expand All @@ -157,7 +157,7 @@ public function iSetTheFollowingPosition(TableNode $payloadTable): void
'childnodeanchor' => $this->findRelationAnchorPointByDataset($dataset)
];

$this->dbalClient->getConnection()->update(
$this->dbal->update(
$this->getTableNamePrefix() . '_hierarchyrelation',
[
'position' => $dataset['newPosition']
Expand All @@ -175,7 +175,7 @@ public function iDetachTheFollowingReferenceRelationFromItsSource(TableNode $pay
{
$dataset = $this->transformPayloadTableToDataset($payloadTable);

$this->dbalClient->getConnection()->update(
$this->dbal->update(
$this->getTableNamePrefix() . '_referencerelation',
[
'nodeanchorpoint' => 7777777
Expand All @@ -193,7 +193,7 @@ public function iSetTheFollowingReferencePosition(TableNode $payloadTable): void
{
$dataset = $this->transformPayloadTableToDataset($payloadTable);

$this->dbalClient->getConnection()->update(
$this->dbal->update(
$this->getTableNamePrefix() . '_referencerelation',
[
'position' => $dataset['newPosition']
Expand Down Expand Up @@ -263,7 +263,7 @@ private function findHierarchyRelationByIds(
DimensionSpacePoint $dimensionSpacePoint,
NodeAggregateId $nodeAggregateId
): array {
$nodeRecord = $this->dbalClient->getConnection()->executeQuery(
$nodeRecord = $this->dbal->executeQuery(
'SELECT h.*
FROM ' . $this->getTableNamePrefix() . '_node n
INNER JOIN ' . $this->getTableNamePrefix() . '_hierarchyrelation h
Expand Down Expand Up @@ -299,7 +299,7 @@ private function transformPayloadTableToDataset(TableNode $payloadTable): array
*/
public function iRunIntegrityViolationDetection(): void
{
$projectionIntegrityViolationDetectionRunner = $this->getContentRepositoryService(new DoctrineDbalProjectionIntegrityViolationDetectionRunnerFactory($this->dbalClient));
$projectionIntegrityViolationDetectionRunner = $this->getContentRepositoryService(new DoctrineDbalProjectionIntegrityViolationDetectionRunnerFactory($this->dbal));
$this->lastIntegrityViolationDetectionResult = $projectionIntegrityViolationDetectionRunner->run();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ Feature: Run integrity violation detection regarding hierarchy relations and nod
| workspaceTitle | "Live" |
| workspaceDescription | "The live workspace" |
| newContentStreamId | "cs-identifier" |
And the graph projection is fully up to date
And I am in the active content stream of workspace "live" and dimension space point {}
And the command CreateRootNodeAggregateWithNode is executed with payload:
| Key | Value |
Expand All @@ -35,7 +34,6 @@ Feature: Run integrity violation detection regarding hierarchy relations and nod
| parentNodeAggregateId | "lady-eleonode-rootford" |
| nodeName | "child-document" |
| nodeAggregateClassification | "regular" |
And the graph projection is fully up to date

Scenario: Detach a hierarchy relation from its parent
When I add the following hierarchy relation:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ Feature: Run integrity violation detection regarding parent relations
| workspaceTitle | "Live" |
| workspaceDescription | "The live workspace" |
| newContentStreamId | "cs-identifier" |
And the graph projection is fully up to date
And I am in the active content stream of workspace "live" and dimension space point {"language":"de"}
And the command CreateRootNodeAggregateWithNode is executed with payload:
| Key | Value |
Expand Down Expand Up @@ -55,7 +54,6 @@ Feature: Run integrity violation detection regarding parent relations
| parentNodeAggregateId | "sir-david-nodenborough" |
| nodeName | "child-document" |
| nodeAggregateClassification | "regular" |
And the graph projection is fully up to date

Scenario: Set a second parent for Nody McNodeface
And I add the following hierarchy relation:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ Feature: Run integrity violation detection regarding reference relations
| workspaceTitle | "Live" |
| workspaceDescription | "The live workspace" |
| newContentStreamId | "cs-identifier" |
And the graph projection is fully up to date
And I am in the active content stream of workspace "live" and dimension space point {"language":"de"}
And the command CreateRootNodeAggregateWithNode is executed with payload:
| Key | Value |
Expand All @@ -46,7 +45,6 @@ Feature: Run integrity violation detection regarding reference relations
| coveredDimensionSpacePoints | [{"language":"de"},{"language":"gsw"},{"language":"fr"}] |
| parentNodeAggregateId | "lady-eleonode-rootford" |
| nodeAggregateClassification | "regular" |
And the graph projection is fully up to date

Scenario: Detach a reference relation from its source
When the command SetNodeReferences is executed with payload:
Expand All @@ -55,7 +53,6 @@ Feature: Run integrity violation detection regarding reference relations
| sourceNodeAggregateId | "source-nodandaise" |
| referenceName | "referenceProperty" |
| references | [{"target": "anthony-destinode"}] |
And the graph projection is fully up to date
And I detach the following reference relation from its source:
| Key | Value |
| contentStreamId | "cs-identifier" |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,11 @@ Feature: Run integrity violation detection regarding sibling sorting
| workspaceTitle | "Live" |
| workspaceDescription | "The live workspace" |
| newContentStreamId | "cs-identifier" |
And the graph projection is fully up to date
And I am in the active content stream of workspace "live" and dimension space point {"language":"de"}
And the command CreateRootNodeAggregateWithNode is executed with payload:
| Key | Value |
| nodeAggregateId | "lady-eleonode-rootford" |
| nodeTypeName | "Neos.ContentRepository:Root" |
And the graph projection is fully up to date

Scenario: Create two siblings and set the sorting to the same value
When the event NodeAggregateWithNodeWasCreated was published with payload:
Expand All @@ -46,7 +44,6 @@ Feature: Run integrity violation detection regarding sibling sorting
| coveredDimensionSpacePoints | [{"language":"de"},{"language":"gsw"},{"language":"fr"}] |
| parentNodeAggregateId | "lady-eleonode-rootford" |
| nodeAggregateClassification | "regular" |
And the graph projection is fully up to date
And I set the following position:
| Key | Value |
| contentStreamId | "cs-identifier" |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,11 @@ Feature: Run integrity violation detection regarding subtree tag inheritance
| workspaceTitle | "Live" |
| workspaceDescription | "The live workspace" |
| newContentStreamId | "cs-identifier" |
And the graph projection is fully up to date
And I am in the active content stream of workspace "live" and dimension space point {"language":"de"}
And the command CreateRootNodeAggregateWithNode is executed with payload:
| Key | Value |
| nodeAggregateId | "lady-eleonode-rootford" |
| nodeTypeName | "Neos.ContentRepository:Root" |
And the graph projection is fully up to date

Scenario: Create nodes, disable the topmost and remove some restriction edges manually
When the event NodeAggregateWithNodeWasCreated was published with payload:
Expand Down Expand Up @@ -64,7 +62,6 @@ Feature: Run integrity violation detection regarding subtree tag inheritance
| nodeAggregateId | "sir-david-nodenborough" |
| affectedDimensionSpacePoints | [{"language":"de"},{"language":"gsw"},{"language":"fr"}] |
| tag | "disabled" |
And the graph projection is fully up to date
And I remove the following subtree tag:
| Key | Value |
| contentStreamId | "cs-identifier" |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ Feature: Run projection integrity violation detection regarding naming of tether
| workspaceTitle | "Live" |
| workspaceDescription | "The live workspace" |
| newContentStreamId | "cs-identifier" |
And the graph projection is fully up to date
And I am in the active content stream of workspace "live" and dimension space point {"language":"de"}
And the command CreateRootNodeAggregateWithNode is executed with payload:
| Key | Value |
Expand All @@ -35,7 +34,6 @@ Feature: Run projection integrity violation detection regarding naming of tether
| parentNodeAggregateId | "lady-eleonode-rootford" |
| nodeName | "document" |
| nodeAggregateClassification | "regular" |
And the graph projection is fully up to date

Scenario: Create node variants of different type
When the event NodeAggregateWithNodeWasCreated was published with payload:
Expand All @@ -48,7 +46,6 @@ Feature: Run projection integrity violation detection regarding naming of tether
| parentNodeAggregateId | "sir-david-nodenborough" |
| nodeName | "to-be-hacked-to-null" |
| nodeAggregateClassification | "tethered" |
And the graph projection is fully up to date
And I change the following hierarchy relation's name:
| Key | Value |
| contentStreamId | "cs-identifier" |
Expand Down
3 changes: 1 addition & 2 deletions Neos.ContentGraph.DoctrineDbalAdapter/composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,7 @@
"license": "GPL-3.0+",
"require": {
"neos/contentrepository-core": "self.version",
"doctrine/dbal": "^2.13",
"doctrine/migrations": "*"
"neos/contentrepository-dbaltools": "*"
},
"autoload": {
"psr-4": {
Expand Down
Loading
Loading