Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XWIKI-22613: Updating the history can take a very long time when there are a lot of objects #3788

Merged
merged 9 commits into from
Jan 31, 2025
Merged

XWIKI-22613: Updating the history can take a very long time when there are a lot of objects #3788

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
07ce23a
XWIKI-22613: Updating the history can take a very long time when ther…
tmortagne Jan 3, 2025
ed17136
XWIKI-22747: Introduce the concept of XWiki Hibernate adapter
tmortagne Jan 29, 2025
7f02893
XWIKI-22747: Introduce the concept of XWiki Hibernate adapter
tmortagne Jan 29, 2025
35bd3db
XWIKI-22747: Introduce the concept of XWiki Hibernate adapter
tmortagne Jan 30, 2025
d887653
XWIKI-22747: Introduce the concept of XWiki Hibernate adapter
tmortagne Jan 30, 2025
5892a5b
XWIKI-22747: Introduce the concept of XWiki Hibernate adapter
tmortagne Jan 31, 2025
c8e2506
XWIKI-22747: Introduce the concept of XWiki Hibernate adapter
tmortagne Jan 31, 2025
28a573e
XWIKI-22747: Introduce the concept of XWiki Hibernate adapter
tmortagne Jan 31, 2025
a07455a
XWIKI-22747: Introduce the concept of XWiki Hibernate adapter
tmortagne Jan 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions ...ex-default/src/main/java/org/xwiki/index/migration/R140300001XWIKI19571DataMigration.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ public String getPreHibernateLiquibaseChangeLog() throws DataMigrationException
PersistentClass persistentClass =
this.hibernateStore.getConfigurationMetadata()
.getEntityBinding(XWikiDocumentIndexingTask.class.getName());
String tableName = this.hibernateStore.getConfiguredTableName(persistentClass);
String tableName = this.hibernateStore.getAdapter().getTableName(persistentClass);
String changeSet = "";
if (exists(databaseMetaData, tableName)) {
saveTasks(session, persistentClass, tableName);
Expand Down Expand Up @@ -138,10 +138,10 @@ protected void hibernateMigrate() throws XWikiException
*/
private boolean exists(DatabaseMetaData databaseMetaData, String tableName) throws SQLException
{
String databaseName = this.hibernateStore.getDatabaseFromWikiName();
String databaseName = this.hibernateStore.getAdapter().getDatabaseFromWikiName();

ResultSet resultSet;
if (this.hibernateStore.isCatalog()) {
if (this.hibernateStore.getAdapter().isCatalog()) {
resultSet = databaseMetaData.getTables(databaseName, null, tableName, null);
} else {
resultSet = databaseMetaData.getTables(null, databaseName, tableName, null);
Expand Down
14 changes: 10 additions & 4 deletions ...efault/src/test/java/org/xwiki/index/migration/R140300001XWIKI19571DataMigrationTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
import org.xwiki.context.ExecutionContext;
import org.xwiki.doc.tasks.XWikiDocumentIndexingTask;
import org.xwiki.index.internal.TasksStore;
import org.xwiki.store.hibernate.HibernateAdapter;
import org.xwiki.test.junit5.mockito.ComponentTest;
import org.xwiki.test.junit5.mockito.InjectMockComponents;
import org.xwiki.test.junit5.mockito.MockComponent;
Expand Down Expand Up @@ -102,6 +103,9 @@ class R140300001XWIKI19571DataMigrationTest
@Mock
private Metadata metadata;

@Mock
private HibernateAdapter adapter;

@Mock
private PersistentClass persistentClass;

Expand All @@ -128,8 +132,10 @@ void setUp() throws Exception
when(this.hibernateStore.getConfigurationMetadata()).thenReturn(this.metadata);
when(this.metadata.getEntityBinding(XWikiDocumentIndexingTask.class.getName()))
.thenReturn(this.persistentClass);
when(this.hibernateStore.getConfiguredTableName(this.persistentClass)).thenReturn("XWIKIDOCUMENTINDEXINGQUEUE");
when(this.hibernateStore.getDatabaseFromWikiName()).thenReturn("dbname");
when(this.hibernateStore.getAdapter()).thenReturn(this.adapter);
when(this.adapter.getTableName(this.persistentClass))
.thenReturn("XWIKIDOCUMENTINDEXINGQUEUE");
when(this.adapter.getDatabaseFromWikiName()).thenReturn("dbname");
when(this.connection.getMetaData()).thenReturn(this.databaseMetaData);
when(this.hibernateStore.getConfiguredColumnName(this.persistentClass, "docId")).thenReturn("DOC_ID");
when(this.hibernateStore.getConfiguredColumnName(this.persistentClass, "version")).thenReturn("VERSION");
Expand All @@ -144,7 +150,7 @@ void setUp() throws Exception
@Test
void hibernateMigrateExistsIsCatalog() throws Exception
{
when(this.hibernateStore.isCatalog()).thenReturn(true);
when(this.adapter.isCatalog()).thenReturn(true);
ResultSet resultSet = mock(ResultSet.class);
when(resultSet.next()).thenReturn(true);
when(this.databaseMetaData.getTables("dbname", null, "XWIKIDOCUMENTINDEXINGQUEUE", null)).thenReturn(resultSet);
Expand Down Expand Up @@ -182,7 +188,7 @@ void hibernateMigrateExistsIsCatalog() throws Exception
@Test
void hibernateMigrateExistsIsNotCatalog() throws Exception
{
when(this.hibernateStore.isCatalog()).thenReturn(false);
when(this.adapter.isCatalog()).thenReturn(false);
ResultSet resultSet = mock(ResultSet.class);
when(resultSet.next()).thenReturn(true);
when(this.databaseMetaData.getTables(null, "dbname", "XWIKIDOCUMENTINDEXINGQUEUE", null)).thenReturn(resultSet);
Expand Down
4 changes: 2 additions & 2 deletions ...orm-core/xwiki-platform-oldcore/src/main/java/com/xpn/xwiki/doc/XWikiDocumentArchive.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,8 +154,8 @@ protected XWikiRCSNodeContent makePatch(XWikiRCSNodeInfo newnode, XWikiDocument
XWikiRCSNodeInfo latestNode = getLatestNode();
if (latestNode != null) {
int nodesCount = getNodes().size();
int nodesPerFull = context.getWiki() == null ? 5
: Integer.parseInt(context.getWiki().getConfig().getProperty("xwiki.store.rcs.nodesPerFull", "5"));
int nodesPerFull = context.getWiki() == null ? 1
: Integer.parseInt(context.getWiki().getConfig().getProperty("xwiki.store.rcs.nodesPerFull", "1"));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if it's not the right moment to also put and document that config option in xwiki.cfg? AFAIR it's not at all documented?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about it, but my feeling is more that we want to bury that option as deep as possible and eventually completely remove that concept.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, I understand but on the other hand you'll indicate in the RN that this breaking change might be reverted with the same property... maybe it's cleaner if we document all of it in xwiki.cfg indicating values, that it's possible to change it but it's not advised to do it and it's even considered as a deprecated property?

eventually completely remove that concept.

I guess if we start documenting it as a way to revert the old behaviour it makes complicated to also open a proposal for removing it, at least not before 19.x to have a full LTS cycle

Copy link
Member Author

@tmortagne tmortagne Jan 29, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that this breaking change

I'm not sure how this is breaking, it just stores a full version every 1 version instead of every 5 versions by default. This does not change the format in any way (i.e. you could still read this database with a previous version of XWiki just fine).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indeed not a breaking change but I assumed when reading your forum proposal that you were going to document the possibility to revert this behaviour, and so this property.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to see it documented

Copy link
Member Author

@tmortagne tmortagne Jan 30, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do plan to indicate a way to go back to the previous behavior in the release note retro compatibility section, but I would really prefer to keep this option hidden in general, since the goal is really not for users to start using something we would like to get rid of one day.

if (nodesPerFull <= 0 || (nodesCount % nodesPerFull) != 0) {
XWikiRCSNodeContent latestContent = latestNode.getContent(context);
latestContent.getPatch().setDiffVersion(latestContent.getPatch().getContent(), doc, context);
Expand Down
Loading
Loading