IGNITE-22557 Create system view for SQL query plans history #11425

oleg-vlsk · 2024-07-10T00:32:10Z

Thank you for submitting the pull request to the Apache Ignite.

In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:

The Contribution Checklist

There is a single JIRA ticket related to the pull request.
The web-link to the pull request is attached to the JIRA ticket.
The JIRA ticket has the Patch Available state.
The pull request body describes changes that have been made.
The description explains WHAT and WHY was made instead of HOW.
The pull request title is treated as the final commit message.
The following pattern must be used: IGNITE-XXXX Change summary where XXXX - number of JIRA issue.
A reviewer has been mentioned through the JIRA comments
(see the Maintainers list)
The pull request has been checked by the Teamcity Bot and
the green visa attached to the JIRA ticket (see TC.Bot: Check PR)

Notes

If you need any help, please email [email protected] or ask anу advice on http://asf.slack.com #ignite channel.

alex-plekhanov · 2024-08-16T12:43:50Z

...es/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/dml/UpdatePlan.java

+            "; key column index=" + (keyColIdx < 0 ? "N/A" : keyColIdx) +
+            "; value column index=" + (valColIdx < 0 ? "N/A" : valColIdx) +
+            "; row count=" + rowsNum +
+            "; values=" + (rows == null ? "N/A" : "[" + rowsToString(rows) + "]") +


Not sure we should add rows to plan. Plan will be different each time and it will wash out all other plan. For me value of DML plans are not clear. Perhaps it will be useful to have only plan for selectQry, but looks like we don't have this plan during update. Perhaps these queries are tracked on map phase.

The concept of plans for DML commands has been completely changed.

alex-plekhanov · 2024-08-16T12:44:47Z

...c/main/java/org/apache/ignite/internal/processors/query/h2/twostep/GridMapQueryExecutor.java

@@ -472,6 +473,17 @@ private void onQueryRequest0(
                            );
                        }

+                        SqlPlanHistoryTracker planHistTracker = ctx.query().runningQueryManager().planHistoryTracker();


h2.runningQueryManager().planHistoryTracker().addPlan(...)?

alex-plekhanov · 2024-08-16T12:45:02Z

...ain/java/org/apache/ignite/internal/processors/query/h2/twostep/GridReduceQueryExecutor.java

@@ -522,6 +523,17 @@ else if (QueryUtils.wasCancelled(err))
                            );
                        }

+                        SqlPlanHistoryTracker planHistTracker = ctx.query().runningQueryManager().planHistoryTracker();


h2.runningQueryManager().planHistoryTracker().addPlan(...)?

alex-plekhanov · 2024-08-16T12:45:33Z

modules/core/src/main/java/org/apache/ignite/internal/processors/query/running/SqlPlanKey.java

+        this.schema = schema;
+        this.loc = loc;
+
+        hash = 31 * (31 * plan.hashCode() + qry.hashCode() + schema.hashCode()) + (loc ? 1 : 0);


Objects.hash(...)?

The Objects.hash() method has been implemented.

alex-plekhanov · 2024-08-16T12:46:36Z

modules/core/src/main/java/org/apache/ignite/internal/processors/query/running/SqlPlan.java

+    private final SqlPlanKey key;
+
+    /** */
+    private final SqlPlanValue val;


Why do we need extra SqlPlanValue class? Looks like everithing can be stored in SqlPlan

All SQL plan history data is now stored in the SqlPlan class; the SqlPlanKey and SqlPlanValue classes have been removed.

alex-plekhanov · 2024-08-16T12:51:14Z

...src/main/java/org/apache/ignite/internal/processors/query/running/SqlPlanHistoryTracker.java

+        if (historySize <= 0)
+            return;
+
+        SqlPlan sqlPlan = new SqlPlan(plan, qry, schema, loc, startTs, engine);


I think instead of three entities SqlPlan, SqlPlanKey, SqlPlanValue only one can be used (SqlPlan)
sqlPlanHistory can be just a map from SqlPlan to timestamp and we can add entries like:
sqlPlanHistory.put(sqlPlan.key(), U.currentTimeMillis());
We don't need extra parameter startTs since almost always it will be equals to U.currentTimeMillis().

Done (see previous comment).

alex-plekhanov · 2024-08-16T12:54:08Z

...src/main/java/org/apache/ignite/internal/processors/query/running/SqlPlanHistoryTracker.java

+    }
+
+    /** */
+    public enum SqlEngine {


The core module shouldn't know about list of query engines. Let's use String instead of this enum. Use IndexingQueryEngineConfiguration.ENGINE_NAME and CalciteQueryEngineConfiguration.ENGINE_NAME from indexing and calcite modules.

alex-plekhanov · 2024-09-24T15:23:14Z

...es/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/dml/UpdatePlan.java

@@ -80,10 +80,10 @@ public final class UpdatePlan {

    /** Number of rows in rows based MERGE or INSERT. */
    private final int rowsNum;
-


Redundant changes (all in this file)

alex-plekhanov · 2024-09-24T15:25:02Z

...src/main/java/org/apache/ignite/internal/processors/query/running/SqlPlanHistoryTracker.java

+
+        SqlPlan sqlPlan = new SqlPlan(plan, qry, schema, loc, engine);
+
+        sqlPlanHistory.put(sqlPlan, sqlPlan.startTime());


Are you sure key will be updated after this call.
You use information only from key (including startTime) in the view, if key is not updated, than view will contain a wrong startTime field value.

Yes, the key wasn't updated during the put() operation. To fix it, I changed the SQL plan history data structure from GridBoundedConcurrentLinkedHashMap to GridBoundedConcurrentLinkedHashSet and edited the addPlan() method, so that newer entries would replace older ones with the same SQL plan.

alex-plekhanov · 2024-09-25T14:46:31Z

...src/main/java/org/apache/ignite/internal/processors/query/running/SqlPlanHistoryTracker.java

+     * @param historySize History size.
+     */
+    public void setHistorySize(int historySize) {
+        this.historySize = historySize;


Let's recreate sqlPlanHistory here. Now this method used only once in test with value 0, but if someone will use this method (perhaps in test too) with another value, result can be unpredicted.

alex-plekhanov · 2024-09-25T14:49:26Z

...gnite/internal/processors/query/calcite/integration/SqlPlanHistoryConfigIntegrationTest.java

+import org.junit.Test;
+
+/** Test for Sql plan history configuration. */
+public class SqlPlanHistoryConfigIntegrationTest extends GridCommonAbstractTest {


Let's move this test to ignite-spring module, instead of adding new dependency to ignite-calcite.

SqlPlanHistoryConfigIntegrationTest has been moved to the ignite-spring module.

alex-plekhanov · 2024-09-25T14:58:23Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+     * @return Cache configuration.
+     */
+    @SuppressWarnings("unchecked")
+    private CacheConfiguration configureCahce(String name, Class<?>... idxTypes) {


Typo: Cahce

alex-plekhanov · 2024-09-25T15:01:48Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+
+/** Tests for SQL plan history (Calcite engine). */
+@RunWith(Parameterized.class)
+public class SqlPlanHistoryIntegrationTest extends GridCommonAbstractTest {


Test already Parameterized. Perhaps it's worth to add parameters queryEngine and clientMode instead of creating child classes?

queryEngine and clientMode have been added to the list of parameters. Test subclasses for H2 and clients have been removed.

alex-plekhanov · 2024-09-25T15:04:45Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+        queryNode().context().query().runningQueryManager().resetPlanHistoryMetrics();
+
+        mapNode().context().query().runningQueryManager().resetPlanHistoryMetrics();


for (IgniteEx ignite: G.allGrids()) ignite.context().query().runningQueryManager().resetPlanHistoryMetrics();

alex-plekhanov · 2024-09-25T15:07:16Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+    /** Checks successful JDBC queries. */
+    @Test
+    public void testJdbcQuery() throws SQLException {
+        if (loc)


To ignore test it's better to use assumeTrue/assumeFalse statements. In your case assumeFalse(loc).
Tests will be marked as ignored instead of passed (and we can also provide a reason, why they are ignored).

SqlPlanHistoryIntegrationTest has been refactored using the assumeFalse statements.

alex-plekhanov · 2024-09-25T15:13:02Z

Please resolve also merge conflicts

alex-plekhanov · 2024-10-04T06:29:19Z

modules/spring/src/test/java/org/apache/ignite/testsuites/IgniteSpringTestSuite.java

-    CdcConfigurationTest.class
+    CdcConfigurationTest.class,
+
+    SqlPlanHistoryConfigTest.class


Let's add comma at the end

Comma added.

…indexing

…tyPlanHistory added, minor Javadoc changes

…g#executeUpdate0 to IgniteH2Indexing#executeUpdate

…dConcurrentLinkedHashSet, SqlPlanHistoryTracker#addPlan changed, SqlPlanHistoryIntegrationTest#testEntryReplacement added

This reverts commit fb2a47c.

…HistoryIntegrationTest refactored with assumeFalse(), SqlPlanHistoryTracker#setHistorySize edited

… to GridBoundedConcurrentLinkedHashMap

alex-plekhanov · 2024-11-02T15:38:45Z

...e/src/main/java/org/apache/ignite/internal/processors/query/running/RunningQueryManager.java

@@ -195,6 +213,11 @@ public RunningQueryManager(GridKernalContext ctx) {
            qryHistTracker.queryHistory().values(),
            SqlQueryHistoryView::new);

+        ctx.systemView().registerView(SQL_PLAN_HIST_VIEW, SQL_PLAN_HIST_VIEW_DESC,


You need to use method registerView with dataSupplier, since planHistTracker can be changed and planHistTracker.sqlPlanHistory() also can be changed, but you rely on the first created instances.

alex-plekhanov · 2024-11-02T15:55:40Z

...src/main/java/org/apache/ignite/internal/processors/query/running/SqlPlanHistoryTracker.java

+     * @param engine SQL engine.
+     */
+    public void addPlan(String plan, String qry, String schema, boolean loc, String engine) {
+        if (historySize <= 0)


It's not thread-safe and there is race possible. If method setHistorySize will be exposed (for example via JMX) someone can call this method with 0 value as historySize, and, in the same time addPlan will be called for query by another thead. In this case we can pass current line, face with null sqlPlanHistory field.
I propose do not store historySize at all.
Instead create instance emptyMap if we got historySize == 0:

public void setHistorySize(int historySize) { sqlPlanHistory = (historySize > 0) ? new GridBoundedConcurrentLinkedHashMap<>(historySize) : Collections.emptyMap(); }

In method addPlan we can check:

Map<SqlPlan, Long> sqlPlanHistory0 = sqlPlanHistory; if (sqlPlanHistory0 == Collections.emptyMap()) return; sqlPlanHistory0.put(sqlPlan, U.currentTimeMillis());

This approach will be thread safe.

Done. Tests verifying correct performance when history size is zero or less have been extended.

alex-plekhanov · 2024-11-02T16:02:43Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+     * @param idxTypes Index types.
+     * @return Cache configuration.
+     */
+    @SuppressWarnings("unchecked")


Remove it and change method to:

private CacheConfiguration<?, ?> configureCache(String name, Class<?>... idxTypes) { return new CacheConfiguration<>() ...

alex-plekhanov · 2024-11-02T16:05:04Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+        if (isClient)
+            assertTrue(node.context().clientNode());
+        else
+            assertFalse(node.context().clientNode());


assertEquals(isClient, node.context().clientNode());

alex-plekhanov · 2024-11-05T13:52:35Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+            {CalciteQueryEngineConfiguration.ENGINE_NAME},
+            {IndexingQueryEngineConfiguration.ENGINE_NAME}
+        }).flatMap(sqlEngine -> Arrays.stream(new Boolean[]{true, false})
+                .flatMap(isClient -> Arrays.stream(new Boolean[]{true, false})


// Local queries can't be executed on client nodes .flatMap(isClient -> Arrays.stream(isClient ? new Boolean[]{false} : new Boolean[]{true, false})

alex-plekhanov · 2024-11-05T15:47:28Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+    /** Checks TextQuery. */
+    @Test
+    public void testTextQuery() {
+        runQueryWithoutPlan(textQry);


As far as I know text query requires text (lucene) index to work. How does it work without index?

Apparently, for caches with built-in classes as data types, marking those classes as indexed types in the cache configuration is sufficient for text queries to work.

However, this is insufficient when custom classes are used as cache data types. In such cases, @QueryTextField annotations for the corresponding custom class fields will indeed be required.

There is a check in the cacheQuery() method that ensures all queries return a result, including those executed without a plan. If necessary, I can rewrite the text query test so that it uses a query from the Person cache and therefore a @QueryTextField annotation will be needed.

alex-plekhanov · 2024-11-05T16:10:52Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+
+        task.accept(cmds);
+
+        checkSqlPlanHistoryDml(3, qrysInfo.get2());


checkSqlPlanHistoryDml(cmds.size(), qrysInfo.get2());?

alex-plekhanov · 2024-11-05T16:17:30Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+                    check = "the following " + (loc ? "local " : "") + "query has been executed:";
+
+                plans = plans.entrySet().stream()
+                    .filter(e -> e.getKey().plan().contains(check))


Why do we filter these queries? Do we have another queries in the view?

Yes, in some cases we may also have plans for those select queries that are executed as part of the DML operation (with the H2 engine only).

alex-plekhanov · 2024-11-05T16:19:11Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+            String plan1 = sortedPlans.get(0).getKey().plan();
+            String plan2 = sortedPlans.get(1).getKey().plan();
+
+            assertTrue(plan2.contains(plan1) && plan2.contains("/* scanCount"));


Why do we have two plans?

The scanCount suffix issue has been resolved, the SqlPlanHistoryIntegrationTest#checkMetrics method has been refactored.

alex-plekhanov · 2024-11-05T16:23:39Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+     * @param size Number of SQL plan entries expected to be in the history.
+     */
+    public void checkSqlPlanHistory(int size) {
+        if (!isReducePhase && isClient && sqlEngine == IndexingQueryEngineConfiguration.ENGINE_NAME)


It's strange that we don't have plans in history for regular queries for client node and H2 engine. Do we need to check H2 & client at all? Map/reduce queries are only applicable to H2 engine and can be reproduced with two server nodes (even without distributed joins, for example with aggregation)

The isClient=TRUE parameter has been deleted from the H2 engine tests and the map/reduce query test has been refactored.

…ze and SqlPlanHistoryIntegrationTest#testResetPlanHistoryMetrics

…toryTracker#addPlan thread-safe

@SuppressWarnings

- get rid of '@SuppressWarnings("unchecked")' in configureCache() - improve readability of queryNode() by shortening the code - optimize params() by omitting TRUE for the isClient parameter - get rid of a hard-coded argument in executeDml()

… and refactor SqlPlanHistoryIntegrationTest#testSqlFieldsQueryWithReducePhase; refactor SqlPlanHistoryIntegrationTest#checkReset

…hat it returns the actual view content

…ier (not a collection) is passed to GridSystemViewManager#registerView

…tionTest#getSqlPlanHistory (order of entries is essential for testing local queries)

…iction - add assertion for waitForCondition() - remove redundant plan history check - change failed function queries to queries with unique placeholders

- move starting 3 grids for checking map nodes to testSqlFieldsQueryWithReducePhase - bring back testSqlFieldsQueryWithReducePhaseFailed - make javadoc for testSqlFieldsQueryWithReducePhase and testSqlFieldsQueryWithReducePhaseFailed more informative - change waiting mechanism in testEntryReplacement

…SqlPlanHistoryIntegrationTest

…hase and testSqlFieldsQueryWithReducePhaseFailed

- fix minor formatting issues - edit javadoc - remove redundant assumeFalse in startTestGrid

…ntegrationTest#checkReset

…= 0 condition, extend tests for verifying correct performance with histSize <= 0

alex-plekhanov · 2024-12-13T10:29:34Z

...src/main/java/org/apache/ignite/internal/processors/query/running/SqlPlanHistoryTracker.java

-
-        sqlPlanHistory = (historySize > 0) ? new GridBoundedConcurrentLinkedHashMap<>(historySize) : null;
+    public SqlPlanHistoryTracker(int histSize) {
+        sqlPlanHistory = (histSize > 0) ? new GridBoundedConcurrentLinkedHashMap<>(histSize) : Collections.emptyMap();


setHistorySize(histSize) to reduce duplication?

alex-plekhanov · 2024-12-13T10:30:13Z

...src/main/java/org/apache/ignite/internal/processors/query/running/SqlPlanHistoryTracker.java

@@ -47,7 +42,7 @@ public SqlPlanHistoryTracker(int historySize) {
     * @param engine SQL engine.
     */
    public void addPlan(String plan, String qry, String schema, boolean loc, String engine) {
-        if (historySize <= 0)
+        if (sqlPlanHistory.getClass().equals(Collections.emptyMap().getClass()))


Why not just sqlPlanHistory == Collections.emptyMap()?

We can't use sqlPlanHistory == Collections.emptyMap() because

Operator '==' cannot be applied to 'java.util.Map<org.apache.ignite.internal.processors.query.running.SqlPlan,java.lang.Long>', 'java.util.Map<java.lang.Object,java.lang.Object>'

We also can't use sqlPlanHistory.equals(Collections.emptyMap()) because AbstractMap#equals identifies two map as equal if they are both empty. So sqlPlanHistory.equals(Collections.emptyMap()) will return TRUE for a newly created sqlPlanHistory with size=0, and the method won't work correctly.

If however we compare classes of sqlPlanHistory and Collections.emptyMap(), then the method works as we intended and sqlPlanHistory.getClass().equals(Collections.emptyMap().getClass()) doesn't return TRUE for a newly created sqlPlanHistory.

Something like

Map<SqlPlan, Long> emptyMap = Collections.emptyMap(); if (sqlPlanHistory == emptyMap) return;

?

Yes, this options works. Change applied.

alex-plekhanov · 2024-12-13T10:42:17Z

modules/core/src/main/java/org/apache/ignite/spi/systemview/view/SqlPlanHistoryView.java

+    /**
+     * @return SQL plan entry.
+     */
+    public Map.Entry<SqlPlan, Long> sqlPlan() {


This method gives access to internals via public API. We should avoid this. Method used only once in test method getSqlPlanHistory. In this method it's more convinient to return something like List<SqlPlanHistoryView> than Map<SqlPlan, Long>. In case of List<SqlPlanHistoryView> - all columns can be checked thats exposed via public API (not only internal representation).

Method SqlPlanHistoryView#sqlPlan deleted. SqlPlanHistoryIntegrationTest#getSqlPlanHistory now returns List<SqlPlanHistoryView>.

alex-plekhanov · 2024-12-13T11:05:10Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

        });
    }

-    /** Checks failed SqlFieldsQuery with reduce phase. */
+    /**
+     * Checks failed SqlFieldsQuery with reduce phase. If the fisrt subquery fails, there should be only one entry in


Typo: fisrt

Test deleted.

alex-plekhanov · 2024-12-13T11:05:28Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

-    /** Checks failed SqlFieldsQuery with reduce phase. */
+    /**
+     * Checks failed SqlFieldsQuery with reduce phase. If the fisrt subquery fails, there should be only one entry in
+     * SQL plan history. If the fisrt subquery is successfully executed but the second one fails, the history should


Typo: fisrt

Test deleted.

alex-plekhanov · 2024-12-13T13:53:26Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+
+            cacheQuery(new SqlFieldsQuery(SQL + " where _val='STR" + i + "'").setLocal(loc), "A");
+
+            qryText[i - 1] = getSqlPlanHistory().keySet().stream().findFirst().map(SqlPlan::query).orElse("");


Check history size?
F.first?

History check added, F.first used instead of findFirst.

alex-plekhanov · 2024-12-13T13:56:56Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

     */
-    public void runQueryWithReducePhase(Runnable task) {
+    public void runQueryWithReducePhase(Runnable task) throws Exception {
+        assumeFalse("Map/reduce queries are only applicable to H2 engine",


assumeFalse(..., x != y) -> assumeTrue(..., x == y) for better readability

alex-plekhanov · 2024-12-13T13:57:22Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

@@ -572,19 +622,27 @@ public void runSqlFieldsDml(IgniteBiTuple<List<String>, Boolean> qrysInfo) {
     * @param qrysInfo DML commands info (queries, simple query flag).
     * @param task Task to be executed.
     */
-    public void executeDml(IgniteBiTuple<List<String>, Boolean> qrysInfo, Consumer<List<String>> task) {
+    public void executeDml(IgniteBiTuple<List<String>, Boolean> qrysInfo, Consumer<List<String>> task) throws Exception {
        assumeFalse("There is no lazy mode for DML operations", !isFullyFetched);


assumeFalse(..., !x) -> assumeTrue(..., x) for better readability

alex-plekhanov · 2024-12-13T14:14:13Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java


-            checkSqlPlanHistory((!isClient && sqlEngine == IndexingQueryEngineConfiguration.ENGINE_NAME) ? 3 : 1);
+            for (int i = 1; i <= 2; i++) {
+                Map<SqlPlan, Long> sqlPlansOnMapNode = grid(i).context().query().runningQueryManager()


Use view, not direct access to tracker.

alex-plekhanov · 2024-12-13T15:08:20Z

...ache/ignite/internal/processors/query/calcite/integration/SqlPlanHistoryIntegrationTest.java

+                    // No-op.
+                }
+
+                checkSqlPlanHistory( i + 1);


It's not intuitive clear that map-subqueries are executed one by one. It's not stated anywhere and subqueries can be run in parallel with some tuning or after some change in future. So, perhaps we should remove this test at all, since can't assert correct behaviour.

Test deleted.

…g setter in constructor

- remove test for failed query with reduce phase - change return type of getSqlPlanHistory to List<SqlPlanHistoryView> (and remove SqlPlanHistoryView#sqlPlan) - increase readability of assume statements - add overloaded getSqlPlanHistory with node as argument - add plan history check to checkReset - minor code corrections

# Conflicts: # modules/calcite/src/test/java/org/apache/ignite/testsuites/IntegrationTestSuite.java

- remove query variables that are used only once - edit javadoc - remove code duplication by moving setLocal to cacheQuery

…l empty map variable in SqlPlanHistoryTracker#add

…lPlanHistoryIntegrationTest

# Conflicts: # modules/calcite/src/test/java/org/apache/ignite/testsuites/IntegrationTestSuite.java

…he#11425. Signed-off-by: Aleksey Plekhanov <[email protected]>

alex-plekhanov reviewed Aug 16, 2024

View reviewed changes

alex-plekhanov reviewed Sep 24, 2024

View reviewed changes

alex-plekhanov reviewed Sep 25, 2024

View reviewed changes

alex-plekhanov reviewed Oct 4, 2024

View reviewed changes

oleg-vlsk and others added 25 commits October 8, 2024 07:17

IGNITE-22557 Tests added

8652ad7

IGNITE-22557 SQL Plan History view introduced

7485803

IGNITE-22557 SQL Plan History Configuration Test removed from ignite-…

2e4f27c

…indexing

IGNITE-22557 Tests added to IgniteCalciteTestSuite

95ab41b

IGNITE-22557 Tests minor refactoring

318ffdf

IGNITE-22557 Minor refactoring

bfd9fe4

IGNITE-22557 Minor refactoring

21b5807

IGNITE-22557 Minor refactoring

40f2291

IGNITE-22557 Minor UpdatePlan#plan refactoring

adeefb5

IGNITE-22557 Sequence of columns in the view changed

63831eb

IGNITE-22557 SqlPlanHistoryH2SelfTest refactoring

91833e4

IGNITE-22557 Description of the view added to system-views.adoc

27f46ab

IGNITE-22557 Tests refactoring, SqlPlanHistoryCalciteSelfTest#testEmp…

a338ccd

…tyPlanHistory added, minor Javadoc changes

IGNITE-22557 Adding plans for DML commands moved from IgniteH2Indexin…

ed7380c

…g#executeUpdate0 to IgniteH2Indexing#executeUpdate

IGNITE-22557 Changes made according to code review

9cb98f9

IGNITE-22557 Minor tests refactoring

febb24f

IGNITE-22557 GridBoundedConcurrentLinkedHashMap changed to GridBounde…

1667294

…dConcurrentLinkedHashSet, SqlPlanHistoryTracker#addPlan changed, SqlPlanHistoryIntegrationTest#testEntryReplacement added

IGNITE-22557 Whitespaces refactoring

8a437a2

IGNITE-22557 Whitespaces refactoring

886c763

Delete changes in UpdatePlan

657b86b

Revert "Delete changes in UpdatePlan"

be4551a

This reverts commit fb2a47c.

IGNITE-22557 Whitespaces refactoring

2177847

IGNITE-22557 Whitespaces refactoring

6bb9e06

IGNITE-22557 SqlPlanHistoryConfigTest moved to ignite-spring, SqlPlan…

776339b

…HistoryIntegrationTest refactored with assumeFalse(), SqlPlanHistoryTracker#setHistorySize edited

IGNITE-22557 GridBoundedConcurrentLinkedHashSet has been changed back…

5fa16ac

… to GridBoundedConcurrentLinkedHashMap

alex-plekhanov reviewed Nov 5, 2024

View reviewed changes

oleg-vlsk added 14 commits November 6, 2024 20:40

IGNITE-22557 Add SqlPlanHistoryIntegrationTest#testResetPlanHistorySi…

08c4907

…ze and SqlPlanHistoryIntegrationTest#testResetPlanHistoryMetrics

IGNITE-22557 Make SqlPlanHistoryTracker#setHistorySize and SqlPlanHis…

8fc1425

…toryTracker#addPlan thread-safe

IGNITE-22557 Remove the 'isClient=TRUE' parameter for H2 engine tests…

afbd7e1

… and refactor SqlPlanHistoryIntegrationTest#testSqlFieldsQueryWithReducePhase; refactor SqlPlanHistoryIntegrationTest#checkReset

IGNITE-22557 Fix SqlPlanHistoryIntegrationTest#getSqlPlanHistory so t…

e25a744

…hat it returns the actual view content

IGNITE-22557 Fix RunningQueryManager constructor so that a data suppl…

662ed3a

…ier (not a collection) is passed to GridSystemViewManager#registerView

IGNITE-22557 Change HashMap to LinkedHashMap in SqlPlanHistoryIntegra…

76ba339

…tionTest#getSqlPlanHistory (order of entries is essential for testing local queries)

IGNITE-22557 Refactor SqlPlanHistoryIntegrationTest#testPlanHistoryEv…

887261c

…iction - add assertion for waitForCondition() - remove redundant plan history check - change failed function queries to queries with unique placeholders

IGNITE-22557 Move grid initiation from beforeTest to actual tests in …

0dffe6e

…SqlPlanHistoryIntegrationTest

IGNITE-22557 Remove code duplication in testSqlFieldsQueryWithReduceP…

b32a74f

…hase and testSqlFieldsQueryWithReducePhaseFailed

IGNITE-22557 Refactor SqlPlanHistoryIntegrationTest

a20ee48

- fix minor formatting issues - edit javadoc - remove redundant assumeFalse in startTestGrid

IGNITE-22557 Change fail() to unique string values in SqlPlanHistoryI…

66c9605

…ntegrationTest#checkReset

IGNITE-22557 Fix SqlPlanHistoryTracker constructor for the histSize <…

e6dc953

…= 0 condition, extend tests for verifying correct performance with histSize <= 0

alex-plekhanov reviewed Dec 13, 2024

View reviewed changes

oleg-vlsk added 10 commits December 15, 2024 11:50

IGNITE-22557 Remove code duplication in SqlPlanHistoryTracker by usin…

e93319f

…g setter in constructor

Merge remote-tracking branch 'origin/master' into ignite-22557

a0a833c

# Conflicts: # modules/calcite/src/test/java/org/apache/ignite/testsuites/IntegrationTestSuite.java

IGNITE-22557 Refactor SqlPlanHistoryIntegrationTest

236c46d

- remove query variables that are used only once - edit javadoc - remove code duplication by moving setLocal to cacheQuery

IGNITE-22557 Edit Javadoc in SqlPlanHistoryIntegrationTest

2b76f6c

IGNITE-22557 Change class comparison to comparison with the additiona…

cbe3a94

…l empty map variable in SqlPlanHistoryTracker#add

IGNITE-22557 Remove code duplication in cross cache query tests in Sq…

0cde912

…lPlanHistoryIntegrationTest

IGNITE-22557 Fix issue with the scanCount suffix in local query plans

e39e452

Merge remote-tracking branch 'origin/master' into ignite-22557

d729e18

# Conflicts: # modules/calcite/src/test/java/org/apache/ignite/testsuites/IntegrationTestSuite.java

IGNITE-22557 Add SqlPlanHistoryIntegrationTest#testNoScanCountSuffix

982f9e5

alex-plekhanov approved these changes Dec 20, 2024

View reviewed changes

asfgit closed this in 2d69600 Dec 23, 2024

luchnikovbsk pushed a commit to luchnikovbsk/ignite that referenced this pull request Jan 31, 2025

IGNITE-22557 Add system view for SQL query plans history - Fixes apac…

85c6849

…he#11425. Signed-off-by: Aleksey Plekhanov <[email protected]>

		@@ -80,10 +80,10 @@ public final class UpdatePlan {

		/** Number of rows in rows based MERGE or INSERT. */
		private final int rowsNum;


		SqlPlan sqlPlan = new SqlPlan(plan, qry, schema, loc, engine);

		sqlPlanHistory.put(sqlPlan, sqlPlan.startTime());

		queryNode().context().query().runningQueryManager().resetPlanHistoryMetrics();

		mapNode().context().query().runningQueryManager().resetPlanHistoryMetrics();


		task.accept(cmds);

		checkSqlPlanHistoryDml(3, qrysInfo.get2());


		cacheQuery(new SqlFieldsQuery(SQL + " where _val='STR" + i + "'").setLocal(loc), "A");

		qryText[i - 1] = getSqlPlanHistory().keySet().stream().findFirst().map(SqlPlan::query).orElse("");

IGNITE-22557 Create system view for SQL query plans history #11425

IGNITE-22557 Create system view for SQL query plans history #11425

Conversation

oleg-vlsk commented Jul 10, 2024

The Contribution Checklist

Notes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-vlsk Sep 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alex-plekhanov commented Sep 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-vlsk Nov 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-vlsk Nov 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-vlsk Nov 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-vlsk Dec 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-vlsk Dec 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-vlsk Sep 25, 2024 •

edited

Loading

oleg-vlsk Nov 8, 2024 •

edited

Loading

oleg-vlsk Nov 8, 2024 •

edited

Loading

oleg-vlsk Nov 8, 2024 •

edited

Loading

oleg-vlsk Dec 14, 2024 •

edited

Loading

oleg-vlsk Dec 15, 2024 •

edited

Loading