comments

proto/substrait/algebra.proto

@@ -328,7 +328,6 @@ message SetRel {
   // inputs.  There must be at least two inputs.
   repeated Rel inputs = 2;
   SetOp op = 3;
-  SetMode mode = 4;
   substrait.extensions.AdvancedExtension advanced_extension = 10;
 
   enum SetOp {
@@ -337,17 +336,10 @@ message SetRel {
     SET_OP_MINUS_MULTISET = 2;
     SET_OP_INTERSECTION_PRIMARY = 3;
     SET_OP_INTERSECTION_MULTISET = 4;
-    // Deprecated: Use SET_OP_UNION in conjunction with the mode field
-    // (SET_OP_MODE_DISTINCT or SET_OP_MODE_ALL) instead.
-    SET_OP_UNION_DISTINCT = 5 [deprecated = true];
-    SET_OP_UNION_ALL = 6 [deprecated = true];
-    SET_OP_UNION = 7;
-  }
-
-  enum SetMode {
-    SET_MODE_UNSPECIFIED = 0;
-    SET_MODE_DISTINCT = 1;
-    SET_MODE_ALL = 2;
+    SET_OP_UNION_DISTINCT = 5;
+    SET_OP_UNION_ALL = 6;
+    SET_OP_MINUS_DISTINCT = 7;
+    SET_OP_INTERSECTION_DISTINCT = 8;
   }
 }
 

site/docs/relations/logical_relations.md

@@ -270,12 +270,14 @@ The set operation type determines both the records that are emitted and the type
 
 | Property                | Description                                                                                                   | Output Nullability
 | ----------------------- | ------------------------------------------------------------------------------------------------------------- | ----------------------------- |
-| Minus (Primary)         | Returns all records from the primary input excluding any matching records from secondary inputs.              | The same as the primary input.
+| Minus (Primary)         | Returns all records (including duplicates) from the primary input excluding any matching records from secondary inputs.              | The same as the primary input.
 | Minus (Multiset)        | Returns all records from the primary input excluding any records that are included in *all* secondary inputs. | The same as the primary input.
-| Intersection (Primary)  | Returns all records from the primary input that match at least one record from *any* secondary inputs.        | If a field is nullable in the primary input and in any of the secondary inputs, it is nullable in the output.
+| Intersection (Primary)  | Returns all records (including duplicates) from the primary input that match at least one record from *any* secondary inputs.        | If a field is nullable in the primary input and in any of the secondary inputs, it is nullable in the output.
 | Intersection (Multiset) | Returns all records from the primary input that match at least one record from *all* secondary inputs.        | If a field is required in any of the inputs, it is required in the output.
 | Union Distinct          | Returns all the records from each set, removing any rows that are duplicated (within or across sets).         | If a field is nullable in any of the inputs, it is nullable in the output.
 | Union All               | Returns all records from each set, allowing duplicates.                                                       | If a field is nullable in any of the inputs, it is nullable in the output. |
+| Minus Distinct          | Returns all records (excluding duplicates) from the primary input excluding any matching records from secondary inputs.       | The same as the primary input. |
+| Intersection Distinct   | Returns all records (excluding duplicates) from the primary input that match any records in secondary inputs.                 | If a field is nullable in the primary input and in any of the secondary inputs, it is nullable in the output. |
 
 Note that for set operations, NULL matches NULL. That is
 ```
@@ -302,6 +304,8 @@ The output type is as follows for the various operations
 | Intersection (Multiset) | (R, R, R, R, R, R, R, N)
 | Union Distinct          | (R, N, N, N, N, N, N, N)
 | Union All               | (R, N, N, N, N, N, N, N)
+| Minus Distinct          | (R, R, R, R, N, N, N, N)
+| Intersection Distinct   | (R, R, R, R, R, N, N, N)
 
 
 === "SetRel Message"