Skip to content

draft: augment yaml definitions with test schema #1659

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 21 commits into
base: v2.x
Choose a base branch
from
Draft

draft: augment yaml definitions with test schema #1659

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
39224b6
WIP
addaleax Feb 18, 2025
56a801b
add tests schema
nirinchev Apr 7, 2025
7e64509
Revert "add tests schema"
nirinchev May 20, 2025
07cd9fc
run yamlfix
nirinchev May 22, 2025
7cb2274
Reformat after running the js generator
nirinchev May 22, 2025
cb2571c
generate schema for accumulators
nirinchev May 22, 2025
e28803d
add test schemas for expressions
nirinchev May 22, 2025
fd06dd6
add test schemas for query
nirinchev May 22, 2025
64a44ad
add search schemas
nirinchev May 22, 2025
6a34eda
schema for stages
nirinchev May 23, 2025
cc2c832
Update with some refs
nirinchev May 26, 2025
c027132
type tweaks
nirinchev May 26, 2025
3602303
fix names to align with docs
nirinchev Jun 3, 2025
7978e4f
regenerate test schemas
nirinchev Jun 3, 2025
b9b1d19
add searchHighlight as a type
nirinchev Jun 4, 2025
c2b1ee5
Add schema for first/last
nirinchev Jun 4, 2025
14623eb
make test name and pipeline required
nirinchev Jun 9, 2025
943deb1
chore: update a few schemas
nirinchev Jun 9, 2025
37a957c
add untypedPipeline type for pipelines that reference foreign collect…
nirinchev Jun 9, 2025
34b9a20
Use proper tags instead of ejson syntax
nirinchev Jun 11, 2025
4a2867b
correctly escape the regex dots
nirinchev Jun 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions generator/config/.yamlfix.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
allow_duplicate_keys = false
comments_min_spaces_from_content = 1
sequence_style = "block_style"
indent_mapping = 4
indent_offset = 4
indent_sequence = 6
none_representation = "~"
explicit_start = false
255 changes: 136 additions & 119 deletions generator/config/accumulator/accumulator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,132 +1,149 @@
# $schema: ../schema.json
name: $accumulator
link: 'https://www.mongodb.com/docs/manual/reference/operator/aggregation/accumulator/'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to keep quotes around strings? This style changes make the diff hard to read. Also, I use quotes by default around every string to prevent unexpected parsing rule from Yaml.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, looks like the quotes usage is inconsistent in these files - I'd be inclined to define a prettier rule and reformat them in a separate PR to make this one less noisy if that'd make sense to you. The reason for these getting removed is that the way we're adding the schema information is parsing the yaml, fetching the schema from the docs, then writing it, which results in the file getting reformatted based on the tool conventions (we use https://www.npmjs.com/package/js-yaml here).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, at least prettier supports YAML out of the box, so that feels like a great way to remove any formatting preference ambiguity

link: https://www.mongodb.com/docs/manual/reference/operator/aggregation/accumulator/
type:
- accumulator
encode: object
description: |
Defines a custom accumulator function.
New in MongoDB 4.4.
arguments:
-
name: init
type:
- javascript
description: |
Function used to initialize the state. The init function receives its arguments from the initArgs array expression. You can specify the function definition as either BSON type Code or String.
-
name: initArgs
type:
- resolvesToArray
optional: true
description: |
Arguments passed to the init function.
-
name: accumulate
type:
- javascript
description: |
Function used to accumulate documents. The accumulate function receives its arguments from the current state and accumulateArgs array expression. The result of the accumulate function becomes the new state. You can specify the function definition as either BSON type Code or String.
-
name: accumulateArgs
type:
- resolvesToArray
description: |
Arguments passed to the accumulate function. You can use accumulateArgs to specify what field value(s) to pass to the accumulate function.
-
name: merge
type:
- javascript
description: |
Function used to merge two internal states. merge must be either a String or Code BSON type. merge returns the combined result of the two merged states. For information on when the merge function is called, see Merge Two States with $merge.
-
name: finalize
type:
- javascript
optional: true
description: |
Function used to update the result of the accumulation.
-
name: lang
type:
- string
description: |
The language used in the $accumulator code.

- name: init
type:
- javascript
description: |
Function used to initialize the state. The init function receives its arguments from the initArgs array expression. You can specify the function definition as either BSON type Code or String.
- name: initArgs
type:
- resolvesToArray
optional: true
description: |
Arguments passed to the init function.
- name: accumulate
type:
- javascript
description: |
Function used to accumulate documents. The accumulate function receives its arguments from the current state and accumulateArgs array expression. The result of the accumulate function becomes the new state. You can specify the function definition as either BSON type Code or String.
- name: accumulateArgs
type:
- resolvesToArray
description: |
Arguments passed to the accumulate function. You can use accumulateArgs to specify what field value(s) to pass to the accumulate function.
- name: merge
type:
- javascript
description: |
Function used to merge two internal states. merge must be either a String or Code BSON type. merge returns the combined result of the two merged states. For information on when the merge function is called, see Merge Two States with $merge.
- name: finalize
type:
- javascript
optional: true
description: |
Function used to update the result of the accumulation.
- name: lang
type:
- string
description: |
The language used in the $accumulator code.
tests:
-
name: 'Use $accumulator to Implement the $avg Operator'
link: 'https://www.mongodb.com/docs/manual/reference/operator/aggregation/accumulator/#use--accumulator-to-implement-the--avg-operator'
pipeline:
-
$group:
_id: '$author'
avgCopies:
$accumulator:
init:
$code: |-
function() {
return { count: 0, sum: 0 }
- name: Use $accumulator to Implement the $avg Operator
link: https://www.mongodb.com/docs/manual/reference/operator/aggregation/accumulator/#use--accumulator-to-implement-the--avg-operator
pipeline:
- $group:
_id: $author
avgCopies:
$accumulator:
init:
$code: |-
function() {
return { count: 0, sum: 0 }
}
accumulate:
$code: |-
function(state, numCopies) {
return { count: state.count + 1, sum: state.sum + numCopies }
}
accumulateArgs:
- $copies
merge:
$code: |-
function(state1, state2) {
return {
count: state1.count + state2.count,
sum: state1.sum + state2.sum
}
accumulate:
$code: |-
function(state, numCopies) {
return { count: state.count + 1, sum: state.sum + numCopies }
}
finalize:
$code: |-
function(state) {
return (state.sum / state.count)
}
lang: js
schema:
books:
_id:
types:
- bsonType: Number
title:
types:
- bsonType: String
author:
types:
- bsonType: String
copies:
types:
- bsonType: Number
- name: Use initArgs to Vary the Initial State by Group
link: https://www.mongodb.com/docs/manual/reference/operator/aggregation/accumulator/#use-initargs-to-vary-the-initial-state-by-group
pipeline:
- $group:
_id:
city: $city
restaurants:
$accumulator:
init:
$code: |-
function(city, userProfileCity) {
return { max: city === userProfileCity ? 3 : 1, restaurants: [] }
}
initArgs:
- $city
- Bettles
accumulate:
$code: |-
function(state, restaurantName) {
if (state.restaurants.length < state.max) {
state.restaurants.push(restaurantName);
}
accumulateArgs: [ "$copies" ]
merge:
$code: |-
function(state1, state2) {
return {
count: state1.count + state2.count,
sum: state1.sum + state2.sum
}
return state;
}
accumulateArgs:
- $name
merge:
$code: |-
function(state1, state2) {
return {
max: state1.max,
restaurants: state1.restaurants.concat(state2.restaurants).slice(0, state1.max)
}
finalize:
$code: |-
function(state) {
return (state.sum / state.count)
}
lang: 'js'

-
name: 'Use initArgs to Vary the Initial State by Group'
link: 'https://www.mongodb.com/docs/manual/reference/operator/aggregation/accumulator/#use-initargs-to-vary-the-initial-state-by-group'
pipeline:
-
$group:
_id:
city: '$city'
restaurants:
$accumulator:
init:
$code: |-
function(city, userProfileCity) {
return { max: city === userProfileCity ? 3 : 1, restaurants: [] }
}
initArgs:
- '$city'
- 'Bettles'
accumulate:
$code: |-
function(state, restaurantName) {
if (state.restaurants.length < state.max) {
state.restaurants.push(restaurantName);
}
return state;
}
accumulateArgs: ['$name']
merge:
$code: |-
function(state1, state2) {
return {
max: state1.max,
restaurants: state1.restaurants.concat(state2.restaurants).slice(0, state1.max)
}
}
finalize:
$code: |-
function(state) {
return state.restaurants
}
lang: 'js'
}
finalize:
$code: |-
function(state) {
return state.restaurants
}
lang: js
schema:
restaurants:
_id:
types:
- bsonType: Number
name:
types:
- bsonType: String
city:
types:
- bsonType: String
cuisine:
types:
- bsonType: String
92 changes: 62 additions & 30 deletions generator/config/accumulator/addToSet.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# $schema: ../schema.json
name: $addToSet
link: 'https://www.mongodb.com/docs/manual/reference/operator/aggregation/addToSet/'
link: https://www.mongodb.com/docs/manual/reference/operator/aggregation/addToSet/
type:
- accumulator
- window
Expand All @@ -9,39 +9,71 @@ description: |
Returns an array of unique expression values for each group. Order of the array elements is undefined.
Changed in MongoDB 5.0: Available in the $setWindowFields stage.
arguments:
-
name: expression
type:
- expression

- name: expression
type:
- expression
tests:
-
name: 'Use in $group Stage'
link: 'https://www.mongodb.com/docs/manual/reference/operator/aggregation/addToSet/#use-in--group-stage'
pipeline:
- $group:
- name: Use in $group Stage
link: https://www.mongodb.com/docs/manual/reference/operator/aggregation/addToSet/#use-in--group-stage
pipeline:
- $group:
_id:
day:
$dayOfYear:
date: '$date'
date: $date
year:
$year:
date: '$date'
date: $date
itemsSold:
$addToSet: '$item'
-
name: 'Use in $setWindowFields Stage'
link: 'https://www.mongodb.com/docs/manual/reference/operator/aggregation/addToSet/#use-in--setwindowfields-stage'
pipeline:
-
$setWindowFields:
partitionBy: '$state'
sortBy:
orderDate: 1
output:
cakeTypesForState:
$addToSet: '$type'
window:
documents:
- 'unbounded'
- 'current'
$addToSet: $item
schema:
sales:
_id:
types:
- bsonType: Number
item:
types:
- bsonType: String
price:
types:
- bsonType: Number
quantity:
types:
- bsonType: Number
date:
types:
- bsonType: Date
- name: Use in $setWindowFields Stage
link: https://www.mongodb.com/docs/manual/reference/operator/aggregation/addToSet/#use-in--setwindowfields-stage
pipeline:
- $setWindowFields:
partitionBy: $state
sortBy:
orderDate: 1
output:
cakeTypesForState:
$addToSet: $type
window:
documents:
- unbounded
- current
schema:
cakeSales:
_id:
types:
- bsonType: Number
type:
types:
- bsonType: String
orderDate:
types:
- bsonType: Date
state:
types:
- bsonType: String
price:
types:
- bsonType: Number
quantity:
types:
- bsonType: Number
Loading
Loading