-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
2 changed files
with
99 additions
and
13 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,4 @@ | ||
combined.json | ||
/misc | ||
*.json | ||
README.md |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,29 +1,114 @@ | ||
# Combine-json | ||
|
||
Combine-json is a CLI tool that combines all json files found in a directory into one big json file using streaming. | ||
Combine-json is a CLI tool that combines all json files found in a directory into one big json file using streaming to avoid out-of-memory. | ||
|
||
It takes as argument a directory in which to find the json files and a output file name in which the json files will combined. | ||
The json file will be created in the directory where the CLI is used. | ||
The output JSON file contains an array with all the inputed JSON files. | ||
So if your JSON file contained `{ "id": 1 }` it will be stored in the output file like this: | ||
|
||
Input files: | ||
|
||
file 1: | ||
```json | ||
{ "id": 1 } | ||
``` | ||
|
||
file 2: | ||
```json | ||
{ "id": 2 } | ||
``` | ||
|
||
Output file: | ||
```json | ||
[ | ||
{ "id": 1 }, | ||
{ "id": 2 } | ||
] | ||
``` | ||
|
||
**Array exception**: | ||
There is one exception to this rule. If your JSON file contains an array, it will be deconstructed in the final file (_could become an option please make an issue if you'd like that_). | ||
|
||
Input files: | ||
|
||
```json | ||
[ 1, 2, 3 ] | ||
``` | ||
|
||
```json | ||
{ "id": 1 } | ||
``` | ||
|
||
Output file: | ||
|
||
```json | ||
[ | ||
1, | ||
2, | ||
3, | ||
{ "id": 1 } | ||
] | ||
``` | ||
|
||
It accepts either json object files and json array files. | ||
It accepts relative path but also fully qualified paths. | ||
|
||
## Installation | ||
|
||
CLI: | ||
|
||
```bash | ||
npm i | ||
npm run link | ||
npx combine-json [options] | ||
``` | ||
|
||
Library: | ||
|
||
```bash | ||
# yarn | ||
yarn add combine-json | ||
|
||
# npm | ||
npm install combine-json | ||
``` | ||
|
||
|
||
## Usage | ||
`combine-json [inputDir] [outputFile(optionnal)]` | ||
|
||
Combine-json has two options: | ||
|
||
- Input dir: the path to the directory containing all the jsons. | ||
- Output file _optional_: path to the output file. | ||
|
||
It accepts relative path but also fully qualified paths. | ||
|
||
### CLI usage: | ||
|
||
```bash | ||
combine-json directory | ||
combine-json --input-dir <dir-path> --output-file <file-path> (default: "combined.json") | ||
``` | ||
default output file is combined.json at the root of where you execute the cli. | ||
|
||
**Example** | ||
```bash | ||
combine-json /data combined_data.json | ||
``` | ||
|
||
### Library usage | ||
|
||
```js | ||
const { combineJson } = require('./src'); | ||
|
||
(async () => { | ||
await combineJson({ inputDir: 'misc', outputFile: 'combined_data.json' }); | ||
})(); | ||
``` | ||
|
||
### Example | ||
|
||
Using the JSON files present in `misc`, you can observe the outputed file [misc_output.json](./misc_output.json) | ||
|
||
## Try it out | ||
Go at the root of the CLI | ||
|
||
At the root of the repository use the following: | ||
|
||
```bash | ||
combine-json misc | ||
npx combine-json misc | ||
``` | ||
it will generate a combined.json file with all the json object found in misc. | ||
|
||
it will generate a combined.json file with all the JSON objects found in misc. |