Merge pull request #632 from commercetools/improve-performance-graphq…

…l-and-cache

Prepare 3.0.1

README.md

@@ -2,9 +2,9 @@
 # commercetools sync
 [![Build Status](https://travis-ci.org/commercetools/commercetools-sync-java.svg?branch=master)](https://travis-ci.org/commercetools/commercetools-sync-java)
 [![codecov](https://codecov.io/gh/commercetools/commercetools-sync-java/branch/master/graph/badge.svg)](https://codecov.io/gh/commercetools/commercetools-sync-java)
-[![Benchmarks 3.0.0](https://img.shields.io/badge/Benchmarks-3.0.0-orange.svg)](https://commercetools.github.io/commercetools-sync-java/benchmarks/)
+[![Benchmarks 3.0.1](https://img.shields.io/badge/Benchmarks-3.0.1-orange.svg)](https://commercetools.github.io/commercetools-sync-java/benchmarks/)
 [![Download](https://api.bintray.com/packages/commercetools/maven/commercetools-sync-java/images/download.svg) ](https://bintray.com/commercetools/maven/commercetools-sync-java/_latestVersion)
-[![Javadoc](http://javadoc-badge.appspot.com/com.commercetools/commercetools-sync-java.svg?label=Javadoc)](https://commercetools.github.io/commercetools-sync-java/v/3.0.0/)
+[![Javadoc](http://javadoc-badge.appspot.com/com.commercetools/commercetools-sync-java.svg?label=Javadoc)](https://commercetools.github.io/commercetools-sync-java/v/3.0.1/)
 [![Known Vulnerabilities](https://snyk.io/test/github/commercetools/commercetools-sync-java/4b2e26113d591bda158217c5dc1cf80a88665646/badge.svg)](https://snyk.io/test/github/commercetools/commercetools-sync-java/4b2e26113d591bda158217c5dc1cf80a88665646)
 
 More at https://commercetools.github.io/commercetools-sync-java
@@ -40,7 +40,7 @@ The library supports synchronising the following entities in commercetools
     - [Ivy](#ivy)
 - [Roadmap](#roadmap)
 - [Release Notes](/docs/RELEASE_NOTES.md)
-- [Javadoc](https://commercetools.github.io/commercetools-sync-java/v/3.0.0/)
+- [Javadoc](https://commercetools.github.io/commercetools-sync-java/v/3.0.1/)
 - [Benchmarks](https://commercetools.github.io/commercetools-sync-java/benchmarks/)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
@@ -67,7 +67,7 @@ commercetools sync is a Java library that imports commercetools platform data in
 
  - Make sure you have `JDK 8` installed.
  - [commercetools-jvm-sdk](https://github.com/commercetools/commercetools-jvm-sdk) as a dependency in your JVM-based 
-  application. (Make sure to use a version `>= 1.54.0`).
+  application. (Make sure to use a version `>= 1.55.0`).
  - a target commercetools project for syncing your source data to.
 
 
@@ -82,24 +82,24 @@ Here are the most popular ones:
 <dependency>
   <groupId>com.commercetools</groupId>
   <artifactId>commercetools-sync-java</artifactId>
-  <version>3.0.0</version>
+  <version>3.0.1</version>
 </dependency>
 ````
 
 #### Gradle
 
 ````groovy
-implementation 'com.commercetools:commercetools-sync-java:3.0.0'
+implementation 'com.commercetools:commercetools-sync-java:3.0.1'
 ````
 
 #### SBT 
 
 ````
-libraryDependencies += "com.commercetools" % "commercetools-sync-java" % "3.0.0"
+libraryDependencies += "com.commercetools" % "commercetools-sync-java" % "3.0.1"
 ````
 
 #### Ivy 
 
 ````xml
-<dependency org="com.commercetools" name="commercetools-sync-java" rev="3.0.0"/>
+<dependency org="com.commercetools" name="commercetools-sync-java" rev="3.0.1"/>
 ````

build.gradle

@@ -16,6 +16,7 @@ ext{
     pmdVersion = '6.14.0'
     jacocoVersion = '0.8.4'
     findbugsVersion = '3.0.1'
+    caffeineVersion = '2.8.5'
 }
 
 apply from: "$rootDir/gradle-scripts/plugins.gradle"
@@ -46,10 +47,11 @@ dependencies {
     implementation "com.commercetools.sdk.jvm.core:commercetools-java-client:${commercetoolsJvmSdkVersion}"
     implementation "com.commercetools.sdk.jvm.core:commercetools-convenience:${commercetoolsJvmSdkVersion}"
     implementation "com.google.code.findbugs:annotations:${findbugsVersion}"
+    implementation "com.github.ben-manes.caffeine:caffeine:${caffeineVersion}"
     testImplementation "org.mockito:mockito-junit-jupiter:${mockitoJunitJupiterVersion}"
     testImplementation "org.junit.jupiter:junit-jupiter-api:${jupiterApiVersion}"
     testImplementation "org.junit.jupiter:junit-jupiter-params:${jupiterApiVersion}"
     testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine:${jupiterApiVersion}"
     testRuntimeOnly "org.junit.vintage:junit-vintage-engine:${jupiterApiVersion}"
     testImplementation "org.assertj:assertj-core:${assertjVersion}"
-}
+}

docs/README.md

@@ -2,9 +2,9 @@
 # commercetools sync
 [![Build Status](https://travis-ci.org/commercetools/commercetools-sync-java.svg?branch=master)](https://travis-ci.org/commercetools/commercetools-sync-java)
 [![codecov](https://codecov.io/gh/commercetools/commercetools-sync-java/branch/master/graph/badge.svg)](https://codecov.io/gh/commercetools/commercetools-sync-java)
-[![Benchmarks 3.0.0](https://img.shields.io/badge/Benchmarks-3.0.0-orange.svg)](https://commercetools.github.io/commercetools-sync-java/benchmarks/)
+[![Benchmarks 3.0.1](https://img.shields.io/badge/Benchmarks-3.0.1-orange.svg)](https://commercetools.github.io/commercetools-sync-java/benchmarks/)
 [![Download](https://api.bintray.com/packages/commercetools/maven/commercetools-sync-java/images/download.svg) ](https://bintray.com/commercetools/maven/commercetools-sync-java/_latestVersion)
-[![Javadoc](http://javadoc-badge.appspot.com/com.commercetools/commercetools-sync-java.svg?label=Javadoc)](https://commercetools.github.io/commercetools-sync-java/v/3.0.0/)
+[![Javadoc](http://javadoc-badge.appspot.com/com.commercetools/commercetools-sync-java.svg?label=Javadoc)](https://commercetools.github.io/commercetools-sync-java/v/3.0.1/)
 [![Known Vulnerabilities](https://snyk.io/test/github/commercetools/commercetools-sync-java/4b2e26113d591bda158217c5dc1cf80a88665646/badge.svg)](https://snyk.io/test/github/commercetools/commercetools-sync-java/4b2e26113d591bda158217c5dc1cf80a88665646)
 
 
@@ -58,18 +58,18 @@ Here are the most popular ones:
 <dependency>
   <groupId>com.commercetools</groupId>
   <artifactId>commercetools-sync-java</artifactId>
-  <version>3.0.0</version>
+  <version>3.0.1</version>
 </dependency>
 ````
 #### Gradle
 ````groovy
-implementation 'com.commercetools:commercetools-sync-java:3.0.0'
+implementation 'com.commercetools:commercetools-sync-java:3.0.1'
 ````
 #### SBT 
 ````
-libraryDependencies += "com.commercetools" % "commercetools-sync-java" % "3.0.0"
+libraryDependencies += "com.commercetools" % "commercetools-sync-java" % "3.0.1"
 ````
 #### Ivy 
 ````xml
-<dependency org="com.commercetools" name="commercetools-sync-java" rev="3.0.0"/>
+<dependency org="com.commercetools" name="commercetools-sync-java" rev="3.0.1"/>
 ````

docs/RELEASE_NOTES.md

@@ -28,6 +28,19 @@
 
 -->
 
+### 3.0.1 - Nov 24, 2020
+[Commits](https://github.com/commercetools/commercetools-sync-java/compare/3.0.0...3.0.1) |
+[Javadoc](https://commercetools.github.io/commercetools-sync-java/v/3.0.1/) | 
+[Jar](https://bintray.com/commercetools/maven/commercetools-sync-java/3.0.1)
+
+- ✨ **Enhancements** (1)
+    -  To improve performance of the library, the services are using graphQL API to fetch resource ids only; also the 
+    `keyToId` caches evict entries which haven't been used for the longest amount of time beyond a maximum size. The cache 
+    size is configurable in the sync options. [#582](https://github.com/commercetools/commercetools-sync-java/issues/582)
+
+- 🛠️ **Dependency Updates** (1)
+    - `commercetools-jvm-sdk` `1.54.0` -> [`1.55.0`](http://commercetools.github.io/commercetools-jvm-sdk/apidocs/io/sphere/sdk/meta/ReleaseNotes.html#v1_55_0)
+
 ### 3.0.0 - Nov 18, 2020
 [Commits](https://github.com/commercetools/commercetools-sync-java/compare/2.3.0...3.0.0) |
 [Javadoc](https://commercetools.github.io/commercetools-sync-java/v/3.0.0/) |

docs/usage/CART_DISCOUNT_SYNC.md

@@ -26,7 +26,7 @@ against a [CartDiscountDraft](https://docs.commercetools.com/http-api-projects-c
 
 #### Prerequisites
 1. Create a `sphereClient`:
-Use the [ClientConfigurationUtils](https://github.com/commercetools/commercetools-sync-java/blob/3.0.0/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L45) which apply the best practices for `SphereClient` creation.
+Use the [ClientConfigurationUtils](https://github.com/commercetools/commercetools-sync-java/blob/3.0.1/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L45) which apply the best practices for `SphereClient` creation.
 If you have custom requirements for the sphere client creation, have a look into the [Important Usage Tips](IMPORTANT_USAGE_TIPS.md).
 
 2. The sync expects a list of `CartDiscountDraft`s that have their `key` fields set to be matched with
@@ -37,7 +37,7 @@ fields set, otherwise they won't be matched.
 Types are matched by their `key`s. Therefore, in order for the sync to resolve the 
 actual ids of the type reference, the `key` of the `Type` has to be supplied.
 
-   - When syncing from a source commercetools project, you can use [`mapToCartDiscountDrafts`](https://commercetools.github.io/commercetools-sync-java/v/3.0.0/com/commercetools/sync/cartdiscounts/utils/CartDiscountReferenceResolutionUtils.html#mapToCartDiscountDrafts-java.util.List-)
+   - When syncing from a source commercetools project, you can use [`mapToCartDiscountDrafts`](https://commercetools.github.io/commercetools-sync-java/v/3.0.1/com/commercetools/sync/cartdiscounts/utils/CartDiscountReferenceResolutionUtils.html#mapToCartDiscountDrafts-java.util.List-)
     method that maps from a `CartDiscount` to `CartDiscountDraft` in order to make them ready for reference resolution by the sync:
 
     ````java

docs/usage/CATEGORY_SYNC.md

@@ -27,7 +27,7 @@ against a [CategoryDraft](https://docs.commercetools.com/http-api-projects-categ
 
 #### Prerequisites
 1. Create a `sphereClient`:
-Use the [ClientConfigurationUtils](https://github.com/commercetools/commercetools-sync-java/blob/3.0.0/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L45) which apply the best practices for `SphereClient` creation.
+Use the [ClientConfigurationUtils](https://github.com/commercetools/commercetools-sync-java/blob/3.0.1/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L45) which apply the best practices for `SphereClient` creation.
 If you have custom requirements for the sphere client creation, have a look into the [Important Usage Tips](IMPORTANT_USAGE_TIPS.md).
 
 2. The sync expects a list of `CategoryDraft`s that have their `key` fields set to be matched with
@@ -38,7 +38,7 @@ otherwise they won't be matched.
 These references are matched by their `key`s. Therefore, in order for the sync to resolve the 
 actual ids of the references, their `key`s has to be supplied.
 
-   - When syncing from a source commercetools project, you can use [`mapToCategoryDrafts`](https://commercetools.github.io/commercetools-sync-java/v/3.0.0/com/commercetools/sync/categories/utils/CategoryReferenceResolutionUtils.html#mapToCategoryDrafts-java.util.List-)
+   - When syncing from a source commercetools project, you can use [`mapToCategoryDrafts`](https://commercetools.github.io/commercetools-sync-java/v/3.0.1/com/commercetools/sync/categories/utils/CategoryReferenceResolutionUtils.html#mapToCategoryDrafts-java.util.List-)
      method that maps from a `Category` to `CategoryDraft` in order to make them ready for reference resolution by the sync:
      ````java     
      final List<CategoryDraft> categoryDrafts = CategoryReferenceResolutionUtils.mapToCategoryDrafts(categories);

docs/usage/CUSTOMER_SYNC.md

@@ -26,7 +26,7 @@ against a [CustomerDraft](https://docs.commercetools.com/api/projects/customers#
 
 #### Prerequisites
 1. Create a `sphereClient`:
-Use the [ClientConfigurationUtils](https://github.com/commercetools/commercetools-sync-java/blob/3.0.0/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L45) which apply the best practices for `SphereClient` creation.
+Use the [ClientConfigurationUtils](https://github.com/commercetools/commercetools-sync-java/blob/3.0.1/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L45) which apply the best practices for `SphereClient` creation.
 If you have custom requirements for the sphere client creation, have a look into the [Important Usage Tips](IMPORTANT_USAGE_TIPS.md).
 
 2. The sync expects a list of `CustomerDraft`s that have their `key` fields set to be matched with customers in the 
@@ -43,7 +43,7 @@ Any reference that is not expanded will have its id in place and not replaced by
 resources on the target commercetools project and the library will issue an update/create an API request without reference
 resolution.
 
-     - When syncing from a source commercetools project, you can use [`mapToCustomerDrafts`](https://commercetools.github.io/commercetools-sync-java/v/3.0.0/com/commercetools/sync/customers/utils/CustomerReferenceResolutionUtils.html#mapToCustomerDrafts-java.util.List-)
+     - When syncing from a source commercetools project, you can use [`mapToCustomerDrafts`](https://commercetools.github.io/commercetools-sync-java/v/3.0.1/com/commercetools/sync/customers/utils/CustomerReferenceResolutionUtils.html#mapToCustomerDrafts-java.util.List-)
     method that maps from a `Customer` to `CustomerDraft` to make them ready for reference resolution by the sync:
 
     ````java

docs/usage/CUSTOM_OBJECT_SYNC.md

@@ -24,7 +24,7 @@ against a [CustomObjectDraft](https://docs.commercetools.com/http-api-projects-c
 
 #### Prerequisites
 1. Create a `sphereClient`:
-Use the [ClientConfigurationUtils](https://github.com/commercetools/commercetools-sync-java/blob/3.0.0/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L45) which apply the best practices for `SphereClient` creation.
+Use the [ClientConfigurationUtils](https://github.com/commercetools/commercetools-sync-java/blob/3.0.1/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L45) which apply the best practices for `SphereClient` creation.
 If you have custom requirements for the sphere client creation, have a look into the [Important Usage Tips](IMPORTANT_USAGE_TIPS.md).
 
 2. The sync expects a list of `CustomObjectDraft`s that have their `key` and `container` fields set to be matched with

docs/usage/IMPORTANT_USAGE_TIPS.md

@@ -31,7 +31,7 @@ productSync.sync(batch1)
 By design, scaling the sync process should **not** be done by executing the batches themselves in parallel. However, it can be done either by:
 
  - Changing the number of [max parallel requests](https://github.com/commercetools/commercetools-sync-java/tree/master/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L116) within the `sphereClient` configuration. It defines how many requests the client can execute in parallel.
- - or changing the draft [batch size](https://commercetools.github.io/commercetools-sync-java/v/3.0.0/com/commercetools/sync/commons/BaseSyncOptionsBuilder.html#batchSize-int-). It defines how many drafts can one batch contain.
+ - or changing the draft [batch size](https://commercetools.github.io/commercetools-sync-java/v/3.0.1/com/commercetools/sync/commons/BaseSyncOptionsBuilder.html#batchSize-int-). It defines how many drafts can one batch contain.
 
 The current overridable default [configuration](https://github.com/commercetools/commercetools-sync-java/tree/master/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L45) of the `sphereClient` 
 is the recommended good balance for stability and performance for the sync process.

docs/usage/INVENTORY_SYNC.md

@@ -27,7 +27,7 @@ against a [InventoryEntryDraft](https://docs.commercetools.com/http-api-projects
 
 #### Prerequisites
 1. Create a `sphereClient`:
-Use the [ClientConfigurationUtils](https://github.com/commercetools/commercetools-sync-java/blob/3.0.0/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L45) which apply the best practices for `SphereClient` creation.
+Use the [ClientConfigurationUtils](https://github.com/commercetools/commercetools-sync-java/blob/3.0.1/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L45) which apply the best practices for `SphereClient` creation.
 If you have custom requirements for the sphere client creation, have a look into the [Important Usage Tips](IMPORTANT_USAGE_TIPS.md).
 
 2. The sync expects a list of `InventoryEntryDraft`s that have their `sku` fields set,
@@ -37,7 +37,7 @@ If you have custom requirements for the sphere client creation, have a look into
    references are matched by their `key`s. Therefore, in order for the sync to resolve the actual ids of those references,
    their `key`s has to be supplied.
 
-   - When syncing from a source commercetools project, you can use [`mapToInventoryEntryDrafts`](https://commercetools.github.io/commercetools-sync-java/v/3.0.0/com/commercetools/sync/inventories/utils/InventoryReferenceResolutionUtils.html#mapToInventoryEntryDrafts-java.util.List-)
+   - When syncing from a source commercetools project, you can use [`mapToInventoryEntryDrafts`](https://commercetools.github.io/commercetools-sync-java/v/3.0.1/com/commercetools/sync/inventories/utils/InventoryReferenceResolutionUtils.html#mapToInventoryEntryDrafts-java.util.List-)
      method that that maps from a `InventoryEntry` to `InventoryEntryDraft` in order to make them ready for reference resolution by the sync:
      ````java
      final List<InventoryEntryDraft> inventoryEntryDrafts = InventoryReferenceResolutionUtils.mapToInventoryEntryDrafts(inventoryEntries);

docs/usage/PRODUCT_SYNC.md

@@ -27,7 +27,7 @@ against a [ProductDraft](https://docs.commercetools.com/http-api-projects-produc
 
 #### Prerequisites
 1. Create a `sphereClient`:
-Use the [ClientConfigurationUtils](https://github.com/commercetools/commercetools-sync-java/blob/3.0.0/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L45) which apply the best practices for `SphereClient` creation.
+Use the [ClientConfigurationUtils](https://github.com/commercetools/commercetools-sync-java/blob/3.0.1/src/main/java/com/commercetools/sync/commons/utils/ClientConfigurationUtils.java#L45) which apply the best practices for `SphereClient` creation.
 If you have custom requirements for the sphere client creation, have a look into the [Important Usage Tips](IMPORTANT_USAGE_TIPS.md).
 
 2. The sync expects a list of `ProductDraft`s that have their `key` fields set to be matched with
@@ -41,7 +41,7 @@ all the variants in the target project are expected to have the `sku` fields set
 of the product also have prices, where each price also has some references including a reference to the `Type` of its 
 custom fields and a reference to a `channel`. All these referenced resources are matched by their `key`s. Therefore, in 
 order for the sync to resolve the actual ids of those references, those `key`s have to be supplied in the following way:
-    - When syncing from a source commercetools project, you can use [`mapToProductDrafts`](https://commercetools.github.io/commercetools-sync-java/v/3.0.0/com/commercetools/sync/products/utils/ProductReferenceResolutionUtils.html#mapToProductDrafts-java.util.List-)
+    - When syncing from a source commercetools project, you can use [`mapToProductDrafts`](https://commercetools.github.io/commercetools-sync-java/v/3.0.1/com/commercetools/sync/products/utils/ProductReferenceResolutionUtils.html#mapToProductDrafts-java.util.List-)
      method that maps from a `Product` to `ProductDraft` in order to make them ready for reference resolution by the sync:
      ````java
      final List<ProductDraft> productDrafts = ProductReferenceResolutionUtils.mapToProductDrafts(products);