Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
CDF-20156: fixup fatjar pom file (#858)
Currently https://repo1.maven.org/maven2/com/cognite/spark/datasource/cdf-spark-datasource-fatjar_2.13/3.2.1044/cdf-spark-datasource-fatjar_2.13-3.2.1044.pom includes ```xml <dependencies> <dependency> <groupId>org.scala-lang</groupId> <artifactId>scala-library</artifactId> <version>2.13.8</version> </dependency> <dependency> <groupId>com.cognite.spark.datasource</groupId> <artifactId>cdf-spark-datasource_2.13</artifactId> <version>3.2.1044</version> </dependency> <dependency> <groupId>io.scalaland</groupId> <artifactId>chimney_2.13</artifactId> <version>0.5.3</version> </dependency> </dependencies> ``` Which isn't expected, fatjar should have (almost) no dependencies. The jar itself though contains all the classes and applies shading as expected. It is then possible that pulling fatjar from maven central would pull duplicated classes and if non-fatjar unshaded versions are picked up or a mix with fatjar ones, the datasource would runtime-crash when used for example in Spark in DataBricks. Let's remove chimney from commonSettings and move to per-target dependencies. And for main library, looks like the simplest way to both have it as part of fatJar and not have it in pom is to patch generated pom. As for scala-library keeping it as-is, older fatjar setup also had it, so not investigating for now. Verified pom contents by running `fatJarShaded/publishLocal` and inspecting pom from filesystem. And used `fatJarShaded/assembly` to generate fatjar and inspect with `jar tf` to check that is has our classes and applies shading rules. Did not yet verify in databricks, but even if this PR isn't enough to fix running there it is still a step in the right directiion. [CDF-20156]
- Loading branch information