Metafacture Core Distribution 5.4.0

Changes

New Features

• HTTP: Enable other methods to HttpOpener than just GET - i.e. DELETE, HEAD, OPTIONS, POST, PUT and TRACE #463
• HTTP: Enable generic header support in HttpOpener #456
• File: Add support for compressed files as FileMap input #455
• JSON: Implement JSON booleans and numbers #458

Other

• Update OAI-PMH harvester library #360
• "./gradlew fluxCommands" to see the currently available options (743e90e)

... and various smaller fixes and improvements (e.g. Mark deprecated Flux command options in help output #451)

Metafacture Core Distribution 5.3.2

Changes

Bug fixes

• Strings: Call LineRecorder receiver's closeStream() #433
• Flux: Throw ANTLR errors in FluxParser #421
• XML/biblio: Do not trim control field values in MARCXML handler #440

Other

• Use protected on all on...() methods #435

Metafacture Core Distribution 5.3.2 release candidate

Changes

Bug fixes

• Strings: Call LineRecorder receiver's closeStream() #433
• Flux: Throw ANTLR errors in FluxParser #421
• XML/biblio: Do not trim control field values in MARCXML handler #440

Other

• Use protected on all on...() methods #435

Metafacture Core Distribution 5.3.1

Changes

Reverted

• The dependency upgrade of log4j:1.2.12 to log4j-core:2.14.1 can be problematic, so this is reverted in 36ed969 and the dependency is only minor updated to log4j:1.2.17.
• an API break concerning the accessibility of FilenameUtilin FilenameExtractor is reverted in 7c1ea04.

Possible caveats

Using a metamorph.xsd of your own and making use of FileMap you have to also update your locally metamorph.xsd like:

-      <attribute name="separator" type="string" use="optional" default="\t">
+      <attribute name="separator" type="string" use="optional" default="&#09;">

This will occur only quite rarely. This is the result of a bug fix (d528ac9) (in effect the default separator defined (falsely) in metamorph.xsd has had no effect at all).

Metafacture Core Distribution 5.3.0

Consider this release as deprecated .

The update of the dependency of log4j:log4j:1.2.12 to org.apache.logging.log4j:log4j-core:2.14.1 will not play along if you use metafacture as a library along with log4j-slf4j-impl in an other version than org.apache.logging.log4j:log4j-slf4j-impl:2.14.1 and results in:

[...] java.lang.NoClassDefFoundError: Could not initialize class org.metafacture.metamorph.Metamorph
Caused by:
java.lang.ClassNotFoundException: org.apache.logging.log4j.util.ReflectionUtil [...]

Use the upcoming Metafacture Core Distribution 5.3.1 to be on the safe side.
(The update of the log4j dependency will be part of the upcoming Metafacture Core Distribution 6.0.0)

Changes

Bug fixes

• XML/biblio: Fix creation of Marc XML namespaces #403
• XML/biblio: Fix Namespace-prefixes of elements and attributes #377
• XML/biblio: Marc-XML-encoder: record-type written as controlfield not as attribut of record-field #402
• XML/biblio: Improve handling of XML attributes and element values #394
• XML/biblio: Encode top-level MARC record leader as proper XML element instead of control field. #336
• XML/biblio: Make simple XML encoder value tag name configurable #379
• JSON: Fix _elseNested loses array-key in JSON #374
• Metamorph: Fix _elseNested only outputs two hierachy levels #378
• Metamorph: Fix "setreplace" using a FileMap #381
• Metamorph: Guarantee that tests should verify that no unexpected interactions occurred #339

New Features

• JSON: Make JSON encoder array marker configurable #393
• JSON: Add or enhance a function to extract JSON-Records from an JSON-API #382
• Mangling: Split up event stream into records #385
• Metamorph: Allow empty values in setreplace map #420
• Triples: Sort triples numerically #380
• YAML: Add YAML Encoder/Decoder #399

Other

• Update release and publish process #311
• Checkstyle and javadoc #389 #396
• Update and apply EditorConfig file #388
• Add initial CONTRIBUTING.md #382
• Fix insecure logging configuration #364

... and various smaller fixes and improvements (e.g. #417)

Metafacture Core Distribution 5.2.0

Changes

Bug fixes

• Fix flux.bat for running on Windows #315
• OAI-Pmh fails with SAXParseException #334
• Ignore null values in Regexp function. #337
• Metamorph tests verify that no unexpected interactions occurred (first batch). #341
• Escape feedback char in _else data. #344
• Don't handle empty '[]' as character class #348
• _elseNested: Entity with more than one subfield results in multiple entities #338

New Features

• Add HTML input support #312
• Allow to omit id in JsonToElasticsearchBulk #323
• Add original OAI-PMH opener #320
• Support flux options via setters in JsonToElasticsearchBulk #326
• Add constructor to allow Entity creation w/o Metamorph #327
• Make MARCXML namespace for record elements configurable. #331
• Allow flushing collectors to only emit when complete #332
• Pass data through metamorph else #333
• ADD contains and not-contains to metamorph; CHG extent filter to work… #317
• Allow comments in JSON files. #345
• Add option to set record IDs in ObjectToLiteral #354
• Optionally decompress concatenated streams. #358
• Add filter-null-values Flux command for NullFilter #362
• Support HTML attribute values as subfields #361

Other

• Tweak help output for Markdown formatting #316
• Migrate from travis to github actions to test build #352

Metafacture Core Distribution 5.1.0

Changes

Bug fixes

• Prevent possible hanging when using ObjectPipeDecoupler (see 6ece930)

New Features

• Fix hbz/lobid-resources#967: Added ObjectThreader which allows to use multithreading (see eb1cb22); also provided as a flux command (see a432554)
• Fix #296: handle not-normalized Pica+ by improving PicaDecoder (see 25b125f)
• Fix #300: provided MarcXmlEncoder (see 78aaca3)

Other

• Added an example of how to use XPointer to address only portions of inclusions of another Metamorph (thanks @cboehme, see ccdad61).
• Added an example how to convert Pica+ to Marc-Xml (see 3407f14)
• Fix #308 : Added a flux command encode-xml to reference the SimpleXmlEncoder. The original flux command was stream-to-xml and is kept for being backwards compatible

Metafacture Core Distribution 5.0.1

metafacture-core-5.0.1

Publish first bug-fix release of Metafacture 5

Metafacture Core Distribution 5.0.0

This is the first release of the Metafacture 5 line. With Metafacture 5 the migration from a monolithic library to smaller domain-specific libraries is completed.

Important: This release is published with new Maven coordinates and uses a new package root. As Metafacture has been split-up into domain-specific libraries there is no longer a single Maven dependency. Instead there is one for each domain-specific library. The readme explains how to find the Maven coordinates of these dependencies. The root package has been changed from org.culturegraph.mf to org.metafacture.

Additionally, metafacture-runner has been merged back into the metafacture-core repository. This makes it easier to keep it up-to-date with new releases of metafacture-core.

Updating to Metafacture 5

1. If you are still using metafacture-core 3.5.0 or older you should first update to metafacture-core 4.0.0. In this release many classes were relocated. By updating first to metafacture-core 4.0.0 you avoid having to handle the relocation and the split-up of the metafacture-core library at the same time.
2. Search for org.culturegraph.mf in your project and replace it with org.metafacture. This should mostly affect import statements.
3. Remove the Maven depedency on metafacture-core and follow the explanation in the readme to add the new domain-specific library dependencies.

Changes

Breaking changes

• Split up the monolithic library into smaller domain-specific libraries and changed the Maven group id from org.culturegraph to org.metafacture. The readme explains how to find the Maven dependencies (see 2234874)
• Changed package root from org.culturegraph.mf to org.metafacture (see 2234874)
• Merge metafacture-runner (see 7d07e91)

Bug fixes

• Fixed #267: XmlUtil.escape does not handle Unicode corretly for codepoints above U+10000 (see 95ff1ab)
• Changed JsonEncoder to not prefix pretty-print with spaces (thanks @blackwinter, see de8e7b3)
• Included metafacture-files in Metafacture Distribution (see a6dc848)

New Features

• Added decoder for JSON (thanks @blackwinter, see 8fe600e, 60b36aa)
• Added handler for CO-MARC (thanks @larsgsvensson, see fd3dd48)
• Added option to XmlEncoder for setting encoding and version attributes in the xml header (thanks @eberhardtj, see 087e20b)

Build infrastructure

• Migrated from Maven to Gradle (see 0f7a3b2)
• Automated the release process. A release is now triggered by pushing an annotated tag to the Github repository. The release is automatically published on Maven Central and the distribution files are uploaded to Github (see 04276e4, b544371, 52ab09b)
• Version numbers are generated from SCM information (see bc4a3d3)
• Added Sonarqube analysis to the CI build process (see 8a1f500, 91a4448)

Metafacture Core Distribution 5.0.0-rc2

This is the second and last release candidate for Metafacture 5. With Metafacture 5 the migration from a monolithic library to smaller domain-specific libraries is completed.

Changes

Bug fixes

• Changed JsonEncoder to not prefix pretty-print with spaces (thanks @blackwinter, see de8e7b3)
• Included metafacture-files in Metafacture Distribution (see a6dc848)