Records harvesting from koha to vufind

[Londheamar]

Hello,

We've successfully installed the latest version of VuFind and established connections with two Koha instances using the ILSDI method. Currently, we are attempting to import records from Koha to VuFind using the OAI method.

Upon executing the command php $VUFIND_HOME/harvest/harvest_oai.php the import process initiates and retrieves records from Koha. However, it appears that all records, including deleted ones, are being imported. Koha stores deleted records in the deletedbiblio_metadata and deletedbiblio tables. Consequently, VuFind is importing these deleted records as well. Is there a way to prevent this during the import process?

OAI configuration

../vufind/local/harvest/oai.ini

[koha-instance-name]
url = http://opac.domain.com/cgi-bin/koha/oai.pl
metadataPrefix = oai_dc
idSearch[] = "/oai:opac.domain.com:/"
idReplace[] = "koha-instance-name-"
idSearch[] = "/\//"
idReplace[] = "-"
injectDate = "datestamp"
injectId = "identifier"
dateGranularity = auto
harvestedIdLog = harvest.log

[demiankatz]

Londhe, I think this is actually more of a Koha question than a VuFind one. VuFind seems to be working as expected, but the question is why Koha's OAI server is behaving the way it does. I don't have any personal Koha experience, so I don't know the answer. It's possible that other Koha/VuFind users here will have an idea, but if you don't get a helpful answer, I'd suggest reaching out to the Koha community for assistance. If they can tell you how to configure Koha to omit deleted records, I think your problem will be resolved. - Demian Sent from my Verizon, Samsung Galaxy smartphone Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Londhe Amar ***@***.***> Sent: Friday, March 8, 2024 2:57:30 AM To: vufind-org/vufind ***@***.***> Cc: Subscribed ***@***.***> Subject: [EXTERNAL] [vufind-org/vufind] Records harvesting from koha to vufind (Discussion #3492) Hello, We've successfully installed the latest version of VuFind and established connections with two Koha instances using the ILSDI method. Currently, we are attempting to import records from Koha to VuFind using the OAI method. Upon executing the command php $VUFIND_HOME/harvest/harvest_oai.php the import process initiates and retrieves records from Koha. However, it appears that all records, including deleted ones, are being imported. Koha stores deleted records in the deletedbiblio_metadata and deletedbiblio tables. Consequently, VuFind is importing these deleted records as well. Is there a way to prevent this during the import process? OAI configuration ../vufind/local/harvest/oai.ini [koha-instance-name] url = http://opac.domain.com/cgi-bin/koha/oai.pl metadataPrefix = oai_dc idSearch[] = "/oai:opac.domain.com:/" idReplace[] = "koha-instance-name-" idSearch[] = "/\//" idReplace[] = "-" injectDate = "datestamp" injectId = "identifier" dateGranularity = auto harvestedIdLog = harvest.log — Reply to this email directly, view it on GitHub<#3492>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AACLOTPZTPZXD652FNEMQYTYXFVOVAVCNFSM6AAAAABEMMAXMWVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZWGM2DAMRSGM>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[EreMaijala]

If you're actually getting deleted records as non-deleted, there's a problem in Koha, like Demian said. However, It is normal behavior for OAI to include headers for deleted records so that the harvester knows to delete any records that were harvested normally sometimes earlier. And due to how Koha's database is structured, it will first return headers for all deleted records, then proceed to the non-deleted records.

[Londheamar]

Hello,
Thanks for your response. I will try to take up this issue with Koha community as you suggested.

[Londheamar]

The issue is related to Koha's OAI-PMH, where Koha is sending deleted records as well.

Thank you for your suggestions.

[EreMaijala]

It is normal, and expected, for Koha to return headers for deleted records so that the harvester knows to mark them deleted, but I believe that if you receive full records, Koha doesn't think the records are actually deleted. How do you find out that the records are actually deleted if they come in as normal ones?