Skip to content

Adding support to the 64bit version of xxhash #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 27 commits into
base: develop
Choose a base branch
from

Conversation

nheimann1
Copy link

I needed the 64 bit version, I assumed other people will need it too. The rest of the details are in the README.md file.

General note:
Since PHP doesn't support unsigned long values, the xxhash64() function can return negative values (this is the same case with the JAVA library). In case you are working in multiple platforms you can use the xxhash64Unsigned() version I added, or casting the results in other platforms from unsigned long to long

@arcataroger
Copy link

Thank you for adding this! Was looking for a 64-bit implementation.

@GeThi255
Copy link

GeThi255 commented Nov 5, 2016

I'd like to have a php function call for createing a xxhash from a file not from a string. To handle with strings is very poor for very large data. Like md5_file ().

Thanks a lot.

GeThi255

@arcataroger
Copy link

Can you do xxhash(file_get_contents('file'))? At work we do this with PDFs up to 20 MB or so and it works fine.

@GeThi255
Copy link

GeThi255 commented Nov 5, 2016

Thank you very much for the solution. But this kills the performance.

1st try:
$xxhash = xxhash32(file_get_contents($file));
1.32 second for 1.4 GB

2nd try:
$xxhash = xxhash64(file_get_contents($file));
1.08 second for 1.4 GB (same file)

3rd try:
$xxhash = exec ('/usr/local/bin/xxh64sum ' . escapeshellarg($file) . ' 2> /dev/null', $RetVal, $RetVar);
0.34 second for 1.4 GB (same file)

My largest file has 143 GB. It's a VM image.

@GeThi255
Copy link

GeThi255 commented Nov 5, 2016

And the max file size is limited by the RAM.

Tried with 17 GB file:
PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted

jmwebservices and others added 11 commits February 15, 2018 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants