Can't open FLAC file with read on macOS (get unimplemented format error)

[ArnoXf]

Hi!

I got an issue when trying to open a FLAC file using the following simple code:

>>> import soundfile as sf
>>> data, sr = sf.read('test.flac')

That results in the following traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda3/envs/mqa/lib/python3.6/site-packages/soundfile.py", line 373, in read
    subtype, endian, format, closefd) as f:
  File "/usr/local/anaconda3/envs/mqa/lib/python3.6/site-packages/soundfile.py", line 740, in __init__
    self._file = self._open(file, mode_int, closefd)
  File "/usr/local/anaconda3/envs/mqa/lib/python3.6/site-packages/soundfile.py", line 1265, in _open
    "Error opening {0!r}: ".format(self.name))
  File "/usr/local/anaconda3/envs/mqa/lib/python3.6/site-packages/soundfile.py", line 1455, in _error_check
    raise RuntimeError(prefix + _ffi.string(err_str).decode('utf-8', 'replace'))
RuntimeError: Error opening '/Users/afuhrmann/Desktop/test.flac': File contains data in an unimplemented format.

The available formats don't show FLAC:

>>> sf.available_formats()
{'AIFF': 'AIFF (Apple/SGI)', 'AU': 'AU (Sun/NeXT)', 'AVR': 'AVR (Audio Visual Research)', 'CAF': 'CAF (Apple Core Audio File)', 'HTK': 'HTK (HMM Tool Kit)', 'SVX': 'IFF (Amiga IFF/SVX8/SV16)', 'MAT4': 'MAT4 (GNU Octave 2.0 / Matlab 4.2)', 'MAT5': 'MAT5 (GNU Octave 2.1 / Matlab 5.0)', 'MPC2K': 'MPC (Akai MPC 2k)', 'PAF': 'PAF (Ensoniq PARIS)', 'PVF': 'PVF (Portable Voice Format)', 'RAW': 'RAW (header-less)', 'RF64': 'RF64 (RIFF 64)', 'SD2': 'SD2 (Sound Designer II)', 'SDS': 'SDS (Midi Sample Dump Standard)', 'IRCAM': 'SF (Berkeley/IRCAM/CARL)', 'VOC': 'VOC (Creative Labs)', 'W64': 'W64 (SoundFoundry WAVE 64)', 'WAV': 'WAV (Microsoft)', 'NIST': 'WAV (NIST Sphere)', 'WAVEX': 'WAVEX (Microsoft)', 'WVE': 'WVE (Psion Series 3)', 'XI': 'XI (FastTracker 2)'}

but the subformats with format flac does:

>>> sf.available_subtypes(format='FLAC')
{'PCM_S8': 'Signed 8 bit PCM', 'PCM_16': 'Signed 16 bit PCM', 'PCM_24': 'Signed 24 bit PCM'}

Also FLAC is in the supported formats in both soundfile.py and sndfile.h

    SF_FORMAT_FLAC          = 0x170000,     /* FLAC lossless file format */

Also, FLAC files can be played in macOS.

Does anyone know how to resolve that issue?

Thanks in advance

[bastibe]

How did you install soundfile? My first guess would be that you did not install the wheel version (which comes with a full-featured libsndfile), but used some other installation method instead that does not come with its own libsndfile, thus using your system's libsndfile, which is lacking FLAC support.

[asapsmc]

I am getting the same error. I did a conda install librosa, which took care of libsndfile dependencies
But I'm using libsndfile v1.0.28 (in the current conda environment), which is supposed to support FLAC.
@bastibe how do I correct this?

[asapsmc]

I'm working in a conda environment with:

• pysoundfile 0.9.0.post1
• libsndfile 1.0.28

[bastibe]

The problem in your case is probably that you're using pysoundfile, instead of soundfile. We deprecated the use of pysoundfile a while ago, and the version is no longer maintained.

[asapsmc]

That was a problem, but I uninstalled pysoundfile, and now installed soundfile (v0.10.3).
But I'm getting the same error: "RuntimeError: Error opening '/Users/.../audio/hains006.flac': File contains data in an unimplemented format.
Also, when I do soundfile.available_formats(), FLAC doesn't appear.

[bastibe]

You'll have to check which version of libsndfile is actually used by soundfile. It might still be using the one from conda, which would explain why the newer version of soundfile (which comes with a fully-configured libsndfile) didn't fix the problem.

Since you seem to be on macOS, you can also try installing libsndfile through homebrew. If I remember correctly, homebrew has options for enabling FLAC support.

If all else fails, you can use LD_PRELOAD to force loading a specific version of libsndfile and see if that fixes the issue. And then work backwards from there to figure out why soundfile is not loading the version you prefer.

[mgeier]

@MR-T77 You can try this to find out which libsndfile is used on your system:

import soundfile as sf
print(sf._libname)

[asapsmc]

Thanks for your help @bastibe and @mgeier. Unfortunately, I'm a little bit lost on what to do next...
Yesterday I had installed libsndfile through homebrew, but that didn't fix. (I tried also brew install libsndfile --with-flac but got "Error: invalid option: --with-flac" - on Macos Mojave - 10.14)
If I do print print(sf._libname) I get "/.../anaconda/envs/dynamite/bin/../lib/libsndfile.dylib" but I don't know how to interpret this info.
Any other path I may explore before LD_PRELOAD, which seems to require a higher level of expertise than the one I have.

[mgeier]

I get "/.../anaconda/envs/dynamite/bin/../lib/libsndfile.dylib" but I don't know how to interpret this info.

This tells you from which file on your system the libsndfile library is loaded.

The path you are getting points somewhere inside your Anaconda folder. It's not the one distributed with the soundfile wheel package.

There seem to be FLAC problems with the macOS version of the Anaconda package, see conda-forge/libsndfile-feedstock#2 and librosa/librosa#847 (comment). There doesn't seem to be a resolution.

You should try to uninstall libsndfile with conda, then it probably works.

[j9ac9k]

I just stumbled across this very issue, but the source of the conflicting libsndfile.dylib was from homebrew. libsndfile is a dependency of ffmpeg, portaudio, sox and other packages commonly distributed via the brew package manager on macOS.

Unfortunately, soundfile picks up the libsndfile.dylib file that homebrew distrubutes

$ python
Python 3.7.7 (default, Jun 19 2020, 13:21:43)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import soundfile as sf
>>> sf._libname
'/usr/local/lib/libsndfile.dylib'
>>> import ctypes
>>> ctypes.util.find_library("sndfile")
'/usr/local/lib/libsndfile.dylib'

After uninstalling libsndfile from homebrew via brew uninstall --ignore-dependencies libsndfile soundfile works as expected

$ python
Python 3.7.7 (default, Jun 19 2020, 13:21:43)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import soundfile as sf
>>> sf._libname
'libsndfile.dylib'

Unfortunately, tools like ffmpeg, sox, etc I use routinely, so removing the homebrew bundled dependency for it is rather problematic. Would there be a workaround to specify which libsndfile.dylib file should be used?

[bastibe]

You could probably LD_PRELOAD it. (I think that works on macOS, right?)

[j9ac9k]

AFAIK LD_PRELOAD does not work on macOS/OSX, but I'm operating out of my depth here. After re-installing libsndfile via homebrew:

For the record, this is the location of my libsndfile.dylib library provided by soundfile: /Users/ogi/.zinit/plugins/pyenv---pyenv/versions/3.7.7/envs/fred/lib/python3.7/site-packages/_soundfile_data/libsndfile.dylib

$ python
Python 3.7.7 (default, Jun 19 2020, 13:21:43)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import soundfile as sf
>>> sf._libname
'/usr/local/lib/libsndfile.dylib'
>>> exit()

$ DYLD_INSERT_LIBRARIES="/Users/ogi/.zinit/plugins/pyenv---pyenv/versions/3.7.7/envs/fred/lib/python3.7/site-packages/_soundfile_data/libsndfile.dylib" python
Python 3.7.7 (default, Jun 19 2020, 13:21:43)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import soundfile as sf
>>> sf._libname
'/usr/local/lib/libsndfile.dylib'
>>> exit()

$ LD_PRELOAD="/Users/ogi/.zinit/plugins/pyenv---pyenv/versions/3.7.7/envs/fred/lib/python3.7/site-packages/_soundfile_data/libsndfile.dylib" python
Python 3.7.7 (default, Jun 19 2020, 13:21:43)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import soundfile as sf
>>> sf._libname
'/usr/local/lib/libsndfile.dylib'
>>>

Looking through the documentation for ctypes.util.find_library() I found this blurb

If wrapping a shared library with ctypes, it may be better to determine the shared library name at development time, and hardcode that into the wrapper module instead of using find_library() to locate the library at runtime.

[mgeier]

If wrapping a shared library with ctypes, it may be better to determine the shared library name at development time, and hardcode that into the wrapper module instead of using find_library() to locate the library at runtime.

The soundfile module actually does that, but only on the second attempt.

First, it tries to find the library with ctypes.util.find_library(). If that fails, it tries the hard-coded path in _soundfile_data.

The problem here is that the first attempt does not fail ...

As a work-around, you can monkey-patch the find_library() function to make it fail unconditionally:

def dummy_find_library(name):
    return None

import ctypes.util
ctypes.util.find_library = dummy_find_library

import soundfile as sf

...

[mgeier]

Another idea: Why not just overwrite /usr/local/lib/libsndfile.dylib?

I assume this is a symbolic link provided by brew, right?

You could just copy the libsndfile.dylib from the soundfile module there, and hope that all other programs like ffmpeg, portaudio, sox continue working.

[j9ac9k]

First, it tries to find the library with ctypes.util.find_library(). If that fails, it tries the hard-coded path in _soundfile_data.

After I posted my reply, I noticed that too! I don't have enough experience doing this sort of thing to make a recommendation on how it should be done so I leave that to the judgment of more experienced folks. I am curious though if it would be worthwhile to try loading the hard-coded path first and then fall back on find_library()?

Another idea: Why not just overwrite /usr/local/lib/libsndfile.dylib?

I think this is what I'll end up doing for the time being. I must admit I am curious why their version of libsndfile does not support flac files, but that's not a topic for this issue tracker.

Thanks for your input @mgeier !