avpkg - Available Packages

[ghost]

Hi. I've wrote a little utility for displaying the available packages for fpm. This is how it works: 1) downloads the registry .toml in the system TEMP directory, 2) displays the package list in format ready to be copy pasted into your package .toml file. I hope you find it useful.

Repository

https://github.com/brocolis/avpkg

Dependencies

https://github.com/urbanjost/M_strings.git
https://github.com/interkosmos/fortran-curl

Thanks fpm, urbanjost & interkosmos.

[awvwgk]

You can also use toml-f to read the registry file and write TOML output.

[ghost]

Hi awvwgk . Good idea. I'll take a look at toml-f.

v0.2.0

avpkg can now add dependency to fpm.toml file, thanks to LKedward 's fhash.

Here is the screenshot.

[LKedward]

This is another really nice example of what can be achieved with fpm with only a few lines of code and exploiting existing packages, nice work @brocolis! I think eventually we'd like similar functionality to be incorporated into fpm (#201,#4).

---

If you want a bigger challenge, you could include package descriptions and maybe a search functionality using the fpm-registry index file, though this requires parsing json (see jsonff).

[urbanjost]

Beautiful. Not only is this a nice utility by itself but I was just sketching out how I wanted to change my own preprocessor to allow the $INCLUDE directive to allow accessing files via libcurl and adding a new $USE directive that would work to automatically add an external fpm dependency and you basically laid out almost all of it. I was picturing something in fpm that would let you search for a routine by keywords and then read it's document and then if it fit the bill add the package as a dependency to a project via a utility an this already does several of the steps.

[ivan-pi]

Great project @brocolis!

As suggested previously, if it could display descriptions of packages that would be really helpful.

[ghost]

Hi. Thank you all for the feedback. I'm working on the search functionality by using regular expressions (binding to libpcre). I'm however having trouble implementing proper parsing with jsonff and toml-f.

[everythingfunctional]

@brocolis , if there is something specifically you need help with in jsonff, feel free to open an issue in the repository. FYI, I'm currently in the middle of a major update to it that will make some (relatively minor?) modifications to the API.

[urbanjost]

I think addressing the issues users noted as stopping them from using fpm, along with an easy-to-use registry that allows you to search the registry, and to make your packages visible for searching via the fpm command line the two biggest items for fpm. Perhaps a registry system would be a suggestion for the "Google Summer of Coding". I don't think anyone has been collecting them, but I think most people were stopped by linking and command line options, especially go parallel programming. I think we need to pay particular attention to what stops anyone from adaptation in an organized manner.

[urbanjost]

Also, does anyone have any ideas on contacting users of popular repositories and soliciting or helping them make their packages available via fpm? I think we could learn a lot by seeing how difficult the transitions are. And between the interest in adding more computation routines to stdlib and the fact that just about every computation language has support of netlib routines or their derivititves would a "fpm-netlib" joint project have any legs?

[awvwgk]

I added fpm support to some projects already, wavebitscientific/functional-fortran#18 and interkosmos/fortran-curl#1, and will probably continue as need arises. I suggest to just contact the authors of the project you are interested in and offer them to contribute fpm support. There are some candidates suggested in #17 as well.

[ghost]

v0.4.0

New version of avpkg can search for packages descriptions. It works on Windows and thanks to @LKedward it also works on Linux.

Examples

avpkg search lapack
avpkg search "thermodynamics|mechanics"
avpkg info datetime
avpkg search numeric

Repository

https://github.com/brocolis/avpkg

[ghost]

v0.5.0

New command avpkg add package [tag] will print the package's data in toml format.

Examples:

[ghost]

v0.8.0

New command options

--registry URI

# Use HTTPS by default if the URL is missing a scheme name
fpm-search math --registry fortran.rocks/my.json

# Use any protocol supported by libcurl
fpm-search string --registry https://my-fpm-registry/index.json
fpm-search string --registry file://localhost/my-fpm-registry/index.json

--force-download

fpm-search currently downloads the registry database (index.json) from remote only if the local registry timestamp is older than 1 hour. Use --force-download if you want to force the download.

# force-downloads index.json from the official fpm-registry.
fpm-search --force-download

# force-downloads index.json from an unofficial registry
fpm-search --force-download --registry file://localhost/my-fpm-registry/index.json

New Fortran dependencies

• M_strings
• stdlib-fpm

[ghost]

New in v0.9.0

• libjson-c C dependency removed 👍. fpm-search now works with json-fortran by @jacobwilliams.
• displays the packages' dependencies and dev-dependencies.
• groups available versions in a single entry (no more duplicates).

Known issues

• minor issue with the --toml mode that occurs only if the user requests an specific git-tag.
• for some unknown reason the program displays truncated strings on my linux VM. It works (by accident?) on Windows.

Repository

https://github.com/brocolis/fpm-search

[ghost]

• for some unknown reason the program displays truncated strings on my linux VM. It works (by accident?) on Windows.

My Windows machine was using a cached, outdated version of M_strings. I can now reproduce the bug and will work on a fix.

[ghost]

It should work now, if we start from scratch

git clone https://github.com/brocolis/fpm-search
cd fpm-search
fpm build
fpm install

[urbanjost]

Very nice. Much more convenient from a CLI environment than bringing up the browser. I think the quick searching will become even more important as the number and size of repositories grows. I have been using it to get the dependency strings for my own repositories!

I find myself doing a verbose listing and then a TOML listing. I like to stick some of the metadata description into the fpm.toml file as comments. Maybe a verbose mode for --toml mode too where the metadata is preceded by a "# " would be nice?

Was the bug from a new version of M_strings? Anything need fixed there?

[ghost]

I find myself doing a verbose listing and then a TOML listing. I like to stick some of the metadata description into the fpm.toml file as comments. Maybe a verbose mode for --toml mode too where the metadata is preceded by a "# " would be nice?

I think this is a good idea.

Was the bug from a new version of M_strings? Anything need fixed there?

I haven't checked it properly yet. I will do that in weekend.

[urbanjost]

I added a personal repository

fpm-search --registry https://github.com/urbanjost/fpm-registry/raw/master/index.json

for testing and find myself wishing a config file or environment variable (would probably have to use ";" as a separator as ":" appears in so many curl names) could be used to search multiple registries. Most repo tools allow you to add and delete repo names from your default and to pull the packages as well as list them. Any thoughts along those lines? Duplicate names in particular could be an issue.
At some point a central "registry of registries" will hopefully be needed as well. Just some thoughts, but this is bound to grow. This seems so integral now to fpm(1) it should have a mention on the main fpm site IMO. If fpm is reorganized soon into an fpm package a snapshot of this should be distributed directly with fpm; albeit it probably needs to be in a non-default directory because of the non-Fortran dependencies or something like that.

[ghost]

Very nice registry @urbanjost, thanks! Yes, I agree with you that we need to create a configuration file to store the URLs and other information (name, time-to-live). We can add commands --add-registry URL and --time-to-live 1h. We can also remove the time to live option so that all registries live forever by default. The user should then run the --download-force command to update his local copies.

Regarding non-Fortran dependencies, we can replace libpcre with M_matches, and replace libcurl with shell command curl or wget. We can also statically link libcurl.