Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/thicknesses_ #112

Closed
wants to merge 73 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
7d2d659
fix: squashed commits of: - fixing the non-use of minimum_fault_lengt…
AngRodrigues Jun 19, 2024
3a48005
tests: add more functionality
AngRodrigues Jun 19, 2024
8b3c348
fix: leave as is for now
AngRodrigues Jun 19, 2024
44bfa06
fix: finalise the thickness calculator outputs
AngRodrigues Jun 20, 2024
c60ed2c
fix: update tests/ small fixes
AngRodrigues Jul 8, 2024
77e7233
fix: make sure dependencies are right
AngRodrigues Jul 8, 2024
9f701d1
Merge branch 'master' of https://github.com/Loop3D/map2loop into fix/…
AngRodrigues Jul 8, 2024
f256ca8
fix: update thickness tests//fix minor typos
AngRodrigues Jul 8, 2024
2451679
fix: correct some typos/update tests
AngRodrigues Jul 8, 2024
31073dc
fix: typo in tests
AngRodrigues Jul 8, 2024
de35af5
Update README.md
AngRodrigues Jul 8, 2024
3f42692
fix: add the basal_contacts_abnormal_check
AngRodrigues Jul 8, 2024
5983ca2
fix: added check for collocated points
rabii-chaarani Aug 7, 2024
49202dd
Merge branch 'master' of https://github.com/Loop3D/map2loop into fix/…
AngRodrigues Aug 9, 2024
7a483ef
fix: upd gdal as per master
AngRodrigues Aug 9, 2024
6887ae3
fix: remove test from server
AngRodrigues Aug 9, 2024
31255ea
fix: issue 122
AngRodrigues Aug 8, 2024
d7b09e0
fix: update str format
AngRodrigues Aug 9, 2024
bdfd9aa
fix: now use DBSCAN to aggregate collocated poins
rabii-chaarani Aug 9, 2024
45c1798
fix: removed addition
rabii-chaarani Aug 9, 2024
8b406ea
fix: added warning when collocated points detected
rabii-chaarani Aug 9, 2024
543c122
Merge branch 'master' of https://github.com/Loop3D/map2loop into fix/…
AngRodrigues Aug 12, 2024
c10c2e3
fix: add str for paths - gdal does not accept pathlibs
AngRodrigues Aug 12, 2024
80a27f6
Merge branch 'master' of https://github.com/Loop3D/map2loop into fix/…
AngRodrigues Sep 25, 2024
e78ea7e
fix: update minimum_fault_length as per code review
AngRodrigues Sep 26, 2024
fb5efae
fix: add pathlibs where possible in m2model_wrapper
AngRodrigues Sep 26, 2024
8892241
fix: add pathlib in mapdata.py
AngRodrigues Sep 26, 2024
a9265b0
fix: merge coll points branch and remove linalg catch
AngRodrigues Sep 26, 2024
5a5e0d7
fix: revert pathlib in map2model.run
AngRodrigues Sep 26, 2024
6b95466
fix: proj should not have temp files
AngRodrigues Sep 26, 2024
23873fa
fix: update the minimum_fault_length
AngRodrigues Sep 26, 2024
e491b29
tests: add tests for minimum_fault_length
AngRodrigues Sep 26, 2024
94ea056
fix: remove redundancy from project.py
AngRodrigues Sep 26, 2024
0857f23
fix: update calculate minimum_fault_length for 5% of the map area
AngRodrigues Oct 1, 2024
d73b748
fix: linting
AngRodrigues Oct 1, 2024
dbc8495
fix: typo
AngRodrigues Oct 1, 2024
a5a29e4
fix: remove repetitive print stat
AngRodrigues Oct 22, 2024
f8e855c
fix: adjust thickness_calc workflow for modularity
AngRodrigues Oct 22, 2024
da9d5ad
fix: remove unnecessary fields
AngRodrigues Oct 22, 2024
4096603
fix: update thickness tests
AngRodrigues Oct 22, 2024
c041ff5
Merge branch 'master' of https://github.com/Loop3D/map2loop into fix/…
AngRodrigues Oct 22, 2024
58edbbf
fix: add ignore_fault_codes and tests for ignore_lithology_code and i…
AngRodrigues Oct 22, 2024
0c236b8
fix: try relative path ../_datasets
rabii-chaarani Oct 22, 2024
a797855
fix: use only path instead of pkg_resources
rabii-chaarani Oct 22, 2024
b2ea63f
fix: try full path
rabii-chaarani Oct 22, 2024
dd1234d
fix: load only using path /hamersley
rabii-chaarani Oct 22, 2024
ab64683
fix: try ./map2loop/_datasets/geodata_files/
rabii-chaarani Oct 22, 2024
bcf114b
fix: use correct path
rabii-chaarani Oct 22, 2024
ac61527
refactor: back to original code
rabii-chaarani Oct 23, 2024
d494296
fix: add the whole hamersley folder
AngRodrigues Oct 30, 2024
52c6724
fix: update docker for docs
AngRodrigues Oct 31, 2024
ff27e8b
fix: update the docs-deploy for master branch only
AngRodrigues Oct 31, 2024
ef9289e
fix: update CI to build on master only
AngRodrigues Oct 31, 2024
e412c17
fix: remove redundancy
AngRodrigues Oct 31, 2024
53e9e91
fix: update the server test and handle timeout properly
AngRodrigues Oct 31, 2024
084774c
fix: updated ReadMe to have the right install steps (temporary measure)
AngRodrigues Oct 31, 2024
9296a9c
fix: update CI - test wheels on master only
AngRodrigues Nov 1, 2024
6359c1c
fix: add test to sdist build in CI to ensure tests are run in the non…
AngRodrigues Nov 1, 2024
db70684
fix: update pytest in ci
AngRodrigues Nov 1, 2024
334822f
fix: revert ci
AngRodrigues Nov 1, 2024
2be26cc
fix: add pytest in the right place
AngRodrigues Nov 1, 2024
23a5092
fix: update pytest location
AngRodrigues Nov 1, 2024
900a73f
fix: manifest update
AngRodrigues Nov 1, 2024
da8d88a
fix: update ci again
AngRodrigues Nov 1, 2024
4b76f5a
investigate
AngRodrigues Nov 1, 2024
f825700
find the install folder
AngRodrigues Nov 1, 2024
2eb2630
try again
AngRodrigues Nov 1, 2024
074a3d7
update ci
AngRodrigues Nov 1, 2024
86ba08b
fix: add checkout step to build sdist
AngRodrigues Nov 1, 2024
cf21ab5
fix: run pytest with test sdist action
AngRodrigues Nov 1, 2024
ec24c84
fix: revert CI and let tests only run on main branch
AngRodrigues Nov 1, 2024
af63731
fix: add version check new class to check install of dependencies.tx…
AngRodrigues Nov 1, 2024
4aa4a3d
fix: organise manifest
AngRodrigues Nov 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .github/workflows/CD.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:

- name: Build SDist
run: |
pip install build
pip install build
python -m build

- uses: actions/upload-artifact@v4
Expand Down Expand Up @@ -112,6 +112,7 @@ jobs:
name: Test wheels on ${{ matrix.os }}
needs: pypi-build-wheels
runs-on: ${{ matrix.os }}
if : github.ref == 'refs/heads/master'
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -162,6 +163,7 @@ jobs:
conda-build:
name: Conda Build and Deploy ${{ matrix.os }} - Python Version-${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
if : github.ref == 'refs/heads/master'
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -225,6 +227,7 @@ jobs:

documentation-test:
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/master'
steps:
- uses: actions/checkout@v4
- run: |
Expand All @@ -240,7 +243,8 @@ jobs:
documentation-deploy:
runs-on: ubuntu-latest
needs: [release-please, documentation-test]
if: ${{ needs.release-please.outputs.release_created }}
if: ${{ needs.release-please.outputs.release_created }} && github.ref == 'refs/heads/master'

steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
Expand Down
8 changes: 7 additions & 1 deletion MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,2 +1,8 @@
include LICENSE
include README.md

include map2loop/_datasets/clut_files/*.csv
include map2loop/_datasets/config_files/*.json
include map2loop/_datasets/config_files/*.json
include map2loop/_datasets/geodata_files/hamersley/*

recursive-include tests *.py
14 changes: 11 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Map2Loop 3.0
# Map2Loop 3.1

Generate 3D geological model inputs from geological maps — a high-level implementation and extension of the original map2loop code developed by Prof. Mark Jessell at UWA. To see an example interactive model built with map2loop and LoopStructural, follow this link:

Expand All @@ -23,12 +23,20 @@ conda config --add channels conda-forge

### Run

To just use map2loop, issue the following
To just use map2loop, issue the following. <sup>*</sup>

```bash
conda install -c conda-forge -c loop3d map2loop -y
git clone https://github.com/Loop3D/map2loop.git

cd map2loop

conda install -c loop3d --file dependencies.txt

pip install .
```

<p><sup>*</sup> We're actively working towards a better approach - stay tuned! </p>

### Documentation

If you can call it that, is available <a href="https://loop3d.org/map2loop/">here</a>
Expand Down
3 changes: 2 additions & 1 deletion dependencies.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ shapely
networkx
owslib
map2model
loopprojectfile==0.1.3
hjson
loopprojectfile==0.1.4
beartype
gdal==3.8.4
hjson
Expand Down
4 changes: 2 additions & 2 deletions docs/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ RUN apt-get update -qq && \
libgl1\
libtinfo5\
libtiff6\
libgl1-mesa-glx

libgl1-mesa-glx\
libarchive13

COPY . /map2loop

Expand Down
2 changes: 0 additions & 2 deletions docs/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
version: "3"

services:
map2loop:
build:
Expand Down
2 changes: 1 addition & 1 deletion docs/source/_static/m2l_code_template.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@
"from map2loop.m2l_enums import VerboseLevel\n",
"from map2loop.m2l_enums import Datatype\n",
"from map2loop.sampler import SamplerSpacing, SamplerDecimator\n",
"from map2loop.sorter import SorterUseHint, SorterUseNetworkX, SorterAgeBased, SorterAlpha\n",
"from map2loop.sorter import SorterAlpha\n",
"import time"
]
},
Expand Down
116 changes: 116 additions & 0 deletions map2loop/__init__.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,118 @@
from .project import Project
from .version import __version__

import warnings # TODO: convert warnings to logging
from packaging import version as pkg_version # used instead of importlib.version because adheres to PEP 440 using pkg_version.parse
import pathlib
import re
import pkg_resources #import get_distribution, DistributionNotFound # Use pkg_resources for version checking

class DependencyChecker:
'''
A class to check installation and version compatibility of each package in dependencies.txt

Attributes:
package_name (str): Name of the package
dependency_file (str): path to dependencies.txt
required_version (str or None): required version of the package as in dependencies.txt
installed_version (str or None): installed version of the package in the current environment
'''

def __init__(self, package_name, dependency_file="dependencies.txt"):

self.package_name = package_name
self.dependency_file = pathlib.Path(__file__).parent.parent / dependency_file
self.required_version = self.get_required_version()
self.installed_version = self.get_installed_version()

## 1) get required version of each dependency from dependencies.txt
def get_required_version(self):
'''
Get the required package version for each package from dependencies.txt;

Returns:
str or None: The required version of the package (if specified), otherwise None.
'''
try:
with self.dependency_file.open("r") as file:
# search for the dependency line in dependencies.txt
for line in file:
if line.startswith(f"{self.package_name}=="):
match = re.match(rf"^{self.package_name}==([\d\.]+)", line.strip())
if match:
return match.group(1) # get the version if found
elif line.strip() == self.package_name:
# dependency listed without a version
return None
# if dependency line not found in dependencies.txt (shouldn't happen, but just in case)
print(f"{self.package_name} version not found in {self.dependency_file}.")
except FileNotFoundError: # in case dependencies.txt is not found (shouldn't happen)
warnings.warn(
f"{self.dependency_file} not found. Unable to check {self.package_name} version compatibility.",
UserWarning
)
return None

## 2) Check the installed version of the dependency and compare with required version
def get_installed_version(self):
'''
Get the installed version of the package.

Returns:
str: The installed version of the package.
'''
try:
# Get installed version of the package using pkg_resources
return pkg_resources.get_distribution(self.package_name).version
except pkg_resources.DistributionNotFound:
# Raise ImportError if package is not installed
raise ImportError(f"{self.package_name} is not installed. Please install {self.package_name}.")

## 3) Compare the installed version of the dependency with the required version, if applicable
def check_version(self):
'''
Checks if the installed version of the package matches the required version uin the dependencies.txt file.

'''
if self.required_version is None:
# No required version specified, only check if installed
if self.installed_version is None:
raise ImportError(f"{self.package_name} is not installed. Please install {self.package_name} before using map2loop.")
# else:
# print(f"{self.package_name} is installed.")
else:
# Compare versions if required version is specified
if self.installed_version is None or pkg_version.parse(self.installed_version) != pkg_version.parse(self.required_version):
raise ImportError(
f"Installed version of {self.package_name}=={self.installed_version} does not match required version=={self.required_version}. "
f"Please install the correct version of {self.package_name}."
)

# else:
# print(f"{self.package_name} version is compatible.")

# check all dependencies & versions in dependencies.txt
def check_all_dependencies(dependency_file="dependencies.txt"):
dependencies_path = pathlib.Path(__file__).parent.parent / dependency_file
try:
with dependencies_path.open("r") as file:
for line in file:
line = line.strip()
# skip the gdal line
if line.startswith("gdal"):
continue
if line:
# separate package name and version (if available)
if "==" in line:
package_name, _ = line.split("==")
else:
package_name = line

# check version for each package
checker = DependencyChecker(package_name, dependency_file=dependency_file)
checker.check_version()
except FileNotFoundError:
ImportError(f"{dependency_file} not found. No dependencies checked for map2loop", UserWarning)

# Run check for all dependencies listed in dependencies.txt
check_all_dependencies()
4 changes: 2 additions & 2 deletions map2loop/_datasets/config_files/NSW.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"intrusive_text": "intrusive",
"volcanic_text": "volcanic",
"objectid_column": "ID",
"ignore_codes": ["cover"]},
"ignore_lithology_codes": ["cover"]},
"fault": {
"structtype_column": "boundaryty",
"fault_text": "Fault",
Expand All @@ -36,7 +36,7 @@
"dipestimate_column": "faultdipan",
"dipestimate_text": "Moderate,Listric,Steep,Vertical",
"name_column": "name",
"objectid_column": "unique_id"
"objectid_column": "unique_id",
},
"fold": {
"structtype_column": "codedescpt",
Expand Down
2 changes: 1 addition & 1 deletion map2loop/_datasets/config_files/QLD.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"intrusive_text": "Y",
"volcanic_text": "VOLCANIC",
"objectid_column": "ID",
"ignore_codes": ["cover"]
"ignore_lithology_codes": ["cover"]
},
"fault": {
"structtype_column": "type",
Expand Down
2 changes: 1 addition & 1 deletion map2loop/_datasets/config_files/SA.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"intrusive_text": "intrusive",
"volcanic_text": "volc",
"objectid_column": "ID",
"ignore_codes": ["cover"]
"ignore_lithology_codes": ["cover"]
},
"fault": {
"structtype_column": "descriptio",
Expand Down
2 changes: 1 addition & 1 deletion map2loop/_datasets/config_files/TAS.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"intrusive_text": "granit",
"volcanic_text": "volc",
"objectid_column": "ID",
"ignore_codes": ["cover"]
"ignore_lithology_codes": ["cover"]
},
"fault": {
"structtype_column": "TYPE",
Expand Down
2 changes: 1 addition & 1 deletion map2loop/_datasets/config_files/VIC.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"intrusive_text": "intrusion",
"volcanic_text": "volc",
"objectid_column": "ID",
"ignore_codes": ["cover"]
"ignore_lithology_codes": ["cover"]
},
"fault": {
"structtype_column": "featuretyp",
Expand Down
2 changes: 1 addition & 1 deletion map2loop/_datasets/config_files/WA.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"intrusive_text": "intrusive",
"volcanic_text": "volcanic",
"objectid_column": "ID",
"ignore_codes": ["cover"]
"ignore_lithology_codes": ["cover"]
},
"fault": {
"structtype_column": "feature",
Expand Down
Binary file not shown.
Loading