Skip to content

Commit

Permalink
test(tests and docs): added new test for open datasoft session start …
Browse files Browse the repository at this point in the history
…and ammended documentation [2024-11-26]
  • Loading branch information
CHRISCARLON committed Nov 26, 2024
1 parent 8ef8910 commit c87c435
Show file tree
Hide file tree
Showing 2 changed files with 43 additions and 10 deletions.
23 changes: 13 additions & 10 deletions documentation/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,14 @@ if __name__ == "__main__":

#### Initialisation
```python
from cat_session import CatSession
from cat_explorer import OpenDataSoftCatExplorer
import HerdingCats as hc

def main():
with hc.CatSession(hc.OpenDataSoftDataCatalogues.UK_POWER_NETWORKS) as session:
explore = hc.OpenDataSoftCatExplorer(session)

with CatSession("ukpowernetworks.opendatasoft.com") as session:
explorer = OpenDataSoftCatExplorer(session)
if __name__ == "__main__":
main()
```

#### Methods
Expand Down Expand Up @@ -131,10 +134,10 @@ def main():
polars_df = loader.polars_data_loader(resource_list)
pandas_df = loader.pandas_data_loader(resource_list)

# Load into in-memory DuckDB
# Load into in-memory DuckDB and specify db name and table name
loader.duckdb_data_loader(resource_list, "cycle_hire_db", "daily_hires")

# Load into S3 as Parquet
# Load into S3 as Parquet - AWS creds need to be configured with something like AWS vault for this
loader.aws_s3_data_loader(
resource_list,
"my-data-bucket",
Expand All @@ -146,9 +149,9 @@ if __name__ == "__main__":
main()
```

## Supported Data Formats for Resource Loader
## Supported File Types for Resource Loader

The library currently supports the following data formats:
The Resource Loader currently supports the following data formats:
- Excel (.xlsx) ✅
- CSV ✅
- JSON (partial support) ✅
Expand All @@ -159,9 +162,9 @@ Future format support planned for:
- Shapefile
- GeoJSON

## Storage Solutions
## Data Formats and Storage Solutions

Current storage solutions supported:
Current data formats and storage solutions supported:
- Polars DataFrame ✅
- Pandas DataFrame ✅
- DuckDB (local) ✅
Expand Down
30 changes: 30 additions & 0 deletions tests/open_data_soft/test_ods_session_creation.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import pytest
from HerdingCats.session.cat_session import CatSession

@pytest.fixture
def domain():
return "ukpowernetworks.opendatasoft.com"

def test_cat_session_creation(domain):
"""
Check that a valid Ckan session can be created
"""
try:
session = CatSession(domain)
assert isinstance(
session, CatSession
), "OpenDataSoftCatSession object should be created"
assert session.domain == domain, "OpenDataSoftCatSession should have the correct domain"
assert (
session.base_url == f"https://{domain}"
), "OpenDataSoftCatSession should have the correct base URL"
except Exception as e:
pytest.fail(f"Failed to create OpenDataSoftCatSession: {str(e)}")


def test_cat_session_start(domain):
try:
with CatSession(domain) as session:
assert session.session is not None, "Session object should be created"
except Exception as e:
pytest.fail(f"Failed to start CkanCatSession: {str(e)}")

0 comments on commit c87c435

Please sign in to comment.