Watt Time v3 Support #532

vaughanknight · 2024-06-17T06:44:55Z

Pull Request

Issue Number: #510

Summary

PR for all WattTime v3 updates

Changes

TBC

Checklist

Local Tests Passing?
CICD and Pipeline Tests Passing?
Added any new Tests?
Documentation Updates Made?
Are there any API Changes? If yes, please describe below.
This is not a breaking change. If it is, please describe it below.

Are there API Changes?

If yes, what are the expected API Changes? Please link to an API-Comparison
workflow with the API Diff.

Is this a breaking change?

If yes, what workflow does this break?

Anything else?

Other comments, collaborators, etc.

Please follow
GitHub's suggested syntax
to link Pull Requests to Issues via keywords

This PR Closes #<issue_number>

First draft of the ADR for watt time v3 changes. Looking at path mappings and parameters. Still plenty to work on.

Moved ADR to correct location

Signed-off-by: Vaughan Knight <[email protected]>

…e-Foundation/carbon-aware-sdk into wattime-v3-support

updated notes for BA Signed-off-by: Vaughan Knight <[email protected]>

First draft of the ADR for watt time v3 changes. Looking at path mappings and parameters. Still plenty to work on.

Moved ADR to correct location

Signed-off-by: Vaughan Knight <[email protected]>

updated notes for BA Signed-off-by: Vaughan Knight <[email protected]>

More updates.

Added AuthenticationBaseUrl to the configuration and updated Authentication to leverage the v3 authenication path - note the API is not updated and will require further updates.

Updated start and end configuration to new values

Balancing Authority Renamed to Region. Does not include updates to API, just the Query String parameter.

Updates for historical data API

Removed accidental file

Lots of test updates, need to do some fixes.

Historical forecasts updated

I, Vaughan Knight <[email protected]>, hereby add my Signed-off-by to this commit: b9490e4 I, Vaughan Knight <[email protected]>, hereby add my Signed-off-by to this commit: b991bac I, Vaughan Knight <[email protected]>, hereby add my Signed-off-by to this commit: e4f1494 I, Vaughan Knight <[email protected]>, hereby add my Signed-off-by to this commit: b443e9e I, Vaughan Knight <[email protected]>, hereby add my Signed-off-by to this commit: ab1205d I, Vaughan Knight <[email protected]>, hereby add my Signed-off-by to this commit: 7c115fa I, Vaughan Knight <[email protected]>, hereby add my Signed-off-by to this commit: e047c9a I, Vaughan Knight <[email protected]>, hereby add my Signed-off-by to this commit: aa81382 I, Vaughan Knight <[email protected]>, hereby add my Signed-off-by to this commit: 8640c8c I, Vaughan Knight <[email protected]>, hereby add my Signed-off-by to this commit: 880fcf7 Signed-off-by: Vaughan Knight <[email protected]>

Many tests reworked, a few to go. Consolidated a lot of the hand crafted json objects into objects that get serialized as the purist JsonObject format was prone to errors - in some cases tests were passing even with bad typing.

Further test updates

…e-Foundation/carbon-aware-sdk into wattime-v3-support

Further updates, just 1 test left to remediate

…ests Updated to add authentication client to the service builder for the tests. All tests now passing.

Renaming of Balancing Authority to Region through all code and comments. This will also need updating through documentation.

src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs

Fixed spelling error in latitude

src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs

src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs

Updates based on code review for WattTime Tests. Mostly cleanup of constants which were removed elsewhere in tests.

vaughanknight · 2024-06-19T06:47:45Z

...ataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs

These updates will need propagating to the version upgrade documentation @danuw just tagging you so we don't forget. RegionCacheTTL replaces BalancingAuthorityCacheTTL.

src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs

src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs

Cleaned up a lot of the string literals. They were causing too much fragility in the code base, and made it complex when updating the WattTime API.

More cleanup of some of the strings. Creating consistency for using the test data on parameters and not just reponses also.

Updates to documentation and changelog

Willmish · 2024-06-25T07:13:21Z

EDIT: False alarm, the below tests succeed on a fresh clone of the repo, the below stack trace I am leaving for openness’s sake, but it was due to a remnant appsettings.Development.json file sitting in CarbonAware.WebApi/src.

The failing test suite:
carbon-aware-sdk/src/CarbonAware.WebApi/test/integrationTests
Stack trace from failed integration test (Ran by dotnet test in /src folder on a development docker container):
Trace here: https://pastebin.com/LPvpZQkZ

Willmish

LGTM, I left 3 comments for small changes, which would be nice to see but happy for it to get merged without them as well. Thanks for moving WattTime test magic numbers to a single file btw :)

Willmish · 2024-06-25T08:40:57Z

src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs

@@ -238,7 +249,8 @@ private async Task UpdateAuthTokenAsync()
    private void SetBasicAuthenticationHeader()
    {
        var authToken = Encoding.UTF8.GetBytes($"{this._configuration.Username}:{this._configuration.Password}");
-        this._client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(AuthenticationHeaderTypes.Basic, Convert.ToBase64String(authToken));
+        //this._client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(AuthenticationHeaderTypes.Basic, Convert.ToBase64String(authToken));


We can drop the comment? (this._client will not exist after the PR comes through in this context)

Willmish · 2024-06-25T08:50:45Z

...bonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsDataResponse.cs

+
+
+    [JsonPropertyName("meta")]
+    public GridEmissionsMetaData Meta { get; set; }


nit: constructor will leave null value if called without params, maybe make nullable (add ?) or return empty.

Willmish · 2024-06-25T08:54:50Z

...ware.DataSources/CarbonAware.DataSources.WattTime/src/Model/ForecastEmissionsDataResponse.cs

+
+
+    [JsonPropertyName("meta")]
+    public GridEmissionsMetaData Meta { get; set; }


NIT: same as before, can make nullable or by default initialise an empty GridEmissionsMetaData object

YaSuenag

Both ElectricityMaps and ElectricityMapsFree datasources works fine on my machine, and also all tests has passed on Visual Studio 2022 Community.

danuw

LGTM

vaughanknight added 23 commits May 7, 2024 16:56

First draft of the ADR for watt time v3 changes

b9490e4

First draft of the ADR for watt time v3 changes. Looking at path mappings and parameters. Still plenty to work on.

Moved ADR to correct location

b991bac

Moved ADR to correct location

Further updates for the watt time v2 to v3 upgrade

f4bbf52

Signed-off-by: Vaughan Knight <[email protected]>

Further updates for the watt time v2 to v3 upgrade

d6ad67c

Signed-off-by: Vaughan Knight <[email protected]>

Merge branch 'wattime-v3-support' of https://github.com/Green-Softwar…

15a6f9c

…e-Foundation/carbon-aware-sdk into wattime-v3-support

Update 0016-watt-time-v3.md

cb46877

updated notes for BA Signed-off-by: Vaughan Knight <[email protected]>

First draft of the ADR for watt time v3 changes

e324f36

First draft of the ADR for watt time v3 changes. Looking at path mappings and parameters. Still plenty to work on.

Moved ADR to correct location

be6663c

Moved ADR to correct location

Further updates for the watt time v2 to v3 upgrade

17210ed

Signed-off-by: Vaughan Knight <[email protected]>

Update 0016-watt-time-v3.md

7920786

updated notes for BA Signed-off-by: Vaughan Knight <[email protected]>

Create 0016-watt-time-v3.md

e4f1494

More updates.

Added base url to the configuration with validation

b443e9e

Added AuthenticationBaseUrl to the configuration and updated Authentication to leverage the v3 authenication path - note the API is not updated and will require further updates.

Updated start and end configuration

ab1205d

Updated start and end configuration to new values

Balancing Authority Parameter Renamed to Region

7c115fa

Balancing Authority Renamed to Region. Does not include updates to API, just the Query String parameter.

Updates for historical data API

e047c9a

Updates for historical data API

Removed accidental file

aa81382

Removed accidental file

Lots of test updates

8640c8c

Lots of test updates, need to do some fixes.

Historical forecasts updated

880fcf7

Historical forecasts updated

Many tests reworked, a few to go

39e45a6

Many tests reworked, a few to go. Consolidated a lot of the hand crafted json objects into objects that get serialized as the purist JsonObject format was prone to errors - in some cases tests were passing even with bad typing.

Further test updates

fbfcac1

Further test updates

Merge branch 'wattime-v3-support' of https://github.com/Green-Softwar…

0080ae8

…e-Foundation/carbon-aware-sdk into wattime-v3-support

Merge branch 'wattime-v3-support' of https://github.com/Green-Softwar…

f3fd570

…e-Foundation/carbon-aware-sdk into wattime-v3-support

danuw mentioned this pull request Jun 18, 2024

🎯 2024-06-18 Carbon Aware SDK Weekly Meeting Agenda #533

Closed

45 tasks

vaughanknight added 3 commits June 18, 2024 17:13

Further updates, just 1 test left to remediate

9aabd27

Further updates, just 1 test left to remediate

Updated to add authentication client to the service builder for the t…

8127a92

…ests Updated to add authentication client to the service builder for the tests. All tests now passing.

Renaming of Balancing Authority to Region

4ff0151

Renaming of Balancing Authority to Region through all code and comments. This will also need updating through documentation.

github-advanced-security bot found potential problems Jun 18, 2024

View reviewed changes

src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs Fixed Show fixed Hide fixed

src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs Fixed Show fixed Hide fixed

Fixed spelling error in latitude

d93320f

Fixed spelling error in latitude

github-advanced-security bot found potential problems Jun 18, 2024

View reviewed changes

src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs Dismissed Show dismissed Hide dismissed

src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs Dismissed Show dismissed Hide dismissed