Passing and returning complex data with a ResourceReader

[rnapier]

I'm trying to use Pkl as a configuration language for a Swift financial analysis program. Currently the configuration is extremely complex and written in Swift directly. For example, in the current code there is the following rule:

PortfolioExpense(name: "Retirement Spending", amount: retirementIncome/12, active: retirementDate...),

retirementIncome is passed into this function, and is based on computations from reading a database of past spending. retirementDate is also passed into this function, and is iterated during program execution to do analysis of outcomes based on different dates.

There are many such rules, including options for real vs nominal values, whether it is a fixed values or should be randomized during monte carlo simulation, etc. A key point is that the configuration often refers to values that are only known at runtime, and may even need to generate new configuration items based on runtime information.

To make this work in Pkl, I've been exploring ResourceReader, which seems ideal. It allows Pkl to query the program asynchronously for data while the program is querying Pkl for configuration.

Finally, my question. Some data that needs to be passed to the ResourceReader is not trivial. As a simple example, I would like to work on the current value of various accounts, and so have the following Pkl:

allAccounts: Listing<String> = new Listing<String> {
    "Assets:Money Market"
    "Assets:Savings Bonds"
    "Assets:CDs"
    "Assets:Brokerage"
    "Assets:401k"
    "Assets:HSA"
}

currentSavings: Float = read("ledger:getBalance?accounts=\(allAccounts.join(",").replaceAll(" ", "%20"))").text.toFloat()

This is matched with a ResourceReader:

struct LedgerReader: ResourceReader {
    enum Error: Swift.Error {
        case unknownCommand
    }
    func read(url: URL) async throws -> [UInt8] {
        let comp = URLComponents(url: url, resolvingAgainstBaseURL: false)!
        let command = comp.path

        switch command {
        case "getBalance":
            let accounts = comp.queryItems!.first(where: { $0.name == "accounts" })!.value!.split(separator: ",").map(String.init)
            // `ledger` is a type in the program capable of querying the database.
            return Array(try await ledger.getBalance(accounts: accounts).formatted(.number.grouping(.never)).utf8)
        default:
            throw Error.unknownCommand
        }
    }
    
    let scheme = "ledger"
    let isGlobbable = false
    let hasHierarchicalUris = true
    func listElements(uri: URL) async throws -> [PklSwift.PathElement] { [] }
}

This is a very ugly and fragile way to handle this interaction (particularly things like manually percent-encoding spaces). I think I would like to send it as Base64-encoded JSON (or even BinaryPlist), but I haven't been able to figure out how to use JsonRenderer to do this. Is there a fairly natural way to pass non-trivial data to ResourceReader (or is there a different tool I should be using)?

[bioball]

Resource readers and query params is the right direction for this.

For URI encoding, there's an experimental URI package available that you can use; docs here: https://pkl-lang.org/package-docs/pkg.pkl-lang.org/pkl-pantry/pkl.experimental.uri/current/URI/index.html#encodeComponent%28%29

You can then combine this with JSON encoding if you wish:

import "pkl:json"
import "package://pkg.pkl-lang.org/pkl-pantry/[email protected]#/URI.pkl"

local jsonRenderer = new json.Renderer {}

allAccounts: Listing<String> = new {
    "Assets:Money Market"
    "Assets:Savings Bonds"
    "Assets:CDs"
    "Assets:Brokerage"
    "Assets:401k"
    "Assets:HSA"
}

local allAccountsRendererd = jsonRenderer.renderValue(allAccounts)

currentSavings: Float = read("ledger:getBalance?params=\(URI.encodeComponent(allAccountsRendered))").text.toFloat()

FYI: resources are cached by their URI. So, this approach wouldn't work if you wanted multiple reads to have different results.

[rnapier]

Thanks; this is exactly the kind of thing I was trying to get my head around. I hadn't really internalized how local worked yet.