-
Notifications
You must be signed in to change notification settings - Fork 67
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(middleware): capability end-point manifest #27
Comments
This is a really awesome idea. It seems to me that capabilities will often be implemented as other middleware themselves, and then simply mounted onto the "base" gateway express app. So in order for this capabilities middleware to know all the capabilities of the gateway, each middleware that is mounted will somehow need to "broadcast" the capabilities it is adding to the gateway, if any. Each gateway has a SQLite Database. I propose the following approach:
name TEXT
version TEXT
metadata TEXT (serialized JSON?)
G8way Middleware shape proposalIt would be great if each middleware was self-encapsulated, following a certain shape, and having certain dependencies injected to it by the gateway. A potential approach could be middleware that follows this signature: type ArweaveG8WayMiddleware = (context: ArweaveG8WayContext) => Promise<(app: Express) => Express>
type ArweaveG8WayContext = {
addCapability: (capability: { name: string, version: string, ...rest }): Promise<{ ok: boolean }>
log:
... // other apis the gateway would like to inject into middleware
} by injecting A middleware implementation might look like this: const awesomeArweaveG8WayMiddleware: ArweaveG8WayMiddleware = async ({ addCapability, log }) => {
await addCapability({ name: 'awesome-capability', version: '1.2.3', some: 'other', data: 'here' }).then(...).catch(...)
return (app) => {
app.get(...)
....
return app
}
} Then finally, the base gateway can simply fold over middleware to gain capabilities: const app = express()
... // base gateway things ie. mount built-in routes (just another fold with internal deps?)
const context = {
addCapability: () => {...}
}
const middlewares = [
awesomeArweaveG8WayMiddleware,
anotherMiddleware
]
const $gateway = middlewares.reduce(
async ($app, middleware) => middleware(context).then(m => m(await $app)),
app
)
$gateway.then(gateway => gateway.listen()) These are just ideas and open to scrutiny and feedback of course. I wouldn't mind taking a stab at the implementation, in the base gateway, and the capabilities middleware itself, if folks thought this was a cool approach. |
Thanks for the thoughtful proposal! Before we get too far into implementation details, let's discuss the spec itself. As written, it looks like there's no way to support multiple versions of the same capability. Is the assumption that capabilities are always backwards compatible? |
Absolutely 👍
That's how I understood the spec. Though I could definitely see gateways wanting/needing to support multiple versions of the same capability; the use case being the ability for gateways to deprecate capability version support and allow migration to new versions, without breaking user-space.
The spec describes Maybe the spec should allow for Tangentially related to this question: what ought to be responsible for defining how a capability's different versions are exposed? The gateway? Right now, I lean towards the specific capability spec; all of the possible capability use-cases aren't clear to me. The obvious use-case is adding public routes, but some maybe could add support for certain headers, and then some may event augment internal behavior of the gateway. So until a pattern emerges, I lean towards keeping the constraints as local to the specific capability as possible. Edits: wording clarification, typos |
I'd be inclined to say backwards compatibility is required and new capabilities should require a new name (e.g. append a '-v2'). In that context, I don't think semantic versions make much sense. A single number that increases as features are added would likely be better. (see this talk for some of the rationale for this)
Mostly doesn't matter as long as we don't break backwards compatibility. We just allow a single version of each capability. Anything breaking becomes a new capability.
We should be extremely cautious about extending or modify the core API. Headers might be a gray area since most probably will not conflict, but in general I think extensions should live under there own routes. That allows space for experimentation without breaking the core API. Eventually I imagine some will stabilize and their functionality can be integrated more deeply. |
All your points on versioning lgtm, and makes other issues w.r.t deprecating and migrating to new versions moot 👍 . The spec draft would need to be revised to reflect those points.
💯 absolutely. My proposal for core to inject apis into middleware stems from that desire, only allowing middleware to call black-box apis that are provided to it, by core. It helps ensure middleware can augment in ways that core allows. Only once a capability stabilizes and becomes almost ubiquitous with gateways ought it be considered for deeper integration in core (which could simply be core internally composing it instead of the gateway runner). But being new here, I admit I don't know all of the use-cases for middleware 😄
Seems reasonable. This could just be a convention for middleware developers to follow. But if it's decided that the core gateway ought to enforce that, then core would need to provide an api that each middleware could use ie. The impl of |
I wonder if there should be any distinction between capabilities and extensions. 🤔 It seems like we could have a completely self-contained extension mechanism the enables self-describing extensions (could be a single 'extensions' capability), and use capabilities as more of a way to describe features provided by the base protocol. |
Reminds me, I should have asked this earlier - do you have a particular use case in mind yourself? |
#26 and https://specs.g8way.io/#/view/lXLd0OPwo-dJLB_Amz5jgIeDhiOkjXuM3-r0H_aiNj0 is interesting as it would enable apps to not have to use hash-based client-side routing. Also ANS-108 is really interesting as it means gateways could serve up applications designed to render certain kinds of content. Worth noting that both of those are not standalone routes to be mounted on a gateway but change behavior of the gateway itself. |
Gotcha. That context is helpful. I think we should probably split this into two tickets - one about future extension mechanisms and the other (this ticket) about the capabilities end-point. We can focus on the path manifests change first and the capabilities shortly capabilities end-point after that. @TillaTheHun0, can you add a new issue about the future extension mechanism including a rationale for it and a list of requirements based on that rationale? Re ANS-108, it's an interesting proposal, but it's also a breaking change. It both changes the behavior for existing content tagged with Render-With and breaks any existing data consumer that expects to retrieve unaltered without having to worry about how it's tagged. If we decide to implement it, we'll need to give plenty of warning about it in advance. |
Maybe we could consider an endpoint like `raw` but for rendering.
Like`/render/:tx` or something or `/tx?render=true`, the capabilities
endpoint can inform developers of gateways supporting the render option.
Also, this could be an opt-in and not a breaking change.
We could potentially experiment via a `/labs` route where proposed routes
and functionality can be in a sandbox to iterate on design and
implementation without impacting production. These experiments do not even
have to be merged in to the main code base, they could implemented
downstream in forks and can be evaluated in the real world.
Just a thought.
…On Tue, Aug 15, 2023 at 3:50 PM David Whittington ***@***.***> wrote:
@TillaTheHun0 <https://github.com/TillaTheHun0> Gotcha. That context is
helpful. I think we should probably split this into two tickets - one about
future extension mechanisms and the other (this ticket) about the
capabilities end-point. We can focus on the path manifests change first and
the capabilities shortly capabilities end-point after that.
@TillaTheHun0 <https://github.com/TillaTheHun0> Can you add a new ticket
about the future extension mechanism including a rationale for the it and a
list of requirements based on that rationale?
Re ANS-108, it's an interesting proposal, but it's also a breaking change.
It both changes the behavior for existing content tagged with Render-With
and breaks any existing data consumer that expects to retrieve unaltered
without having to worry about how it's tagged. If we decide to implement
it, we'll need to give plenty of warning about it in advance.
—
Reply to this email directly, view it on GitHub
<#27 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAFGLBXINMTOZFLOWN4PELXVPHJTANCNFSM6AAAAAA3KCMEWQ>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
I like @twilson63 suggestion for a As for
Sure. I can do that. |
I'll drop some thoughts on the new ticket when I get more time, but I agree the spirit of this. I'd like extension to mostly live outside the main code base and be loadable as modules. |
…lity via ANS-101 impl ar-io#27
Good idea! An intentionally more awkward URL is proposed here: #34 (comment) |
Continuing the discussion on how to describe capabilities: I think we need the following updates to the existing ANS:
We should also continue thinking about:
|
Background:
Since there will likely be many different gateways with many different capabilities, it is helpful to implement a middleware component that provides capability insights to applications as they interact with the gateway.
ANS-101: Proposes this here: https://specs.g8way.io/#/view/hLSKTSwd5_3xB71zciyK_WFEpK9wVX2IeGzxk9Yl2xY
This could be implemented with a middleware attached to the
/info/capabilities
route and returns a JSON document describing the supported capabilities of this gateway. This could allow applications to let users choose their own gateways and enable features based on the capabilities of each gateway.The text was updated successfully, but these errors were encountered: