Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inject extension scopes while running the resolution algorithm #1170

Draft
wants to merge 83 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
aa18f14
Add test case which broke with query engine not skipping trivia in on…
ggiraldez Oct 24, 2024
42bfde6
Make using directives inherited in Solidity < 0.7.0
ggiraldez Oct 25, 2024
634b33b
Resolve elementary types in expressions and add their built-ins
ggiraldez Oct 25, 2024
458855c
Bind constants as library members
ggiraldez Oct 25, 2024
bcd0d1f
Propagate dynamic scopes when escaping a contract's lexical scope
ggiraldez Oct 28, 2024
915fbd8
Bind elementary casting and struct construction to the resulting type
ggiraldez Oct 28, 2024
f2ab055
Fix calling parent constructors in constructors
ggiraldez Oct 28, 2024
13cf7e5
Bind arithmetic, logical, bit-wise, comparison, etc. expressions
ggiraldez Oct 29, 2024
5d8c379
Propagate the dynamic scope with resolving contract bases
ggiraldez Oct 29, 2024
ddac849
Define a new scope for internal state variables for derived contracts
ggiraldez Oct 29, 2024
d3032c8
Propagate dynamic scope also from libraries
ggiraldez Oct 29, 2024
a5f3eda
Resolve using dynamic scope from an interface/contract casting
ggiraldez Oct 29, 2024
901501a
Resolve attached functions when applied to namespace qualified types
ggiraldez Oct 30, 2024
1d566b4
Public getters are used as functions, so bind them as such
ggiraldez Oct 30, 2024
a38e8bd
Functions can be called in derived contracts by qualifying with the name
ggiraldez Oct 30, 2024
3aab65d
Add support for binding unnamed function declarations
ggiraldez Oct 30, 2024
9d620d7
Support old `var` declarations
ggiraldez Oct 30, 2024
7daa1f4
Make inherited state vars accessible in all parent contracts
ggiraldez Oct 30, 2024
0d2802f
Normalize value type aliases to bind attached functions correctly
ggiraldez Oct 31, 2024
58ced15
Support binding literal hex addresses
ggiraldez Oct 31, 2024
438cb81
Prior to 0.5.0, `this` could be used as an `address`
ggiraldez Oct 31, 2024
9cbb4fc
Add binding test cases for most of the remaining Sanctuary issues
ggiraldez Nov 1, 2024
28640a4
Make pushing the dynamic scope optional for libraries
ggiraldez Nov 1, 2024
f7655e7
`_slot`/`_offset` suffixes for storage variables in Solidity < 0.7.0
ggiraldez Nov 4, 2024
b3bcf54
Bind to the output of both operands for all arithmetic and bitwise op…
ggiraldez Nov 4, 2024
d6b7174
Bind legacy call options `.value()` and `.gas()` on Solidity < 0.7.0
ggiraldez Nov 4, 2024
5ec110b
Bind user defined types `.wrap()` and `.unwrap()`
ggiraldez Nov 4, 2024
ac13986
Allow binding attached functions on typed casted expressions
ggiraldez Nov 4, 2024
6a560d5
Bind modifiers in libraries
ggiraldez Nov 4, 2024
4df6b54
Bind legacy constructor parent invocations
ggiraldez Nov 4, 2024
6268c9d
Allow qualified access to bind to internal members of parent contract
ggiraldez Nov 4, 2024
d8b5d90
Resolve qualified members in any position in the inheritance chain
ggiraldez Nov 4, 2024
92f5ebc
Support `.selector` field for custom error types
ggiraldez Nov 5, 2024
f798acd
Binding rules: remove @contract.state_vars, rename type_members to ns
ggiraldez Nov 5, 2024
6725fe5
Define `super` and push `using` extensions only when necessary
ggiraldez Nov 5, 2024
ad343ac
Reworked how resolution interacts with `using` directive extensions
ggiraldez Nov 6, 2024
5c4754a
Fix binding state variable initialization value through the extended …
ggiraldez Nov 7, 2024
b027ba5
Allow `using` directives to be inherited in Solidity < 0.7.0
ggiraldez Nov 14, 2024
d0945ef
Fix extension binding for function return parameters
ggiraldez Nov 7, 2024
05c9335
External functions have legacy call options
ggiraldez Nov 7, 2024
172d7a8
Ensure extensions are pushed before resolving `super` and `this`
ggiraldez Nov 7, 2024
0f7ff81
Attempt to simplify contract/interface instance scope
ggiraldez Nov 11, 2024
7aa82df
Remove the generic type resolution for arrays and hard-code built-in …
ggiraldez Nov 12, 2024
172e5ed
Bind user defined type's `wrap` and `unwrap` functions return types
ggiraldez Nov 12, 2024
757111d
Only connect the star extension scope `using for *` when there is one
ggiraldez Nov 12, 2024
aecab6f
Bind array and mappings accesses through public getter functions
ggiraldez Nov 12, 2024
a9f0986
Support attaching functions to mappings with `using`
ggiraldez Nov 12, 2024
54217b2
Clean up and add more documentation to the binding rules
ggiraldez Nov 15, 2024
05d27d6
Use PascalCase for most built-in types
ggiraldez Nov 18, 2024
4579182
Bind members of Yul external variables
ggiraldez Nov 20, 2024
9aab1b3
Fix binding structs as public getter results
ggiraldez Nov 20, 2024
2b3ae36
Rename built-in type for User Defined Value Types
ggiraldez Nov 20, 2024
1163b07
Add names to most parameters of built-in functions
ggiraldez Nov 20, 2024
576ae45
Define a `%this` built-in to use in the context of libraries
ggiraldez Nov 21, 2024
c6b73c6
Bind inherited public getters
ggiraldez Nov 21, 2024
a015082
Fix scope bindings for modifiers in libraries
ggiraldez Nov 22, 2024
fc929d3
Bind attached functions when there's a `using .. for *` in a library
ggiraldez Nov 22, 2024
90a4f26
Allow Yul functions to access external constants
ggiraldez Nov 22, 2024
d5823c0
`this` and `super` don't generate references
ggiraldez Nov 25, 2024
4768079
Refactor elementary types handling in binding rules
ggiraldez Nov 26, 2024
5ed161b
Bind `x_slot` in Solidity <= 0.5.0 to the `slot` built-in
ggiraldez Nov 26, 2024
a6b610c
Add test case with Yul identifiers with dots
ggiraldez Nov 26, 2024
4c19da0
Tweak some parameters of built-in functions
ggiraldez Nov 26, 2024
b62e37e
Merge remote-tracking branch 'upstream/main' into sanctuary-fixes-only
ggiraldez Nov 27, 2024
f6d3113
Update snapshots after merge
ggiraldez Nov 27, 2024
a865d10
Update performance test counts and ignore built-ins in them
ggiraldez Nov 27, 2024
c278a6c
Fix formatting
ggiraldez Nov 27, 2024
14dea5a
Use extension scope and extension hooks to graph and customize resolver
ggiraldez Nov 19, 2024
89133bc
Add extension hooks and scopes and disable old scope stack rules for …
ggiraldez Nov 21, 2024
a4dc6a4
Add resolver options to disable certain operations
ggiraldez Nov 21, 2024
7ad1e0c
Disable remaining rules used for extension pushing via rules
ggiraldez Nov 21, 2024
a9b12c2
Allow some ranking in resolver when running in reentrant safety
ggiraldez Nov 21, 2024
da00828
Clean up resolver code, removing commented code and unnecessary code
ggiraldez Nov 27, 2024
e96335b
Simplify binding rules eliminating extended scopes
ggiraldez Nov 27, 2024
152a0d4
Inherited extensions are now indicated from the rules themselves
ggiraldez Nov 27, 2024
362b35c
Rename `simple_resolve` to `non_recursive_resolve`
ggiraldez Nov 27, 2024
d8730de
Refactor: rename `jump_to_definition` to `resolve_definition`
ggiraldez Nov 27, 2024
ad4e21e
Move `lookup_definition_by_name` to tests crate
ggiraldez Nov 27, 2024
fb68f12
Remove unused `NoCancellation` struct
ggiraldez Nov 27, 2024
9d37caa
Update public_api.txt
ggiraldez Nov 27, 2024
3a64d71
Move built-ins parsing and ingestion to `slang_solidity` crate
ggiraldez Nov 27, 2024
a34c3ec
Update public_api.txt
ggiraldez Nov 27, 2024
58fa2b8
Added documentation on the added graph attributes
ggiraldez Nov 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions crates/metaslang/bindings/generated/public_api.txt

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

117 changes: 108 additions & 9 deletions crates/metaslang/bindings/src/builder/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@
//! To do this, add a `source_node` attribute, whose value is a syntax node capture:
//!
//! ``` skip
//! @func [FunctionDefinition ... [FunctionName @id [Identifier]] ...] {
//! @func [FunctionDefinition [FunctionName @id [Identifier]]] {
//! node def
//! attr (def) type = "pop_symbol", symbol = (source-text @id), source_node = @func, is_definition
//! }
Expand Down Expand Up @@ -161,7 +161,7 @@
//! `syntax_type` attribute, whose value is a string indicating the syntax type.
//!
//! ``` skip
//! @func [FunctionDefinition ... [FunctionName @id [Identifier]] ...] {
//! @func [FunctionDefinition [FunctionName @id [Identifier]]] {
//! node def
//! ; ...
//! attr (def) syntax_type = "function"
Expand All @@ -175,7 +175,7 @@
//! `definiens_node` attribute, whose value is a syntax node that spans the definiens.
//!
//! ``` skip
//! @func [FunctionDefinition ... [FunctionName @id [Identifier]] ... @body [FunctionBody] ...] {
//! @func [FunctionDefinition [FunctionName @id [Identifier]] @body [FunctionBody]] {
//! node def
//! ; ...
//! attr (def) definiens_node = @body
Expand All @@ -189,7 +189,7 @@
//! To connect two stack graph nodes, use the `edge` statement to add an edge between them:
//!
//! ``` skip
//! @func [FunctionDefinition ... [FunctionName @id [Identifier]] ...] {
//! @func [FunctionDefinition [FunctionName @id [Identifier]]] {
//! node def
//! attr (def) type = "pop_symbol", symbol = (source-text @id), source_node = @func, is_definition
//! node body
Expand All @@ -201,7 +201,7 @@
//! you can add a `precedence` attribute to each edge to indicate which paths are prioritized:
//!
//! ``` skip
//! @func [FunctionDefinition ... [FunctionName @id [Identifier]] ...] {
//! @func [FunctionDefinition [FunctionName @id [Identifier]]] {
//! node def
//! attr (def) type = "pop_symbol", symbol = (source-text @id), source_node = @func, is_definition
//! node body
Expand All @@ -220,7 +220,7 @@
//! ``` skip
//! global ROOT_NODE
//!
//! @func [FunctionDefinition ... [FunctionName @id [Identifier]] ...] {
//! @func [FunctionDefinition [FunctionName @id [Identifier]]] {
//! node def
//! attr (def) type = "pop_symbol", symbol = (source-text @id), source_node = @func, is_definition
//! edge ROOT_NODE -> def
Expand All @@ -235,14 +235,80 @@
//! a scope node with a kind as follows:
//!
//! ``` skip
//! @func [FunctionDefinition ... [FunctionName @id [Identifier]] ...] {
//! @func [FunctionDefinition [FunctionName @id [Identifier]]] {
//! ; ...
//! node param_scope
//! attr (param_scope) debug_kind = "param_scope"
//! ; ...
//! }
//! ```
//!
//! ### Other node attributes introduced in Slang's usage of stack-graphs
//!
//! #### `tag` attribute
//!
//! This is used to attach a specific meaning to the node, to alter the ranking
//! algorithm used when attempting to disambiguate between multiple definitions
//! found for a reference. This is an optional string attribute.
//!
//! Possible values:
//!
//! - "alias": marks a definition node as a semantic alias of another definition
//! (eg. an import alias)
//!
//! - "c3": used to mark a function/method definition to be a candidate in
//! disambiguation using the C3 linearisation algorithm. In order for C3
//! linearisation to be possible, type hierarchy attributes need to be provided
//! as well (see `parents` attribute below).
//!
//! - "super": marks a reference as a call to super virtual call. This modifies
//! the C3 linearisation algorithm by eliminating the candidates that are at
//! or further down the hierarchy of where the reference occurs. To determine
//! where the reference occurs, we also use the `parents` attribute.
//!
//! #### `parents` attribute
//!
//! Is used to convey semantic hierarchy. Can be applied to both definitions and
//! references. It's an optional, list of graph nodes attribute.
//!
//! For references it can indicate in which language context the reference
//! occurs (eg. in which method or class). For definitions it can indicate the
//! enclosing type of the definition, or parent classes in a class hierarchy.
//! The parent handles themselves can refer to definitions or references. In the
//! later case, generally speaking they will need to be resolved at resolution
//! time in order to be useful.
//!
//! #### `export_node` and `import_nodes`
//!
//! These are used to define static fixed edges to add via `set_context()`.
//! Using `set_context()` will modify the underlying stack graph by inserting
//! edges from the `import_nodes` of all parents (resolved recursively) of the
//! given context, to the `export_node` associated with the context.
//!
//! This can be used to inject virtual method implementations defined in
//! subclasses in the scope of their parent classes, which are otherwise
//! lexically inaccessible.
//!
//! `export_node` is an optional graph node attribute, and `import_nodes` is an
//! optional list of graph nodes. Both apply only to definition nodes.
//!
//! #### `extension_hook`, `extension_scope` and `inherit_extensions`
//!
//! These attributes enable the bindings API to resolve extension methods by
//! injecting specific scopes at potentially unrelated (lexically speaking)
//! nodes in the stack graph. Availability and application of extension scopes
//! depend on the call site (ie. the reference node). Thus, the extension scope
//! to (potentially) apply when resolving a reference is computed by looking up
//! the `parents` of the reference and then querying those parent nodes for
//! their `extension_scope` (an optional scope node). Any extension providing
//! node can also have the `inherit_extensions` attribute (a boolean) which
//! indicates that the algorithm should recurse and resolve its parents to
//! further look for other extensions scopes.
//!
//! Finally, the attribute `extension_hook` defines where in the graph should
//! these extension scopes be injected. This is typically the root lexical
//! scope. This attribute applies to any scope node and is boolean.
//!

mod cancellation;
mod functions;
Expand Down Expand Up @@ -282,6 +348,9 @@ static IS_DEFINITION_ATTR: &str = "is_definition";
static IS_ENDPOINT_ATTR: &str = "is_endpoint";
static IS_EXPORTED_ATTR: &str = "is_exported";
static IS_REFERENCE_ATTR: &str = "is_reference";
static EXTENSION_HOOK_ATTR: &str = "extension_hook";
static EXTENSION_SCOPE_ATTR: &str = "extension_scope";
static INHERIT_EXTENSIONS_ATTR: &str = "inherit_extensions";
static PARENTS_ATTR: &str = "parents";
static SCOPE_ATTR: &str = "scope";
static SOURCE_NODE_ATTR: &str = "source_node";
Expand All @@ -302,6 +371,8 @@ static POP_SCOPED_SYMBOL_ATTRS: Lazy<HashSet<&'static str>> = Lazy::new(|| {
EXPORT_NODE_ATTR,
IMPORT_NODES_ATTR,
SYNTAX_TYPE_ATTR,
EXTENSION_SCOPE_ATTR,
INHERIT_EXTENSIONS_ATTR,
])
});
static POP_SYMBOL_ATTRS: Lazy<HashSet<&'static str>> = Lazy::new(|| {
Expand All @@ -315,6 +386,8 @@ static POP_SYMBOL_ATTRS: Lazy<HashSet<&'static str>> = Lazy::new(|| {
EXPORT_NODE_ATTR,
IMPORT_NODES_ATTR,
SYNTAX_TYPE_ATTR,
EXTENSION_SCOPE_ATTR,
INHERIT_EXTENSIONS_ATTR,
])
});
static PUSH_SCOPED_SYMBOL_ATTRS: Lazy<HashSet<&'static str>> = Lazy::new(|| {
Expand All @@ -336,8 +409,14 @@ static PUSH_SYMBOL_ATTRS: Lazy<HashSet<&'static str>> = Lazy::new(|| {
PARENTS_ATTR,
])
});
static SCOPE_ATTRS: Lazy<HashSet<&'static str>> =
Lazy::new(|| HashSet::from([TYPE_ATTR, IS_EXPORTED_ATTR, IS_ENDPOINT_ATTR]));
static SCOPE_ATTRS: Lazy<HashSet<&'static str>> = Lazy::new(|| {
HashSet::from([
TYPE_ATTR,
IS_EXPORTED_ATTR,
IS_ENDPOINT_ATTR,
EXTENSION_HOOK_ATTR,
])
});

// Edge attribute names
static PRECEDENCE_ATTR: &str = "precedence";
Expand All @@ -362,6 +441,7 @@ pub(crate) struct Builder<'a, KT: KindTypes + 'static> {
cursors: HashMap<Handle<Node>, Cursor<KT>>,
definitions_info: HashMap<Handle<Node>, DefinitionBindingInfo<KT>>,
references_info: HashMap<Handle<Node>, ReferenceBindingInfo>,
extension_hooks: HashSet<Handle<Node>>,
}

pub(crate) struct BuildResult<KT: KindTypes + 'static> {
Expand All @@ -370,6 +450,8 @@ pub(crate) struct BuildResult<KT: KindTypes + 'static> {
pub cursors: HashMap<Handle<Node>, Cursor<KT>>,
pub definitions_info: HashMap<Handle<Node>, DefinitionBindingInfo<KT>>,
pub references_info: HashMap<Handle<Node>, ReferenceBindingInfo>,
// Nodes where we want to inject extensions
pub extension_hooks: HashSet<Handle<Node>>,
}

impl<'a, KT: KindTypes + 'static> Builder<'a, KT> {
Expand All @@ -392,6 +474,7 @@ impl<'a, KT: KindTypes + 'static> Builder<'a, KT> {
cursors: HashMap::new(),
definitions_info: HashMap::new(),
references_info: HashMap::new(),
extension_hooks: HashSet::new(),
}
}

Expand Down Expand Up @@ -480,6 +563,7 @@ impl<'a, KT: KindTypes + 'static> Builder<'a, KT> {
cursors: self.cursors,
definitions_info: self.definitions_info,
references_info: self.references_info,
extension_hooks: self.extension_hooks,
})
}

Expand Down Expand Up @@ -896,6 +980,15 @@ impl<'a, KT: KindTypes> Builder<'a, KT> {
None => Vec::new(),
};

let extension_scope = match node.attributes.get(EXTENSION_SCOPE_ATTR) {
Some(extension_scope) => {
Some(self.node_handle_for_graph_node(extension_scope.as_graph_node_ref()?))
}
None => None,
};

let inherit_extensions = Self::load_flag(node, INHERIT_EXTENSIONS_ATTR)?;

self.definitions_info.insert(
node_handle,
DefinitionBindingInfo {
Expand All @@ -904,13 +997,19 @@ impl<'a, KT: KindTypes> Builder<'a, KT> {
parents,
export_node,
import_nodes,
extension_scope,
inherit_extensions,
},
);
} else if stack_graph_node.is_reference() {
self.references_info
.insert(node_handle, ReferenceBindingInfo { tag, parents });
}

if Self::load_flag(node, EXTENSION_HOOK_ATTR)? {
self.extension_hooks.insert(node_handle);
}

Ok(())
}

Expand Down
48 changes: 36 additions & 12 deletions crates/metaslang/bindings/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ use metaslang_cst::cursor::Cursor;
use metaslang_cst::kinds::KindTypes;
use metaslang_graph_builder::ast::File;
use metaslang_graph_builder::functions::Functions;
use resolver::Resolver;
use resolver::{ResolveOptions, Resolver};
use semver::Version;
use stack_graphs::graph::StackGraph;

Expand All @@ -32,10 +32,10 @@ pub(crate) struct DefinitionBindingInfo<KT: KindTypes + 'static> {
definiens: Option<Cursor<KT>>,
tag: Option<Tag>,
parents: Vec<GraphHandle>,
#[allow(dead_code)]
export_node: Option<GraphHandle>,
#[allow(dead_code)]
import_nodes: Vec<GraphHandle>,
extension_scope: Option<GraphHandle>,
inherit_extensions: bool,
}

pub(crate) struct ReferenceBindingInfo {
Expand All @@ -53,6 +53,7 @@ pub struct Bindings<KT: KindTypes + 'static> {
cursor_to_definitions: HashMap<CursorID, GraphHandle>,
cursor_to_references: HashMap<CursorID, GraphHandle>,
context: Option<GraphHandle>,
extension_hooks: HashSet<GraphHandle>,
}

pub enum FileDescriptor {
Expand Down Expand Up @@ -136,6 +137,7 @@ impl<KT: KindTypes + 'static> Bindings<KT> {
cursor_to_definitions: HashMap::new(),
cursor_to_references: HashMap::new(),
context: None,
extension_hooks: HashSet::new(),
}
}

Expand Down Expand Up @@ -187,6 +189,7 @@ impl<KT: KindTypes + 'static> Bindings<KT> {
self.definitions_info
.extend(result.definitions_info.drain());
self.references_info.extend(result.references_info.drain());
self.extension_hooks.extend(result.extension_hooks.drain());

result
}
Expand Down Expand Up @@ -258,18 +261,13 @@ impl<KT: KindTypes + 'static> Bindings<KT> {
// cannot be resolved at this point?
self.to_reference(*handle)
.unwrap()
.jump_to_definition()
.non_recursive_resolve()
.ok()
}
})
.collect()
}

pub fn lookup_definition_by_name(&self, name: &str) -> Option<Definition<'_, KT>> {
self.all_definitions()
.find(|definition| definition.get_cursor().unwrap().node().unparse() == name)
}

pub fn get_context(&self) -> Option<Definition<'_, KT>> {
self.context.and_then(|handle| self.to_definition(handle))
}
Expand Down Expand Up @@ -338,6 +336,10 @@ impl<KT: KindTypes + 'static> Bindings<KT> {
}
results
}

pub(crate) fn is_extension_hook(&self, node_handle: GraphHandle) -> bool {
self.extension_hooks.contains(&node_handle)
}
}

struct DisplayCursor<'a, KT: KindTypes + 'static> {
Expand Down Expand Up @@ -401,6 +403,20 @@ impl<'a, KT: KindTypes + 'static> Definition<'a, KT> {
.unwrap_or_default()
}

pub(crate) fn get_extension_scope(&self) -> Option<GraphHandle> {
self.owner
.definitions_info
.get(&self.handle)
.and_then(|info| info.extension_scope)
}

pub(crate) fn inherit_extensions(&self) -> bool {
self.owner
.definitions_info
.get(&self.handle)
.map_or(false, |info| info.inherit_extensions)
}

pub fn to_handle(self) -> DefinitionHandle {
DefinitionHandle(self.handle)
}
Expand Down Expand Up @@ -471,12 +487,20 @@ impl<'a, KT: KindTypes + 'static> Reference<'a, KT> {
.expect("Reference does not have a valid file descriptor")
}

pub fn jump_to_definition(&self) -> Result<Definition<'a, KT>, ResolutionError<'a, KT>> {
Resolver::build_for(self).first()
pub fn resolve_definition(&self) -> Result<Definition<'a, KT>, ResolutionError<'a, KT>> {
Resolver::build_for(self, ResolveOptions::Full).first()
}

pub fn definitions(&self) -> Vec<Definition<'a, KT>> {
Resolver::build_for(self).all()
Resolver::build_for(self, ResolveOptions::Full).all()
}

pub(crate) fn non_recursive_resolve(
&self,
) -> Result<Definition<'a, KT>, ResolutionError<'a, KT>> {
// This was likely originated from a full resolution call, so cut
// recursion here by restricting the resolution algorithm.
Resolver::build_for(self, ResolveOptions::NonRecursive).first()
}

pub(crate) fn has_tag(&self, tag: Tag) -> bool {
Expand Down
Loading
Loading