DO NOT MERGE: v1.12 branch for comparison to master #202
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nickrobinson251
force-pushed
the
v1.12.0-DEV+RAI
branch
4 times, most recently
from
966538b to
9da665d
Compare
d-netto
force-pushed
the
v1.12.0-DEV+RAI
branch
from
9da665d to
48a5871
Compare
nickrobinson251
force-pushed
the
v1.12.0-DEV+RAI
branch
from
48a5871 to
5974904
Compare
DelveCI
force-pushed
the
v1.12.0-DEV+RAI
branch
5 times, most recently
from
50775dd to
fd6934f
Compare
I couldn't find `time_ns` when I was looking for it, nice to make clear the "monotonic" clock is also available in Base. (cherry picked from commit a52de83)
DelveCI
force-pushed
the
v1.12.0-DEV+RAI
branch
from
fd6934f to
1cff7d7
Compare
…uliaLang#57241) The `fork()` we do here relies on `SIGCHLD` to make sure that we don't race against the child. This is easy to see in an embedding application that dynamically links `libjulia`: ```c int main(int argc, char *argv[]) { signal(SIGCHLD, SIG_IGN); void *handle = dlopen("path/to/libjulia.so", RTLD_LAZY); return 0; } ``` Without this change, this fails with an error message: ``` Error during libstdcxxprobe in parent process: waitpid: No child processes ``` Resolves JuliaLang#57240 (cherry picked from commit daf865e)
Restores JuliaLang#57035, undo JuliaLang#57089 for non-FreeBSD. While I suggested doing this change for all platforms, I forgot that means non-FreeBSD platforms become vulnerable again to the very deadlock problems that JuliaLang#57035 was required to prevent. That fix seems to not be viable on FreeBSD due to known libc implementation problems on that platform. However, upon closer inspection of the questionable design implementation decisions they seem to have made here, the platform is likely not currently vulnerable to this libunwind bug in the first place: https://github.com/lattera/freebsd/blob/master/libexec/rtld-elf/rtld_lock.c#L120 (cherry picked from commit 2f0a523)
…57211) It looks like these methods were just missed while overloading for BufferStream. There's also `readbytes!` where the current implementation will fallback to the `LibuvStream` implementation that is currently not threadsafe. What's the best approach there since the implementation is quite a bit more involved? Just duplicate the code but for BufferStream? Should we take the BufferStream lock and invoke the LibuvStream method? Open to ideas there. Also open to suggestions for having tests here? Not easy to simulate the data race of writing and calling readavailable. The fix here will unblock JuliaWeb/HTTP.jl#1213 (I'll probably do some compat shim there until this is fully released). Thanks to @oscardssmith for rubber ducking this issue with me. Probably most helpfully reviewed by @vtjnash. --------- Co-authored-by: Jameson Nash <[email protected]> (cherry picked from commit ffc96bc)
(cherry picked from commit 99fd5d9)
…ce (JuliaLang#57226) (cherry picked from commit ca7cf30)
This is the final PR in the binding partitions series (modulo bugs and tweaks), i.e. it closes JuliaLang#54654 and thus closes JuliaLang#40399, which was the original design sketch. This thus activates the full designed semantics for binding partitions, in particular allowing safe replacement of const bindings. It in particular allows struct redefinitions. This thus closes timholy/Revise.jl#18 and also closes JuliaLang#38584. The biggest semantic change here is probably that this gets rid of the notion of "resolvedness" of a binding. Previously, a lot of the behavior of our implementation depended on when bindings were "resolved", which could happen at basically an arbitrary point (in the compiler, in REPL completion, in a different thread), making a lot of the semantics around bindings ill- or at least implementation-defined. There are several related issues in the bugtracker, so this closes JuliaLang#14055 closes JuliaLang#44604 closes JuliaLang#46354 closes JuliaLang#30277 It is also the last step to close JuliaLang#24569. It also supports bindings for undef->defined transitions and thus closes JuliaLang#53958 closes JuliaLang#54733 - however, this is not activated yet for performance reasons and may need some further optimization. Since resolvedness no longer exists, we need to replace it with some hopefully more well-defined semantics. I will describe the semantics below, but before I do I will make two notes: 1. There are a number of cases where these semantics will behave slightly differently than the old semantics absent some other task going around resolving random bindings. 2. The new behavior (except for the replacement stuff) was generally permissible under the old semantics if the bindings happened to be resolved at the right time. With all that said, there are essentially three "strengths" of bindings: 1. Implicit Bindings: Anything implicitly obtained from `using Mod`, "no binding", plus slightly more exotic corner cases around conflicts 2. Weakly declared bindings: Declared using `global sym` and nothing else 3. Strongly declared bindings: Declared using `global sym::T`, `const sym=val`, `import Mod: sym`, `using Mod: sym` or as an implicit strong global declaration in `sym=val`, where `sym` is known to be global (either by being at toplevle or as `global sym=val` inside a function). In general, you always allowed to syntactically replace a weaker binding by a stronger one (although the runtime permits arbitrary binding deletion now, this is just a syntactic constraint to catch errors). Second, any implicit binding can be replaced by other implicit bindings as the result of changing the `using`'ed module. And lastly, any constants may be replaced by any other constants (irrespective of type). We do not currently allow replacing globals, but may consider changing that in 1.13. This is mostly how things used to work, as well in the absence of any stray external binding resolutions. The most prominent difference is probably this one: ``` set_foo!() = global foo = 1 ``` In the above terminology, this now always declares a "strongly declared binding", whereas before it declared a "weakly declared binding" that would become strongly declared on first write to the global (unless of course somebody had created a different strongly declared global in the meantime). To see the difference, this is now disallowed: ``` julia> set_foo!() = global foo = 1 set_foo! (generic function with 1 method) julia> const foo = 1 ERROR: cannot declare Main.foo constant; it was already declared global Stacktrace: [1] top-level scope @ REPL[2]:1 ``` Before it would depend on the order of binding resolution (although it just crashes on current master for some reason - whoops, probably my fault). Another major change is the ambiguousness of imports. In: ``` module M1; export x; x=1; end module M2; export x; x=2; end using .M1, .M2 ``` the binding `Main.x` is now always ambiguous and will throw on access. Before which binding you get, would depend on resolution order. To choose one, use an explicit import (which was the behavior you would previously get if neither binding was resolved before both imports). (cherry picked from commit 888cf03)
…#57273) Fixes JuliaLang#56864. Ref https://github.com/JuliaLang/julia/blob/99fd5d9a92190e826bc462d5739e7be948a3bf44/stdlib/REPL/src/LineEdit.jl#L504 --------- Co-authored-by: Ian Butterworth <[email protected]> (cherry picked from commit f0446c6)
- Update `JuliaSyntax.jl` to v1.0.1 - Revert workaround changes, use original test case `n37134` Fix JuliaLang#57223 (cherry picked from commit 97c920d)
DelveCI
force-pushed
the
v1.12.0-DEV+RAI
branch
from
1cff7d7 to
1e6e20d
Compare
Addresses review comment in JuliaLang#57212 (comment). The key is that the hand-off of responsibility for verification between the loading code and the ordinary backedge mechanism happens under the world counter lock to ensure synchronization. (cherry picked from commit 34aceb5)
Updating mmtk-julia version to include mmtk/mmtk-julia#228 and fix the MMTk CI. I've also changed the allocation profiler tests to skip all tests instead of just a few since I've seen some spurious errors - they should all be related though, we need to make sure the profiler accounts for fastpath allocation (see JuliaLang#57103) This should fix JuliaLang#57306. (cherry picked from commit 72f8a10)
Similar to JuliaLang#57229, this commit ensures that `Compiler.finish!` properly synchronizes the operations to set `max_world` for cached `CodeInstance`s by holding the world counter lock. Previously, `Compiler.finish!` relied on a narrow timing window to avoid race conditions, which was not a robust approach in a concurrent execution environment. This change ensures that `Compiler.finish!` holds the appropriate lock (via `jl_promote_ci_to_current`). (cherry picked from commit 4ebb50b)
(cherry picked from commit 132057c)
Fixes JuliaLang#54560 (comment) (cherry picked from commit 79e98e3)
This adds a warning for the auto-import of types cases (JuliaLang#25744) that we have long considered a bit of a bug, but didn't want to change because it is too breaking. The reason to do it now is that the binding rework has made this case more problematic (see JuliaLang#57290). To summarize, the question is what happens when the compiler sees `f(x) = ...` and `f` is currently and implicitly imported binding. There are two options: 1. We add a method to the generic function referred to by `f`, or 2. We create a new generic function `f` in the current module. Historically, case 1 has the additional complication that this error'd unless `f` is a type. It is my impression that a lot of existing code did not have a particularly good understanding of the resolved-ness dependence of this behavior. However, because case 1 errors for generic functions, it appears that existing code generally expects case 2. On the other hand, for types, there is existing code in both directions (JuliaLang#57290 is an example of case 2; see JuliaLang#57302 for examples of case 1). That said, case 1 is more common (because types tend to be resolved because they're used in signatures at toplevel). Thus, to retain compatibility, the current behavior on master (where resolvedness is no longer available) is that we always choose case 2 for functions and case 1 for types. This inconsistency is unfortunate, but I tried resolving this in either way (making all situations case 1 or all case 2) and the result was too breaking. Nevertheless, it is problematic that there is existing code that expects case 2 beavior for types and we should help users to know what the correct way to fix it is. The proposed resolution is thus: 1. Retain case 1 behavior for types 2. Make it a warning to use, encouraging people to explicitly import, since we generally consider the JuliaLang#25744 case a bug. Example: ``` julia> module Foo String(i::Int) = i end WARNING: Type Core.String was auto-`import`ed in `Foo`. NOTE: This behavior is deprecated and may change in future Julia versions. NOTE: This behavior may have differed in Julia versions prior to 1.12 depending on binding resolution. Hint: To retain the current behavior, add an explicit `import Core: String` in Foo. Hint: To create a new generic function of the same name use `function String end`. Main.Foo ``` (cherry picked from commit 8c62f42)
introduced in JuliaLang#56144 --------- Co-authored-by: Mosè Giordano <[email protected]> (cherry picked from commit dd13878)
DelveCI
force-pushed
the
v1.12.0-DEV+RAI
branch
from
fbb7d02 to
f36b557
Compare
) ## Context We send a signal 15 to shutdown our servers. We noticed that some of our servers that receive the termination signal are segfaulting in GC, which leads to false alarms in our internal monitors that track GC-related crashes. ## Hypothesis We suspect this pathological case may be happening: - Process receives signal 15, which is captured by the signal listener thread. - Signal listener initiates process' teardown (e.g. through `raise`). - IIRC such operation is not atomic in Linux, i.e. the kernel will gradually kill the threads, but it's possible for us to spent a few ms in a state where part of the threads in the system are alive, and part have already been killed (this point needs some confirmation). - With part of the process alive, and part of the process dead, we try to enter a GC, see a bunch of Julia data structures in an intermediate/corrupted state, which leads us to crash when running the GC. ## Mitigation Since our main goal is to get rid of the GC crashes that happen around server shutdown, we believe that it would be sufficient to just prevent the last bullet point. I.e. we prevent the system from even running a GC when we're about to kill the process, and we wait for any ongoing GC to finish. Co-debugged with @kpamnany. (cherry picked from commit e1e3a46)
This is mostly for GPUCompiler (cherry picked from commit e78cf6c)
DelveCI
force-pushed
the
v1.12.0-DEV+RAI
branch
from
f36b557 to
4abc802
Compare
…aLang#57868) Should prevent some invalidation in the sysimage. (cherry picked from commit f49f46d)
Should make the sysimage less vulnerable to invalidation. As far as I understand this change can't break any code, because: * the macro definition requires `AbstractString` * the way Julia macros work, the argument is either a literal or an `Expr` * `String` is the only `AbstractString` with literals (cherry picked from commit 87f4d3a)
Should make the sysimage less vulnerable to invalidation. As far as I understand this change can't break any code, because: * the macro definition requires `AbstractString` * the way Julia macros work, the argument is either a literal or an `Expr` * `String` is the only `AbstractString` with literals (cherry picked from commit 7acb2a6)
…6924) When SSL_CERT_FILE or SSL_CERT_DIR is set, it is [impossible to set this location](https://github.com/libgit2/libgit2/blob/4dcdb64c6844d76776745cdc25071a72c1af84d6/src/libgit2/settings.c#L206-L222) in LibGit2_jll on Apple and Windows because [it isn't built with support for that](https://github.com/JuliaPackaging/Yggdrasil/blob/7123a60a68102ba6cd953e13a4e45845dc37fd82/L/LibGit2/build_tarballs.jl#L67). Until now we've errored out with a message telling users to set JULIA_SSL_CA_ROOTS_PATH to an empty string, which is a somewhat problematic workaround because the Windows environment variables UI doesn't allow empty values, and [setting it to an empty string from PowerShell unsets it](https://discourse.julialang.org/t/how-to-fix-ssl-cert-issues-in-pkg/115495/7?u=visr). This PR changes the behavior to allow this expected error. Variables like SSL_CERT_FILE are for instance [set by the Conda OpenSSL package on environment activation](https://github.com/conda-forge/openssl-feedstock/blob/83b5e2a793bc95d19e6cc2d9d28068f1a6ff6b79/recipe/activate-win.ps1) used by e.g. Python, ensuring many people cannot use Pkg operations that use LibGit2, like `dev Example`, `add Example#master`. See more user reports [on Discourse](https://discourse.julialang.org/search?q=JULIA_SSL_CA_ROOTS_PATH). Together with JuliaLang/NetworkOptions.jl#37 this should improve the experience of users trying out Julia from a Conda environment. This should also be fine to backport. (cherry picked from commit 7fa969a)
Addresses part of JuliaLang#57547. Currently, we only show the "all-underscore variable name not allowed" error when the offending symbol is in value position according to `compile`, but an earlier pass would keep them as raw symbols regardless. This led to raw symbols making their way out of lowering in an edge case. This change: Reject all-underscore variables unless they're being written to. Also, improve an error message slighly. (cherry picked from commit 3360a44)
Fixes a regression introduced in JuliaLang#57755 seen on PkgEval in JuliaLang#57755 (comment) (cherry picked from commit 1d2e165)
Fixes JuliaLang#57872 (cherry picked from commit 400d0b1)
This reverts commit f15d417.
…s from packages succeed. TODO: remove this once alpha/beta is released
Prevent transparent huge pages (THP) overallocating pysical memory. Co-authored-by: Adnan Alhomssi <[email protected]>
Prepend `[signal (X) ]thread (Y) ` to each backtrace line that is displayed. Co-authored-by: Diogo Netto <[email protected]>
Also show the signal number when we have it.
DelveCI
force-pushed
the
v1.12.0-DEV+RAI
branch
from
4abc802 to
e8c0996
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See also RAICode PR https://github.com/RelationalAI/raicode/pull/22602/commits