Merge branch 'community-backports-jdk21u-master' into mandrel/23.1

Author: jerboaa

BUILDING.md

@@ -0,0 +1,17 @@
+# Building GraalVM Community Edition for JDK 21
+
+To build GraalVM Community Edition for JDK 21, you need to have an OpenJDK 21 build, along with static libraries, installed on your system. You can use the [AdoptOpenJDK](https://adoptopenjdk.net/) distribution or any other OpenJDK 21 build as long as it ships static libraries.
+
+After you have the required OpenJDK 21 build, follow these steps to build GraalVM Community Edition for JDK 21:
+
+```bash
+export JAVA_HOME=/path/to/your/jdk21
+git clone https://github.com/graalvm/graalvm-community-jdk21u graalvm-jdk21
+export MX_VERSION=$(jq -r .mx_version graalvm-jdk21/common.json)
+git clone https://github.com/graalvm/mx.git mx --branch $(MX_VERSION)
+cd graalvm-jdk21
+../mx/mx --primary-suite vm --env ce build
+```
+
+Once the build completes successfully, you can find the GraalVM Community Edition distribution in the directory where the `sdk/latest_graalvm_home` link points to.
+

CONTRIBUTING.md

@@ -1,6 +1,48 @@
-GraalVM welcomes contributors to the core platform and projects that extend that
-platform. There have been significant contributions from both industry
-and academia so far and we thank you for considering to contribute your changes!
+# Contributing to GraalVM Community Edition for JDK 21 maintenance repository
 
-- Learn [how to become a GraalVM contributor](https://www.graalvm.org/community/contributors/).
-- Subscribe and post to [[email protected]](https://oss.oracle.com/mailman/listinfo/graalvm-dev) for questions related to working with the sources or extending the GraalVM ecosystem by creating new languages, tools, or embeddings.
+The maintenance repository for GraalVM Community Edition for JDK 21 welcomes contributions from the community that satisfy the following criteria:
+
+1. The contributor has signed the [Oracle Contributor Agreement](https://oca.opensource.oracle.com/). See https://www.graalvm.org/community/contributors/ for more information.
+2. The contribution contains a) replicas of existing commits applied to the `master` branch in the `oracle/graal` repository or b) patches specific to the maintained version. Such commits are intended to fix or improve existing features, must always keep release stability in mind, and ensure compatibility with published public APIs.
+3. The contribution is considered safe and beneficial for the GraalVM Community Edition for JDK 21. Ideally the contribution should be solving a reported issue and not be proactive.
+
+Please note that feature requests and enhancements are not accepted in this repository. If you have a feature request or enhancement, please open an issue in [GraalVM's main repository](https://github.com/oracle/graal).
+
+## How to Contribute
+
+### Reporting Bugs
+
+Open a [GitHub issue](https://github.com/graalvm/graalvm-community-jdk21u/issues/new?template=1_community_bug.yml) filling in the template with the necessary information.
+
+### Fixing Bugs
+
+1. Make sure an [issue report](https://github.com/graalvm/graalvm-community-jdk21u/issues/new?template=1_community_bug.yml) or [backport request](https://github.com/graalvm/graalvm-community-jdk21u/issues/new?template=0_backport_request.yml) exists in this repository explaining what the problem is and why it should be considered for fixing in the maintenance repository.
+2. Make sure you have signed the [Oracle Contributor Agreement](https://oca.opensource.oracle.com/).
+3. Create a pull request with the fix and reference the issue or backport request in the description.
+
+#### Building GraalVM Community Edition for JDK 21
+
+Please refer to the [BUILDING.md](BUILDING.md) file for instructions on how to build GraalVM Community Edition for JDK 21.
+
+#### Running Style Checks
+
+Before submitting a pull request, please run the following command to ensure that your changes comply with the GraalVM Community Edition code style:
+
+```bash
+../mx/mx --primary-suite vm --env ce checkstyle
+
+ECLIPSE_TAR=eclipse.tar.gz
+ECLIPSE_ORG_VERSION=$(jq -r '.eclipse.short_version' common.json)
+ECLIPSE_ORG_TIMESTAMP=$(jq -r '.eclipse.timestamp' common.json)
+wget --no-verbose https://archive.eclipse.org/eclipse/downloads/drops4/R-${ECLIPSE_ORG_VERSION}-${ECLIPSE_ORG_TIMESTAMP}/eclipse-SDK-${ECLIPSE_ORG_VERSION}-linux-gtk-x86_64.tar.gz -O $ECLIPSE_TAR
+tar -xzf ${ECLIPSE_TAR}
+ECLIPSE_EXE=${PWD}/eclipse/eclipse
+../mx/mx --primary-suite vm --env ce eclipseformat --eclipse-exe $ECLIPSE_EXE
+```
+
+> [!NOTE]
+> The link used in the `wget` command above is only valid for linux x86_64. If you are using a different platform, please adjust the link accordingly.
+
+#### Backporting Fixes
+
+When backporting bug fixes use `git cherry-pick -x` to reference the original commit in the commit message. Additionally, please reference the upstream pull request that your changes are backporting. If the backport is partial explain which parts are being backported and why. Similarly in case of conflicts explain why they are happening and how they are resolved.

MAINTAINING.md

@@ -1,15 +1,29 @@
-# GraalVM Community Backports Repositories Guidelines
+# Merging Pull Requests
 
-1. Commits to a GraalVM Community Edition (CE) release branch or maintenance repository are either a) replicas of existing commits applied to the `master` branch in the `oracle/graal` repository or b) specific to the maintained version. Such commits are intended to fix or improve existing features, must always keep release stability in mind, and ensure compatibility with published public APIs.
+Merging a pull request requires the approval from at least one [maintainer](#current-maintainers). The maintainer should ensure that the pull request satisfies the criteria listed in the [CONTRIBUTING.md](CONTRIBUTING.md) file.
 
-2. Release branches shall be maintained by Oracle in `oracle/graal` for six months per GraalVM CE release. During that time, the community can request backports through pull requests against the corresponding release branch in `oracle/graal`.
+To assist maintainers, especially in areas they might not be familiar with, a number of [reviewers](#current-reviewers) are assigned to the repository.
 
-3. After six months and only for Java LTS releases that the community wants to support long-term (e.g., 21), Oracle shall create a dedicated repository (e.g., `graalvm/graalvm-community-jdk21u`) to maintain community backports and version-specific bug fixes.
+# Creating a New Release
 
-4. Each repository shall be managed by one maintainer, appointed by Oracle, with the permission to approve and merge pull requests.
+1. Switch the `release` field in all `suite.py` files to `True` and create a pull request.
+2. Once the pull request is merged, create two new tags as follows:
+   ```bash
+   git tag -s vm-<version> -m "Community source release of GraalVM <version> for JDK 21"
+   git tag -s jdk-<jdk-version> -m "Community source release of GraalVM <version> for JDK 21"
+   ```
+3. Bump the version and switch the `release` field back to `False`  in all `suite.py` files and create a new pull request.
 
-5. The repositories shall not be used to ship new releases of GraalVM CE or any other GraalVM distribution, only to maintain source code. Distributors should use these repositories as their primary source for creating GraalVM CE-based builds.
+Between step 1 and 3 no new pull requests should be merged.
 
-6. The creation of maintenance repositories for other GraalVM projects (e.g., LabsJDK or GraalJS) can also be requested by the community after six months. The same rules apply.
+# Current Maintainers
 
-7. Maven artifacts under the `org.graalvm` group that are used by both GraalVM CE-based distributions and Oracle GraalVM will be maintained by Oracle.
+* [Foivos Zakkak](https://github.com/zakkak/) (Lead maintainer)
+* [Karm Babacek](https://github.com/Karm/)
+* [Severin Gehwolf](https://github.com/jerboaa/)
+
+# Current Reviewers
+
+* [Boris Ulasevich](https://github.com/bulasevich)
+* [Cesar Soares](https://github.com/JohnTortugo)
+* [Volker Simonis](https://github.com/simonis/)

README.md

@@ -1,19 +1,27 @@
 [![GraalVM](.github/assets/logo_320x64.svg)][website]
 
-[![GraalVM downloads][badge-dl]][downloads] [![GraalVM docs][badge-docs]][docs] [![GraalVM on Slack][badge-slack]][slack] [![GraalVM on Twitter][badge-twitter]][twitter] [![GraalVM on YouTube][badge-yt]][youtube]  [![GraalVM Gate][badge-gate]][gate] [![License][badge-license]](#license)
+[![GraalVM Gate][badge-gate]][gate] [![GraalVM docs][badge-docs]][docs] [![GraalVM on Slack][badge-slack]][slack] [![GraalVM on Twitter][badge-twitter]][twitter] [![GraalVM on YouTube][badge-yt]][youtube]  [![License][badge-license]](#license)
 
-GraalVM is a high-performance JDK distribution designed to accelerate the execution of applications written in Java and other JVM languages along with support for JavaScript, Ruby, Python, and a number of other popular languages.
+Welcome to the **maintenance repository** of GraalVM Community Edition for JDK 21. This is a **source-only** repository maintained by the GraalVM community.
+
+GraalVM is a high-performance JDK distribution designed to accelerate the execution of applications written in Java and other JVM languages along with support for JavaScript, Ruby, Python, and a number of other popular languages. To get the latest GraalVM version visit  the [GraalVM website][downloads].
+
+Please refer to:
+
+1. [BUILDING.md](BUILDING.md) for instructions on how to build GraalVM Community Edition for JDK 21.
+2. [CONTRIBUTING.md](CONTRIBUTING.md) for information on how to contribute to this repository.
+3. [MAINTAINING.md](MAINTAINING.md) for information on how to maintain this repository.
 
 The project website at [https://www.graalvm.org/][website] describes how to [get started][getting-started], how to [stay connected][community], and how to [contribute][contributors].
 
 ## Documentation
 
-Please refer to the [GraalVM website for documentation][docs]. You can find most of the documentation sources in the [_docs/_](docs/) directory in the same hierarchy as displayed on the website. Additional documentation including developer instructions for individual components can be found in corresponding _docs/_ sub-directories. The documentation for the Truffle framework, for example, is in [_truffle/docs/_](truffle/docs/). This also applies to languages, tools, and other components maintained in [related repositories](#related-repositories).
+Please refer to the [GraalVM website for documentation][docs]. You can find most of the documentation sources in the [_docs/_](docs/) directory in the same hierarchy as displayed on the website. Additional documentation including developer instructions for individual components can be found in corresponding _docs/_ sub-directories. The documentation for the Truffle framework, for example, is in [_truffle/docs/_](truffle/docs/).
 
 ## Get Support
 
-* Open a [GitHub issue][issues] for bug reports, questions, or requests for enhancements.
-* Join the [GraalVM Slack][slack] to connect with the community and the GraalVM team.
+* Open a [GitHub issue][issues] for backport requests or bug reports.
+* Join the `#community-lts` channel in the [GraalVM Slack workspace][slack] to connect with the community.
 * Report a security vulnerability according to the [Reporting Vulnerabilities guide][reporting-vulnerabilities].
 
 ## Repository Structure
@@ -37,22 +45,6 @@ Directory | Description
 [`vm/`](vm/) | Components for building GraalVM distributions.
 [`wasm/`](wasm/) | [GraalWasm][reference-graalwasm], an engine for running WebAssembly programs on GraalVM.
 
-## Related Repositories
-
-GraalVM provides additional languages, tools, and other components developed in related repositories. These are:
-
-Name         | Description
------------- | -------------
-[FastR] | Implementation of the R language.
-[GraalJS] | Implementation of JavaScript and Node.js.
-[GraalPy] | Implementation of the Python language.
-[GraalVM Demos][graalvm-demos] | Several example applications illustrating GraalVM capabilities.
-[Native Build Tools][native-build-tools] | Build tool plugins for GraalVM Native Image.
-[SimpleLanguage] | A simple example language built with the Truffle framework.
-[SimpleTool] | A simple example tool built with the Truffle framework. 
-[TruffleRuby] | Implementation of the Ruby language.
-[VS Code Extensions][vscode-extensions] | VS Code extensions for GraalVM.
-
 ## License
 
 GraalVM Community Edition is open source and distributed under [version 2 of the GNU General Public License with the “Classpath” Exception](LICENSE), which are the same terms as for Java. The licenses of the individual GraalVM components are generally derivative of the license of a particular language (see the table below).
@@ -65,39 +57,31 @@ Component(s) | License
 [Sulong](sulong/LICENSE) | 3-clause BSD
 
 
-[badge-dl]: https://img.shields.io/badge/download-latest-blue
 [badge-docs]: https://img.shields.io/badge/docs-read-green
-[badge-gate]: https://github.com/oracle/graal/actions/workflows/main.yml/badge.svg
+[badge-gate]: https://github.com/graalvm/graalvm-community-jdk21u/actions/workflows/main.yml/badge.svg
 [badge-license]: https://img.shields.io/badge/license-GPLv2+CE-green
 [badge-slack]: https://img.shields.io/badge/Slack-join-active?logo=slack
 [badge-twitter]: https://img.shields.io/badge/Twitter-@graalvm-active?logo=twitter
 [badge-yt]: https://img.shields.io/badge/YouTube-subscribe-active?logo=youtube
 [community]: https://www.graalvm.org/community/
 [contributors]: https://www.graalvm.org/community/contributors/
-[docs]: https://www.graalvm.org/latest/docs/
+[docs]: https://www.graalvm.org/jdk21/docs/
 [downloads]: https://www.graalvm.org/downloads/
-[fastr]: https://github.com/oracle/fastr
-[gate]: https://github.com/oracle/graal/actions/workflows/main.yml
-[getting-started]: https://www.graalvm.org/latest/docs/getting-started/
-[graaljs]: https://github.com/oracle/graaljs
-[graalpy]: https://github.com/oracle/graalpython
+[gate]: https://github.com/graalvm/graalvm-community-jdk21u/actions/workflows/main.yml
+[getting-started]: https://www.graalvm.org/jdk21/docs/getting-started/
 [graalvm-demos]: https://github.com/graalvm/graalvm-demos
 [graalvm-sdk]: https://www.graalvm.org/sdk/javadoc/
-[igv]: https://www.graalvm.org/latest/tools/igv/
-[issues]: https://github.com/oracle/graal/issues
-[java-on-truffle]: https://www.graalvm.org/latest/reference-manual/java-on-truffle/
+[igv]: https://www.graalvm.org/jdk21/tools/igv/
+[issues]: https://github.com/graalvm/graalvm-community-jdk21u/issues
+[java-on-truffle]: https://www.graalvm.org/jdk21/reference-manual/java-on-truffle/
 [native-build-tools]: https://github.com/graalvm/native-build-tools
 [native-image]: https://www.graalvm.org/native-image/
-[reference-compiler]: https://www.graalvm.org/latest/reference-manual/java/compiler/
-[reference-graalwasm]: https://www.graalvm.org/latest/reference-manual/wasm/
-[reference-sulong]: https://www.graalvm.org/latest/reference-manual/llvm/
+[reference-compiler]: https://www.graalvm.org/jdk21/reference-manual/java/compiler/
+[reference-graalwasm]: https://www.graalvm.org/jdk21/reference-manual/wasm/
+[reference-sulong]: https://www.graalvm.org/jdk21/reference-manual/llvm/
 [reporting-vulnerabilities]: https://www.oracle.com/corporate/security-practices/assurance/vulnerability/reporting.html
-[simplelanguage]: https://github.com/graalvm/simplelanguage
-[simpletool]: https://github.com/graalvm/simpletool
 [slack]: https://www.graalvm.org/slack-invitation/
 [truffle]: https://www.graalvm.org/graalvm-as-a-platform/language-implementation-framework/
-[truffleruby]: https://github.com/oracle/truffleruby
 [twitter]: https://twitter.com/graalvm
-[vscode-extensions]: https://github.com/graalvm/vscode-extensions
 [website]: https://www.graalvm.org/
 [youtube]: https://www.youtube.com/graalvm

ci/ci_common/bench-common.libsonnet

@@ -31,6 +31,17 @@
       else true
   },
 
+  # max number of threads to use for benchmarking in general
+  # the goal being to limit parallelism on very large servers which may not be respresentative of real-world scenarios
+  bench_max_threads:: {
+   restrict_threads:: 36
+  },
+
+  bench_no_thread_cap:: {
+    restrict_threads:: null,
+    should_use_hwloc:: false
+  },
+
   bench_hw:: {
     _bench_machine:: {
       targets+: ["bench"],
@@ -42,15 +53,16 @@
       numa_nodes:: [],
       is_numa:: std.length(self.numa_nodes) > 0,
       num_threads:: error "num_threads must bet set!",
+      hyperthreading:: true,
       threads_per_node:: if self.is_numa then self.num_threads / std.length(self.numa_nodes) else self.num_threads,
     },
 
-    x52:: common.linux_amd64 + self._bench_machine + {
-      machine_name:: "x52",
-      capabilities+: ["no_frequency_scaling", "tmpfs25g"],
+    e3:: common.linux_amd64 + self._bench_machine + {
+      machine_name:: "e3",
+      capabilities: ["e3", "tmpfs25g", "linux", "amd64"],
       numa_nodes:: [0, 1],
-      default_numa_node:: 0,
-      num_threads:: 72
+      default_numa_node:: 1,
+      num_threads:: 256
     },
     x82:: common.linux_amd64 + self._bench_machine + {
       machine_name:: "x82",
@@ -62,31 +74,29 @@
     xgene3:: common.linux_aarch64 + self._bench_machine + {
       machine_name:: "xgene3",
       capabilities+: [],
-      num_threads:: 32
+      num_threads:: 32,
+      hyperthreading:: false
     },
     a12c:: common.linux_aarch64 + self._bench_machine + {
       machine_name:: "a12c",
       capabilities+: ["no_frequency_scaling", "tmpfs25g"],
       numa_nodes:: [0, 1],
       default_numa_node:: 0,
-      num_threads:: 160
+      num_threads:: 160,
+      hyperthreading:: false
     }
   },
 
-  hwlocIfNuma(numa, cmd, node=0)::
-    if numa then
+  hwloc_cmd(cmd, num_threads, node, hyperthreading, max_threads_per_node)::
+    if num_threads == null then
       ["hwloc-bind", "--cpubind", "node:"+node, "--membind", "node:"+node, "--"] + cmd
     else
-      cmd,
-
-  parallelHwloc(cmd_node0, cmd_node1)::
-    // Returns a list of commands that will run cmd_nod0 on NUMA node 0
-    // concurrently with cmd_node1 on NUMA node 1 and then wait for both to complete.
-    [
-      $.hwlocIfNuma(true, cmd_node0, node=0) + ["&"],
-      $.hwlocIfNuma(true, cmd_node1, node=1) + ["&"],
-      ["wait"]
-    ],
+      local threads = if num_threads != null then num_threads else max_threads_per_node;
+      assert if hyperthreading then threads % 2 == 0 else true: "It is required to bind to an even number of threads on hyperthreaded machines. Got requested "+threads+" threads";
+      assert threads <= max_threads_per_node: "Benchmarking must run on a single NUMA node for stability reasons. Got requested "+threads+" threads but the machine has only "+max_threads_per_node+" threads per node";      local cores = if hyperthreading then "0-"+((threads/2)-1)+".pu:0-1" else "0-"+(threads-1)+".pu:0";
+      local cpu_bind = if hyperthreading then "node:"+node+".core:"+cores else "node:"+node+".core:"+cores+".pu:0";
+      ["hwloc-bind", "--cpubind", cpu_bind, "--membind", "node:"+node, "--"] + cmd
+  ,
 
   // building block used to generate fork builds
   many_forks_benchmarking:: common.build_base + {