Merge branch 'develop' into spring-5.3

.github/workflows/README.md

@@ -22,7 +22,7 @@ The `publish-cws-image` job is the CD component of the workflow, triggered upon
 
 - [**Services**](https://docs.github.com/en/actions/using-containerized-services/about-service-containers):
   - MariaDB
-    - Image: mariadb:10.3
+    - Image: mariadb:10.6
     - Ports: 3306:3306
 - [**checkout**](https://github.com/marketplace/actions/checkout): This action checks out the repository under `$GITHUB_WORKSPACE`, so the workflow can access it.
 - Set up JDK 11:

.github/workflows/camunda.yml

@@ -21,7 +21,7 @@ jobs:
 
     services:
       mdb103:
-        image: mariadb:10.3
+        image: mariadb:10.6
         ports:
           - 3306:3306
         env:
@@ -46,6 +46,13 @@ jobs:
           chmod +x generate-certs.sh
           ./generate-certs.sh
 
+      - name: Set up Keystore storepass
+        run: |
+          mkdir ~/.cws/
+          chmod 700 ~/.cws/
+          echo ${{ secrets.KEYSTORE_PASSWORD }} > ~/.cws/creds
+          chmod 600 ~/.cws/creds
+
       - name: Download Logstash
         uses: carlosperate/download-file-action@v1
         with:
@@ -144,7 +151,7 @@ jobs:
 
     services:
       mdb103:
-        image: mariadb:10.3
+        image: mariadb:10.6
         ports:
           - 3306:3306
         env:
@@ -169,6 +176,13 @@ jobs:
           chmod +x generate-certs.sh
           ./generate-certs.sh
 
+      - name: Set up Keystore storepass
+        run: |
+          mkdir ~/.cws/
+          chmod 700 ~/.cws/
+          echo ${{ secrets.KEYSTORE_PASSWORD }} > ~/.cws/creds
+          chmod 600 ~/.cws/creds
+
       - name: Download Logstash
         uses: carlosperate/download-file-action@v1
         with:

.github/workflows/ldap.yml

@@ -21,7 +21,7 @@ jobs:
 
     services:
       mdb103:
-        image: mariadb:10.3
+        image: mariadb:10.6
         ports:
           - 3306:3306
         env:
@@ -46,6 +46,13 @@ jobs:
           chmod +x generate-certs.sh
           ./generate-certs.sh
 
+      - name: Set up Keystore storepass
+        run: |
+          mkdir ~/.cws/
+          chmod 700 ~/.cws/
+          echo ${{ secrets.KEYSTORE_PASSWORD }} > ~/.cws/creds
+          chmod 600 ~/.cws/creds
+
       - name: Download Logstash
         uses: carlosperate/download-file-action@v1
         with:

.gitignore

@@ -34,4 +34,4 @@ install/logging/logstash-*.zip
 /jacoco-reports
 /test-screenshots
 
-*.cnf
+*.cnf

README.md

@@ -21,7 +21,7 @@ See the [wiki](https://github.com/NASA-AMMOS/common-workflow-service/wiki) for m
   - For Home-brew users:
     - Install Maven using: `brew install maven`
     - Verify installation using: `mvn -v`
-- [**Docker**](https://docs.docker.com/get-docker/): Used to run external Elasticsearch, and create and configure mariaDB database container
+- [**Docker**](https://docs.docker.com/get-docker/): Used to run external Elasticsearch, and create and configure MariaDB database container
   - Recommended minimum system requirements from Docker Resources window:
       - CPUs: 5
       - Memory: 14.00 GB
@@ -34,7 +34,7 @@ See the [wiki](https://github.com/NASA-AMMOS/common-workflow-service/wiki) for m
 - **Logstash 8.8.0+**: Download Logstash for your platform. Uncompress it (only if it is a .tar.gz) and then ZIP back it up with the filename 'logstash-8.8.0.zip' and place in `install/logging/`. This is a temporary workaround while we clean up our installation process. You can find the zip download [here](https://www.elastic.co/downloads/logstash).
 - **Elasticsearch 8.8.0+**: CWS requires an externally-configured elasticsearch cluster to be set up. You can use an SSL Secure Elasticsearch with or without authentication, or an Insecure HTTP Elasticsearch.
   - The "Elasticsearch Setup" instruction below provides a contained Dockerized way of running Elasticsearch. This serves as an alternative to installing Elasticsearch.
-- Tomcat **keystore and truststore files** (needed for CWS web console to work properly):
+- Tomcat **keystore, truststore, storepass files** (needed for CWS web console to work properly). To generate an open-source **.keystore** and **cws_truststore.jks** use the script `./generate-certs.sh` [here](https://github.com/NASA-AMMOS/common-workflow-service/tree/develop/cws-certs)
     - You will need to add your own Tomcat keystore file to this path: `install/.keystore`
     - You will need to add your own truststore file to this path: `install/tomcat_lib/cws_truststore.jks`
     - See: https://tomcat.apache.org/tomcat-9.0-doc/ssl-howto.html
@@ -44,33 +44,37 @@ See the [wiki](https://github.com/NASA-AMMOS/common-workflow-service/wiki) for m
       - Check the exact version installed using `/usr/libexec/java_home -V`
       - Add to your Shell startup (e.g. .zprofile): `export JAVA_HOME=$(/usr/libexec/java_home -v X.X.X)`
         - Replace the X.X.X version above with the OpenJDK 17 output from the `/usr/libexec/java_home -V` command.
+  - **Store Your Keystore Password**: You will need to add your own creds file, which carries the keystore password, to this path: `~/.cws/creds`
+    - Set the permissions for the **~/.cws/** directory and **creds** file as Owner-Only.
+      - **~/.cws/** directory: `chmod 700 ~/.cws/`
+      - **~/.cws/creds** file: `chmod 600 ~/.cws/creds`
 
 
 ### **Development Environment Configuration**
 
 ### _MariaDB Setup_
 
-Generate mariaDB Docker Container and Create Database Instance for CWS:
+Generate MariaDB Docker Container and Create Database Instance for CWS:
 ```
-docker run -d -p 3306:3306 -e MYSQL_DATABASE=__DB_NAME__ -e MYSQL_ROOT_PASSWORD=__ROOT_PW__ -e TZ=America/Los_Angeles --name mdb103 mariadb:10.3
+docker run -d -p 3306:3306 -e MYSQL_DATABASE=__DB_NAME__ -e MYSQL_ROOT_PASSWORD=__ROOT_PW__ -e TZ=America/Los_Angeles --name mdb106 mariadb:10.6
 ```
 
 Replace `__DB_NAME__` with your desired database name. <br />
 Replace `__ROOT_PW__` with your desired password.
 
 `__DB_NAME__` and `__ROOT_PW__` must match parameters set in script file: `<personal-dev>.sh`
 
-Directly access mariaDB with:
+Directly access MariaDB with:
 
 ```
 mysql -h 127.0.0.1 -u root -p
 ```
 Enter the password above when prompted.
 
-_CWS must have been built, in this case using a build script, in order to directly access mariaDB with the MySQL monitor, as the build
+_CWS must have been built, in this case using a build script, in order to directly access MariaDB with the MySQL monitor, as the build
 script contains required information to access the database. See the **Building CWS** section for an example build script._
 
-_Make sure `cws_dev` database in created mariaDB instance before moving forward to build CWS_
+_Make sure `cws_dev` database in created MariaDB instance before moving forward to build CWS_
 
 ### _Elasticsearch Setup_
 Open new Shell terminal designated for running ElasticSearch.
@@ -180,6 +184,8 @@ Start `test.sh` script by running:
 
 This will produce jacoco reports with code coverage measurements.
 
+## [Adaptation Setup Guide](cws-adaptation/README.md)
+
 # Contributing
 
 Please see our [contribution guidelines](https://github.com/NASA-AMMOS/common-workflow-service/blob/main/CONTRIBUTING.md).

ci/ci.sh

@@ -62,6 +62,7 @@ function worker_conf_data() {
   fi
 }
 
+check_java_requirements
 
 cat > ci_console.conf.template <<- EOF
 cam_server_url=NA
@@ -122,6 +123,9 @@ default_cws_auth_scheme=CAMUNDA
 default_history_level=full
 default_shutdown_port=38005
 metrics_publishing_interval=10
+camunda_version=${CAMUNDA_VER}
+java_version=${JAVA_HOME_VERSION}
+java_home=${JAVA_HOME}
 EOF
 
 
@@ -180,6 +184,9 @@ aws_cloudwatch_endpoint=monitoring.us-west-1.amazonaws.com
 default_history_level=full
 default_shutdown_port=38005
 metrics_publishing_interval=10
+camunda_version=${CAMUNDA_VER}
+java_version=${JAVA_HOME_VERSION}
+java_home=${JAVA_HOME}
 EOF
 
 

create_server_dist.sh

@@ -69,6 +69,7 @@ cp ${INSTALL_DIR}/tomcat_lib/css-jaas.cfg                     ${CONFIG_TEMPLATES
 cp ${INSTALL_DIR}/tomcat_bin/setenv.sh                        ${CONFIG_TEMPLATES_DIR}/tomcat_bin
 cp ${INSTALL_DIR}/tomcat_conf/bpm-platform.xml                ${CONFIG_TEMPLATES_DIR}/tomcat_conf
 cp ${INSTALL_DIR}/tomcat_conf/server.xml                      ${CONFIG_TEMPLATES_DIR}/tomcat_conf
+cp ${INSTALL_DIR}/tomcat_conf/server_adaptation.xml           ${CONFIG_TEMPLATES_DIR}/tomcat_conf
 cp ${INSTALL_DIR}/tomcat_conf/web.xml                         ${CONFIG_TEMPLATES_DIR}/tomcat_conf
 cp ${INSTALL_DIR}/tomcat_conf/ldap_plugin_bean.xml            ${CONFIG_TEMPLATES_DIR}/tomcat_conf
 cp ${INSTALL_DIR}/tomcat_conf/ldap_plugin_ref.xml             ${CONFIG_TEMPLATES_DIR}/tomcat_conf
@@ -186,6 +187,17 @@ awk 'NR==FNR { a[n++]=$0; next }
 
 cp ${INSTALL_DIR}/sql/core.afterstartup.sql.template           ${CWS}/sql/cws/core.afterstartup.sql
 
+if [[ -f ${INSTALL_DIR}/sql/adaptation.sql.template && -f ${INSTALL_DIR}/sql/adaptation_core.sql.template && -f ${INSTALL_DIR}/sql/adaptation_external.sql.template ]]; then
+    print "Building adaptation (external) sql..."
+    awk 'NR==FNR { a[n++]=$0; next }
+    /__CUSTOM_METHODS_JAVA__/ { for (i=0;i<n;++i) print a[i]; next }1' ${DIST}/snippets.java ${INSTALL_DIR}/sql/adaptation.sql.template > ${CWS}/sql/cws/adaptation.sql
+    awk 'NR==FNR { a[n++]=$0; next }
+    /__CUSTOM_METHODS_JAVA__/ { for (i=0;i<n;++i) print a[i]; next }1' ${DIST}/snippets.java ${INSTALL_DIR}/sql/adaptation_core.sql.template > ${CWS}/sql/cws/adaptation_core.sql
+    awk 'NR==FNR { a[n++]=$0; next }
+    /__CUSTOM_METHODS_JAVA__/ { for (i=0;i<n;++i) print a[i]; next }1' ${DIST}/snippets.java ${INSTALL_DIR}/sql/adaptation_external.sql.template > ${CWS}/sql/cws/adaptation_external.sql
+fi
+
+
 rm ${DIST}/snippets.java
 rm ${DIST}/snippets.java.bak
 

cws-adaptation/README.md

@@ -0,0 +1,54 @@
+## Adaptation Setup Guide
+
+CWS can be adapted and tailored for a mission. This can be done by modifying certain resources and source code of CWS-CORE.
+
+Create new functionalities by adding:
+
+* External Database Config
+* Custom Java code
+* Custom Rest Service API
+* Custom Initiators
+* etc...
+
+### Adaptation Database
+
+Adaptation Database feature lets you connect to a external database separate from that of the main `cws_dev` database.
+
+**(1)** You must first include the following SQL Template files to the `install/sql/` directory:
+* `install/sql/adaptation.sql.template`          **(Core Database: `cws_dev`)**
+* `install/sql/adaptation_core.sql.template`     **(Core Database: `cws_dev`)**
+* `install/sql/adaptation_external.sql.template` **(Adaptation Database: `cws_external_db`)**
+
+**(2)** Create your schema/tables within the added Adaptation SQL Template files.
+* Example (`adaptation_external.sql.template`):
+```
+CREATE TABLE IF NOT EXISTS `cws_external_table` (
+    ...
+    ...
+) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
+```
+
+**(3)** Write Custom Java Code.
+
+By referencing your adaptation tables, you can connect and communicate with your external database.
+
+Write Java custom code that extends from DbService and use the `jdbcTemplate` or `jdbcAdaptationTemplate` JDBC source to query the core and external databases.
+```
+jdbcAdaptationTemplate  (adaptation external database)
+jdbcTemplate            (core database)
+```
+
+### Adaptation Workers Modal
+
+Custom your Deployment page Workers view with add-ons and filters.
+
+The UI for the Worker listview is customizable with through the javascript `addAdaptationWorkersInfo` function.
+
+File: `cws-ui/src/main/webapp/js/adaptation-workers-modal.js`
+
+Write your custom code here:
+```
+function addAdaptationWorkersInfo(dataProcKey, listWorkers) {
+	return; 
+}
+```

cws-core/src/main/java/jpl/cws/core/db/DbService.java

@@ -9,6 +9,7 @@
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.beans.factory.annotation.Qualifier;
 import org.springframework.jdbc.core.JdbcTemplate;
 import org.springframework.jdbc.core.namedparam.MapSqlParameterSource;
 import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate;
@@ -22,9 +23,15 @@
 public class DbService {
 	private static final Logger log = LoggerFactory.getLogger(DbService.class);
 
-	@Autowired protected JdbcTemplate jdbcTemplate;
-	@Autowired protected NamedParameterJdbcTemplate namedJdbcTemplate;
-
+	@Autowired
+	protected JdbcTemplate jdbcTemplate;
+
+	@Autowired
+	protected NamedParameterJdbcTemplate namedJdbcTemplate;
+
+	@Autowired
+	protected JdbcTemplate jdbcAdaptationTemplate;
+
 	protected static final long SLOW_WARN_THRESHOLD = 100;
 	protected static final int IN_CLAUSE_MAX_ELEMENTS = 100;