Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gh-476] Sanitise HCL variables before storing on job submission #24423

Merged
merged 4 commits into from
Nov 22, 2024

Conversation

Juanadelacuesta
Copy link
Member

@Juanadelacuesta Juanadelacuesta commented Nov 11, 2024

Description

Currently Nomad only handles HCL variables with new lines, any other non alphanumeric character is left untouched and stored unescaped, which can cause errors while re starting a stopped job, particularly from the UI.

it fixes #476

Testing & Reproduction steps

Links

Contributor Checklist

  • Changelog Entry If this PR changes user-facing behavior, please generate and add a
    changelog entry using the make cl command.
  • Testing Please add tests to cover any new functionality or to demonstrate bug fixes and
    ensure regressions will be caught.
  • Documentation If the change impacts user-facing functionality such as the CLI, API, UI,
    and job configuration, please update the Nomad website documentation to reflect this. Refer to
    the website README for docs guidelines. Please also consider whether the
    change requires notes within the upgrade guide.

Reviewer Checklist

  • Backport Labels Please add the correct backport labels as described by the internal
    backporting document.
  • Commit Type Ensure the correct merge method is selected which should be "squash and merge"
    in the majority of situations. The main exceptions are long-lived feature branches or merges where
    history should be preserved.
  • Enterprise PRs If this is an enterprise only PR, please add any required changelog entry
    within the public repository.

@Juanadelacuesta Juanadelacuesta changed the title func: User url rules to scape non alphanumeric values in hcl variables [gh-476] Sanitise HCL variables before storing them to avoid format errors on resubmissions Nov 12, 2024
@Juanadelacuesta Juanadelacuesta changed the title [gh-476] Sanitise HCL variables before storing them to avoid format errors on resubmissions [gh-476] Sanitise HCL variables before storing on job submission Nov 12, 2024
@Juanadelacuesta Juanadelacuesta added the backport/1.9.x backport to 1.9.x release line label Nov 12, 2024
@Juanadelacuesta Juanadelacuesta marked this pull request as ready for review November 12, 2024 09:31
Copy link
Member

@tgross tgross left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

api/jobs.go Show resolved Hide resolved
Copy link
Member

@jrasell jrasell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@philrenaud
Copy link
Contributor

Testing this out made me remember we have two different variable-related properties in Job submission data: VariableFlags, which this PR addresses and are passed via CLI props, and Variables, which are what get entered via -var-file, etc.:

nomad/api/jobs.go

Lines 978 to 984 in e206993

// VariableFlags contains the CLI "-var" flag arguments as submitted with the
// job (hcl2 only).
VariableFlags map[string]string
// Variables contains the opaque variables configuration as coming from
// a var-file or the WebUI variables input (hcl2 only).
Variables string

I think that VariableFlags are affected here, but Variables are not. I'm not sure if the bug in question could show up in that format, but wanted to mention it.

=========================

All that aside, I'm also noticing some perhaps erroneous character additions. Here are screenshots showing VariableFlags and output for a job where a cli-provided multi-line variable has a + character added:

on main:
Screenshot 2024-11-18 at 2 44 53 PM

on f-NET-11421-vars:
Screenshot 2024-11-18 at 2 42 37 PM

command to run was:

nomad job run -var meta_string="line1 \
\
\
\
line2" ../nomad-job-specs/unescaped_var_test.nomad.hcl

Note the + added between "line1" and "line2"

Copy link
Member

@tgross tgross left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Juanadelacuesta Juanadelacuesta merged commit c21dfdb into main Nov 22, 2024
35 of 36 checks passed
@Juanadelacuesta Juanadelacuesta deleted the f-NET-11421-vars branch November 22, 2024 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport/1.9.x backport to 1.9.x release line
Projects
None yet
Development

Successfully merging this pull request may close these issues.

False drift with nomad variables containing newlines
4 participants