Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add @SingleThread annotation to dissect processor #5463

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

graytaylor0
Copy link
Member

@graytaylor0 graytaylor0 commented Feb 26, 2025

Description

Dissect processor is not thread safe. Added "@SingleThread" annotation

Issues Resolved

Resolves #5462

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dlvenable
Copy link
Member

It seems that the main reason this is not thread safe is because the Dissector class mutates its own fields.

private Map<String, SkipField> skipFieldMap;
private Map<String, NormalField> normalFieldMap;
private Map<String, IndirectField> indirectFieldMap;
private Map<String, List<AppendField>> unAppendedFieldsMap;
private final FieldHelper fieldHelper = new FieldHelper();
private final LinkedList<Field> fieldsList = new LinkedList<>();
private final LinkedList<Delimiter> delimiterList = new LinkedList<>();

Might there be other bugs with this behavior?

@graytaylor0
Copy link
Member Author

graytaylor0 commented Feb 26, 2025

It seems that the main reason this is not thread safe is because the Dissector class mutates its own fields.

private Map<String, SkipField> skipFieldMap;
private Map<String, NormalField> normalFieldMap;
private Map<String, IndirectField> indirectFieldMap;
private Map<String, List<AppendField>> unAppendedFieldsMap;
private final FieldHelper fieldHelper = new FieldHelper();
private final LinkedList<Field> fieldsList = new LinkedList<>();
private final LinkedList<Delimiter> delimiterList = new LinkedList<>();

Might there be other bugs with this behavior?

I'm not sure. There could be, but so far this processor works fine with a single thread. Don't you think this is a good short term fix though? Seems like the refactor to avoid this mutation may be larger

@dlvenable
Copy link
Member

It seems that the main reason this is not thread safe is because the Dissector class mutates its own fields.

private Map<String, SkipField> skipFieldMap;
private Map<String, NormalField> normalFieldMap;
private Map<String, IndirectField> indirectFieldMap;
private Map<String, List<AppendField>> unAppendedFieldsMap;
private final FieldHelper fieldHelper = new FieldHelper();
private final LinkedList<Field> fieldsList = new LinkedList<>();
private final LinkedList<Delimiter> delimiterList = new LinkedList<>();

Might there be other bugs with this behavior?

I'm not sure. There could be, but so far this processor works fine with a single thread. Don't you think this is a good short term fix though? Seems like the refactor to avoid this mutation may be larger

Yea, that may be a lot.

Copy link
Collaborator

@oeyh oeyh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@graytaylor0 Good find! I remember seeing a customer issue where dissect processor returns weird results a few times out of like a million processed events. This might have been the root cause.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Dissect processor is not thread safe
3 participants