Skip to content

[ti_anomali] Field data type conflicts between data streams #13585

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
andrewkroh opened this issue Apr 17, 2025 · 1 comment · May be fixed by #13909
Open

[ti_anomali] Field data type conflicts between data streams #13585

andrewkroh opened this issue Apr 17, 2025 · 1 comment · May be fixed by #13909
Assignees
@brijesh-elastic
Labels
bug Something isn't working, use only for issues Integration:ti_anomali Anomali Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] Team:Sit-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors]

Comments

@andrewkroh
Copy link
Member

There are a few fields declared with varying data types across the ti_anomali package. These can cause Kibana data view conflicts.

  • anomali.threatstream.confidence has multiple data types (long, short)
  • anomali.threatstream.import_session_id has multiple data types (keyword, long)
  • anomali.threatstream.trusted_circle_ids has multiple data types (keyword, long)
  • anomali.threatstream.update_id has multiple data types (keyword, long)
$ go run github.com/andrewkroh/[email protected] -a conflict -conflict.ignore-text-family -conflict.ignore-keyword-family -i  packages/ti_anomali packages/**/fields/*.yml
packages/ti_anomali/data_stream/intelligence/fields/fields.yml:10:7 anomali.threatstream.confidence has multiple data types (long, short) (conflict)
  packages/ti_anomali/data_stream/intelligence/fields/fields.yml:10:7 long (conflict)
  packages/ti_anomali/elasticsearch/transform/latest_intelligence/fields/fields.yml:10:7 long (conflict)
  packages/ti_anomali/data_stream/threatstream/fields/fields.yml:13:7 short (conflict)
  packages/ti_anomali/elasticsearch/transform/latest_ioc/fields/fields.yml:14:7 short (conflict)
packages/ti_anomali/data_stream/threatstream/fields/fields.yml:29:7 anomali.threatstream.import_session_id has multiple data types (keyword, long) (conflict)
  packages/ti_anomali/data_stream/threatstream/fields/fields.yml:29:7 keyword (conflict)
  packages/ti_anomali/elasticsearch/transform/latest_ioc/fields/fields.yml:30:7 keyword (conflict)
  packages/ti_anomali/data_stream/intelligence/fields/fields.yml:30:7 long (conflict)
  packages/ti_anomali/elasticsearch/transform/latest_intelligence/fields/fields.yml:30:7 long (conflict)
packages/ti_anomali/data_stream/threatstream/fields/fields.yml:76:7 anomali.threatstream.trusted_circle_ids has multiple data types (keyword, long) (conflict)
  packages/ti_anomali/data_stream/threatstream/fields/fields.yml:76:7 keyword (conflict)
  packages/ti_anomali/elasticsearch/transform/latest_ioc/fields/fields.yml:77:7 keyword (conflict)
  packages/ti_anomali/data_stream/intelligence/fields/fields.yml:120:7 long (conflict)
  packages/ti_anomali/elasticsearch/transform/latest_intelligence/fields/fields.yml:120:7 long (conflict)
packages/ti_anomali/data_stream/threatstream/fields/fields.yml:81:7 anomali.threatstream.update_id has multiple data types (keyword, long) (conflict)
  packages/ti_anomali/data_stream/threatstream/fields/fields.yml:81:7 keyword (conflict)
  packages/ti_anomali/elasticsearch/transform/latest_ioc/fields/fields.yml:82:7 keyword (conflict)
  packages/ti_anomali/data_stream/intelligence/fields/fields.yml:128:7 long (conflict)
  packages/ti_anomali/elasticsearch/transform/latest_intelligence/fields/fields.yml:128:7 long (conflict)
@andrewkroh andrewkroh added bug Something isn't working, use only for issues Integration:ti_anomali Anomali labels Apr 17, 2025
@jamiehynds jamiehynds added the Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] label Apr 18, 2025
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@jamiehynds jamiehynds added the Team:Sit-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors] label Apr 18, 2025
@brijesh-elastic brijesh-elastic linked a pull request May 14, 2025 that will close this issue
Open
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@brijesh-elastic brijesh-elastic

Labels
bug Something isn't working, use only for issues Integration:ti_anomali Anomali Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] Team:Sit-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors]
Projects
None yet
Milestone
No milestone
4 participants
@andrewkroh @elasticmachine @jamiehynds @brijesh-elastic