Allow more customization of query parameters for backstage source

[kunickiaj]

Page size is currently hardcoded at 100

catalog-importer/source/source_backstage.go

Lines 49 to 52 in 89b8c58

	var (
	limit = 100
	offset = 0
	)

Would also be helpful to be able to specify a filter query to avoid loading ALL entities in the catalog -- only the ones that need to be sync'ed to incident.io. For example we have ~20k resource entities that don't need to be loaded (or ~200 pages with the hardcoded page size of 100).

[louisheath]

Hey @kunickiaj, apologies for the delay in response here.

So that I understand the exact desired mechanism here, are you looking for the ability to specify backstage's filter query parameter?

/entities?filter=kind=user,metadata.namespace=default&filter=kind=group,spec.type

[kunickiaj]

That and switch to by-query rather than the deprecated entities endpoint.

This would allow for much more efficient filtering as we can select only relevant entities or even fields as well as being able to adjust the page size.

[kunickiaj]

Any update on this? our catalog keeps growing and this is becoming a real pain point -- not to mention the currently hardcoded endpoint is deprecated.