Release - Federate 8.11.1-33.0

Announcements
1

We’re happy to announce today the release of Siren Federate version 33.0 for Elasticsearch version 8.11.1. This release includes the following improvements:

Improvements:

  • Upgraded to Elasticsearch 8.11.1
  • Added a circuit-breaker mechanism that makes multi-conditional joins fail fast if the INDEX_JOIN strategy needs to perform many index lookups that require a long processing time.
    • The maximum number of allowed lookups is controlled by a new dynamic setting siren.planner.index_join.max_lookups
    • The default value for this setting is 20,000.
  • Updated the off-heap memory settings siren.memory.{root,job,task}.limit to be dynamic, facilitating the setup and configuration of Federate clusters.
  • Added cache capabilities for AQP requests.
  • Enabled Point-in-Time (PIT) for inner join requests.
  • Added the keep-alive parameter for PIT requests.
  • Introduced a resource manager for better handling of the resource life-cycle.
  • Improved documentation with a table of operator compatibility for every data type.
  • Improved documentation on joins with nested fields.

Deprecations:

  • Deprecated the Ingestion API.
    • All requests under /_siren/connector/ingestion are marked as deprecated.
    • The following settings are also marked as deprecated:
      siren.connector.ingestions.index, siren.connector.ingestion.bulk_size, siren.connector.ingestion.fetch_size, siren.connector.ingestion.index_timeout,
  • Deprecated the Job API.
    • All requests under /_siren/connector/jobs are marked as deprecated.

Bug fixes:

  • Fixed an issue when the join query is not specified. A match_all query is used by default.
  • Fixed a query cache miss when changing the order of conditions in a multi-conditional join request.

Known issues:

  • Opening a PIT for remote indices is not supported.
  • Search slicing does not work with PIT.

Download it here .