Release - Federate 38.0 for 8.18.0 & 7.17.28

Announcements
1

We’re happy to announce the release of Siren Federate version 38.0 for Elasticsearch 8.18.0 & Elasticsearch 7.17.28. This release includes the following features, improvements and bug fixes:

New features

  • Set Adaptive Query Planner (AQP) as a default planner.
  • Introduced a Federate transport layer to detect rolling upgrades and support retro-compatible requests during the upgrade process.
    • Introduced a Federate Transport Version.
    • Enriched Request and Response objects with the Federate Transport Version of the sending and receiving nodes.
  • Implemented an input swap optimization for multi-conditional and spatial index joins.
  • Added join capabilities to KNN (K-Nearest Neighbors) Elasticsearch Query.
  • Added preliminary support for Search Guard FLX.

Improvements

  • Upgraded to to Elasticsearch 8.18.0 & 7.17.28
  • Aggregated shard-level operations into a single request per-node (i.e. leverage TransportBroadcastByNodeAction).
  • Improved the physical data signature and DataId in AQP requests.
  • Reduced the amount of data shared if the debug is disabled.
  • Enhanced memory error messages.
  • Optimized lock operations in LRU Cache.
  • Removed the usage of Class.forName for obfuscated classes.
  • Improved resource cleanup when a node becomes isolated.
  • Protected cached nested joins from eviction.
  • Improved the performance of the fingerprint computation.

Deprecations

  • Deprecated the Static Query Planner. Note: In versions 38.0 or later, the Adaptive Query Planner (AQP) is the default planner.
    • The federate.planner.type settings has been changed to “adaptive” by default. If any issues arise, it can be set to "static" and Siren should be contacted for troubleshooting.

Bug fixes

  • Reintroduced the json-ws processor, which was previously removed as part of the connector’s deprecation.
  • Fixed projection issues with floating-point values in nested joins.
  • Fixed pagination of join queries when the join was cached.
  • Fixed index ordering to ensure deterministic request cache keys.
  • Fixed a fingerprint computation issue that ignored the segment life cycle.
  • Fixed incorrect behaviour in Boolean queries using the minimum_should_match parameter on AQP requests.
  • Fixed an issue retrieving debug responses when the track_total_hits parameter was disabled.
  • Fixed a caching issue in Point-In-Time requests.
  • Fixed a bug with cached join queries using function_score in combination with the min_score parameter.
  • Fixed an issue with cached queries using runtime fields.

Breaking change

  • This version introduced a Federate transport layer to allow retro-compatibility in case of rolling upgrade for the following versions. This version is not compatible with previous Federate versions during a rolling upgrade.

Download it here .