Siren Federate - Elasticsearch Java API

Hi All,

Is there any example for a siren federate query using Elasticsearch Java API?

Thanks,
Michael

Hi Michael,

First thanks for evaluating the Federate plugin. We really appreciate.

At the moment the Federate java API is still internal and obfuscated. We are currently working on exposing the client methods and builders. They should be available in the next release. Then it will be possible to use the high level client.

The best option for now is to use the Elastic Search low level client. If you are not yet familiar with it you will find the documentation here:
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/6.5/java-rest-low.html

Let’s take the join request that you can find on our documentation:
https://docs.support.siren.io/10.2.1/platform/en/siren-federate/query-dsl/join-query.html

GET /siren/index1/_search
{
  "query" : {
    "join" : {
      "type": "HASH_JOIN",
      "indices" : ["index2"],
      "types" : ["type"],
      "on" : ["foreign_key", "id"],
      "request" : {
        "query" : {
          "terms" : {
            "tag" : [ "aaa" ]
          }
        }
      }
    }
  }
}

Here is how you would build the request using the RestClient:

import org.apache.http.entity.ContentType;
import org.apache.http.nio.entity.NStringEntity;
import org.apache.http.util.EntityUtils;
import org.elasticsearch.client.Request;
import org.elasticsearch.client.Response;
import org.elasticsearch.client.RestClient;

...

RestClient restClient = RestClient.builder(
     new HttpHost("localhost", 9200, "http"),
     new HttpHost("localhost", 9201, "http")).build();
 
  final Request request = new Request(
      "GET",
      "/siren/index1/_search");
  request.setEntity(new NStringEntity(
      "{\"query\" : {\"join\" : {\"type\": \"HASH_JOIN\"," +
          "\"indices\" : [\"index2\"],\"types\" : [\"type\"]," +
          "\"on\" : [\"foreign_key\", \"id\"],\"request\" : {" +
          "\"query\" : {\"terms\" : { \"tag\" : [ \"aaa\" ]" +
          "}}}}}}",
      ContentType.APPLICATION_JSON));

  Response response = restClient.performRequest(request);
  restClient.close();

Let us know if it works.

1 Like

Thanks!

Will try it and let you know.

When are you expecting to have the next release?

Michale

Great! The next upcoming release (in probably one or two weeks) will be the v10.2.2, but it will not yet contain the high level client. I will update this ticket as soon as I know when this feature is planned and in which version you will find it.

You may also track the following discussion to be notified about the upcoming releases.
https://community.siren.io/c/announcements

Hi Emmanuel,

I think I’m using a later version of Elasticsearch Java SDK.
I changed your code a bit:

RestClient restClient = RestClient.builder(
            new HttpHost("<<elasticsearchHost>>", 9200, "http")).build();

    JoinQueryBuilder joinQueryBuilder = new JoinQueryBuilder()
            .joinIndex("<<indextToJoin>>")
            .joinField("<<joinId>>")
            .joinField("<<foreignJoinId>>");

    SearchQuery searchQuery = new NativeSearchQueryBuilder()
            .withQuery(joinQueryBuilder)
            .withSearchType(SearchType.QUERY_THEN_FETCH)
            .withIndices("<<indexName>>")
            .withTypes("<<indextType>>")
            .build();

    try {
        HttpEntity httpEntity = new StringEntity(searchQuery.getQuery().toString());
        String method = "GET";
        String endpoint = "/siren/<<indexName>>/_search";
        BasicHeader header = new BasicHeader("Content-Type", "application/json");
        Response response = restClient.performRequest(method, endpoint, new HashMap<String, String>(), httpEntity, header);
        RequestLine requestLine = response.getRequestLine();
        HttpHost host = response.getHost();
        int statusCode = response.getStatusLine().getStatusCode();
        Header[] headers = response.getHeaders();
        String responseBody = EntityUtils.toString(response.getEntity());
        
        logger.info("<<<<< \n" + responseBody + "\n>>>>>>");
        restClient.close();
    } catch (Throwable th) {
        logger.error("ERROR", th);
    }

and this is JoinQueryBuilder:

public class JoinQueryBuilder extends AbstractQueryBuilder<JoinQueryBuilder> {

private static ArrayList<String> joinQueryBuilderIndices = new ArrayList<String>();
private static ArrayList<String> joinQueryBuilderjoinFields = new ArrayList<String>();

public JoinQueryBuilder() {
}

public static final String NAME = "join";

public JoinQueryBuilder joinIndex(String index) {
    if (index == null) {
        throw new IllegalArgumentException("inner bool query clause cannot be null");
    }
    joinQueryBuilderIndices.add(index);
    return this;
}

public JoinQueryBuilder joinField(String field) {
    if (field == null) {
        throw new IllegalArgumentException("inner bool query clause cannot be null");
    }
    joinQueryBuilderjoinFields.add(field);
    return this;
}

@Override
protected void doWriteTo(StreamOutput out) throws IOException {
    out.writeStringList(joinQueryBuilderIndices);
    out.writeStringList(joinQueryBuilderjoinFields);
}

@Override
protected void doXContent(XContentBuilder builder, Params params) throws IOException {
    builder.startObject("query");
    builder.startObject(NAME);
    //indices
    builder.startArray("indices")
            .value(joinQueryBuilderIndices.get(0))
            .endArray();
    //on
    builder.startArray("on")
            .value(this.joinQueryBuilderjoinFields.get(0))
            .value(this.joinQueryBuilderjoinFields.get(1))
            .endArray();

    builder.startObject("request")
            .startObject("query")
            .startObject("match_all")
            .endObject()
            .endObject()
            .endObject();

    builder.endObject();
    builder.endObject();

}

@Override
protected Query doToQuery(QueryShardContext context) throws IOException {
    BooleanQuery.Builder booleanQueryBuilder = new BooleanQuery.Builder();
    return booleanQueryBuilder.build();
}

@Override
public String getWriteableName() {
    return NAME;
}

@Override
protected boolean doEquals(JoinQueryBuilder other) {
    return Objects.equals(joinQueryBuilderIndices, other.joinQueryBuilderIndices)
            && Objects.equals(joinQueryBuilderjoinFields, other.joinQueryBuilderjoinFields);
}

@Override
protected int doHashCode() {
    return Objects.hash(Arrays.hashCode(joinQueryBuilderIndices.toArray()), Arrays.hashCode(joinQueryBuilderjoinFields.toArray()));
}
}

That worked! but it’s still a bit bumming that all the parsing, error handling and the high-level Spring Data interface are not available with the low-level client.

Cheers,
Michael

2 Likes