Json-ws and the results

Siren Platform/Investigate Tech Support Community
1

Hi All,
I have a ElasticSearch pipeline on demo articles.csv using the “json-ws” service to query a spacy api and return a json array of entities (like below). Is there best practice method from within the pipeline to further break the response object out like siren-nlp and have classes for each type (i.e., response_text.match.org.keyword)? So when I build the data model and I can build relationships off these classes? Thanks! Jeff

"response_text":  [
   {
      "start": 46,
      "end": 51,
      "text": "Apple",
      "type": "ORG"
      },
  {
    "start": 52,
   "end": 62,
    "text": "Microsoft",
    "type": "ORG"
   },
  <keeps going>
]
2

Hi Jeff,

If you add the following script processor to your pipeline after the json-ws call:

{
  "script": {
    "source": "def cats = new HashMap(); for (item in ctx.response_text) { def catArray = cats.getOrDefault(item.type, []);  catArray.add(item); cats.put(item.type, catArray) } ctx.categorized_response_text = cats;"
  }
}

you will get a new field like this:

"categorized_response_text": {
    "ORG": [
      {
        "start": 46,
        "end": 51,
        "text": "Apple",
        "type": "ORG"
      }, 
      ...

such that you can use the categorized_response_text.ORG.text field in a join.

Please let us know if that works for you.

3 Likes