Siren doesn't found all fields of index

Hi i am trying to create and see some dashboards in siren, but siren just found some of the fields of the index (105/3622). In addition, most of the fields it finds appear the following text: “Field not stored in documents” and cannot be used. I would like to upload other photo but i cant because im new user.

Thanks in advance.

here is the other photo:

Hi Adrian,

It seems the values in the fields are not constant means the value in XYZ field are not constant across the table like the format of the value.
Also do you see any error while ingesting the data when you did it at this page:

Manu Agarwal

the data was ingested with logstash to elasticsearh, and it seems to be constant, also i can’t ingest the data on that page because i ingest it from a .pcap

PD: i can see all data in kibana

Hi Adrian,

Can you share the mappings for the index?
Also is it possible for you to share the sample document?

Manu Agarwal

the documents size is 7.4GB so i can give you the page where i dowloaded:


Hi Adrian,

It will be worth if you can send me the single document from this as it will be more worth to test on single document rather than download all from it.

Also, Siren will also not ingest the fields if the field data is empty, and as elastiflow is another tool I can not assure whether the elastiflow template will work in Siren.


Hello Adrian22,
So, you have ingested some data into Elasticsearch using Logstash but not all fields seem to be detected in Data Model of Investigate whereas Kibana is showing them all.
Is Kibana and Investigate using the same Elasticsearch?

Can you:-

  • Go to Dev Tools.
  • Fire this query: GET elasti*
  • Provide us the response.
1 Like

i can’t attach files here so i only can give you this screenshot and tell you there are 145.000 lines of fields.

Hello @Adrian22 ,
the reason why in Investigate you do not see the full list of possible Elastiflow fields is because Elastiflow provides a pre-defined index patterns that is loaded during the setup (

Without a pre-built index pattern, both Investigate and Kibana will be able to see only the fields that are effectively present in the mappings when the index pattern is created or refreshed, as returned by the field_caps API. This API cannot see the fields declared in dynamic templates if there are no documents that contain them.

In order to see all the possible Elastiflow fields in Investigate, it would be necessary to convert the files in into a format suitable for Investigate, which is something we could consider in the future.

Meanwhile, you could still create visualizations from scratch on the fields that are available in your indices, but reusing the standard elastiflow patterns / dashboards / visualizations it won’t be possible without conversion.