Params for scripted fields, `visState` aggregation logic, and dynamic field values based on user input

Suppose I have an index quantum_cats with one numerical field called “probability” which is a double between 0 and 1 for each record. (This is intended to inform the logic for a scripted field detailed below). The entity table created by this index is used in the dashboard “Quantum Cats” consisting of two visualizations: a scripted panel and a bar chart histogram.

The scripted panel consists of a checkbox, a submit button, and an interactive record table consisting of a single column, “Cat Fate” created by a scripted field “cat_fate”. When the checkbox is unchecked, the table shows each row for “Cat Fate” as “dead, alive”. When the checkbox is checked and the submit button clicked, the “cat_fate” values update on each record to be either “dead” or “alive” depending on the “probability” value from each record. When the checkbox is unchecked and the submit button clicked, the “cat_fate” values go back to “dead, alive”.

The histogram bar chart is intended to show the number of “dead” cats and the number of “alive” cats based on the value of “cat_fate” as described above. Though it is possible to aggregate on scripted fields, it appears that it is only possible if the scripted field is created in the “Scripted fields” tab for the entity table. Note that I want my bar chart to be separate from the scripted panel. I do not want to have to rely on customizing my own bar chart as a scripted panel with Apex charts.

My “cat_fate” field is created in my scripted panel using “script_fields” in an sirenapi.es.search() response. I need the scripted field to depend on if the checkbox is checked (and submit button clicked), and I need the histogram (and other possible visualizations) to be able to aggregate on this scripted field. In general, the dashboard needs to respond to the state of the values of “cat_fate”. However, it seems the only option for aggregating on a scripted field comes from creating a scripted field in the “Scripted fields” tab, and it only accepts Painless. I don’t know how to pass it a params field that is based on my checkbox state. Is this even possible? One option I thought of was creating multiple scripted fields and swap out which field is used for the “Cat Fate” column in my table based on the checkbox/submit state; however, it seems I can only base my histogram aggregation on one scripted field at a time it seems. Can I edit the aggregations in the visState of my bar chart to include logic for which scripted field to use based on the checkbox/submit state?

Moreover, I want “cat_fate” to be a searchable thing within the dashboard. I know that scripted fields cannot be searched, but there has to be a way around this. The only other alternative I could think of is making “cat_fate” a regular field on the index that is updated using the updateByQuery() function. However, this would cause concurrency issues.

Any help on the above issues would be greatly appreciated. Basically, I want my data (that is, how the data presents itself within a field) to be more dynamic based on user inputs without running into concurrency issues.

Hi Trevor,

When creating a scripted field at runtime using the Siren API, the challenge arises because standard visualizations do not recognize values from scripts they are not aware of. Visualizations will only work with fields that have been explicitly defined in the data model.

If the goal is to use values from a scripted field via the Siren API, a corresponding visualization must also be created using the same script, as the standard visualizations cannot inherently understand these runtime scripts

It’s important to note that Elasticsearch and Siren are built to operate on indexed data. Searching on data that isn’t indexed goes against how these systems fundamentally work. Indexing is essential for fast, efficient searches and is core to how Elasticsearch and Siren manage data.

If the goal is to search unindexed data, it would either need to be indexed or brought in through a different process, such as ingesting it into the cluster and indexing it on the fly. Without indexing, the system won’t be able to perform the search operations you’re looking for.

Moreover I will prefer to connect with our sales team sales@siren.io as this is something our services can team help you by better understand the case and provide you the solution.

Regards
Manu

Manu,

I appreciate your response. If I need further assistance, I’ll be sure to follow up with you and the Siren team. In the meantime, I will be looking into an alternative approach that may achieve the functionality I am trying to obtain.

Best Regards,

TC