Skip to content

Virtual Fields

Virtual Fields allow you to derive new values from your data in real time

One of the most powerful features of Axiom are Virtual Fields. With Virtual Fields, there is no need to do any up-front planning of how to structure or transform your data. Instead, send your data as-is and then use Virtual Fields to manipulate your data in real-time during queries.

The feature is also known has "derviced columns/fields", however Axiom's Virtual Fields have some unique properties that make them much more powerful.

In this guide, you'll be introduced to Virtual Fields, their features, how to manage them, and how to get the best out of them.

Creating a Virtual Field

Once a dataset is selected in either the Analytics or Stream views, clicking on the virtual fields icon in the toolbar will open the virtual field slide-out that displays all the virtual fields for a dataset:

virtual fields tool button

virtual fields slideout

Click "Add Virtual Field" to open the virtual field editor dialog.

virtual fields editor

The editor dialog has the following components:

  • Name & Description - help your team understand what thie virtual field is doing
  • Alias - this is the name of the field when used in visualizations, filters, and group-bys
  • Expression - this the expression that is run on every event to derive the virtual field. The expression would produce some output (from a boolean check like the example, to strings, numbeers, objects or more)
  • Preview - use the preview window to test your virtual field expression against live data

The power of virtual fields is in letting you manipulate data 'on read' ineas of 'on write', allowing you and your team members to adjust and update virtual fields over time as well as easily add new ones without worrying that the data has already been indexed.

Usage

Visualizations

Virtual Fields will be available as parameters to visualizations but, as the type of a virtual field can be any of the supported types, it is important to make sure that you are using a virtual field that produces the correct type of argument.

Filters

Virtual Fields will be available in the filter menu and all filter options will be presented. It is important to ensure that you are using a supported filter operation for the type of result your virtual field produces.

Group By

Virtual Fields can be used for segmentation in the same way as any standard field.

Reference

Virtual Fields use a rich expression language that is easy to grasp but powerful in use. This section will

Examples

Literals

  • strings - single and double quotes are supported
  • numbers - 101, 101.1
  • booleans - true and false
  • arrays - ["one", "two", "three"]
  • maps - `{ region: "us-east-1" }
  • nil - nil

Arithmetic Operators

  • + - addition
  • - - subtraction
  • * - multiplication
  • / - division
  • % - modulus
  • ** - pow

Comparison Operators

  • == - equal
  • != - not equal
  • < - less than
  • > - greater than
  • <= - less than or equal to
  • >= - greater than or equal to

Logical Operators

  • and or &&
  • or or ||
  • not or !
  • success ? 'yes' : 'no' - ternary

String Operators

  • + - concatenation
  • matches - regex match
  • contains - string contains
  • startsWith - has prefix
  • endsWith - has suffix

Info

To test the negative case of not matching, wrap the operator in a not() operator:

not ("us-east-1" contains "us")

You must use parentesis because the unary operator not has precedence over the binary operator contains.

Numeric Operators

In addition to the arithmetic operators:

  • .. - numeric range

Example

age in 18..45

Tip

The range is inclusive: 1..3 == [1, 2, 3]

Membership Operators

  • in - contains
  • not in - does not contain

Example

Arrays: metadata.region in ["us-east-1", "us-east-2"]
Maps: 'region' in { region: 'us-east-1 } // true

Builtins

  • len - length of an array, map, or string
  • all - will return true if all element satisfies the predicate
  • none - will return true if all element does NOT satisfies the predicate
  • any - will return true if any element satisfies the predicate
  • one - will return true if exactly ONE element satisfies the predicate
  • filter - filter array by the predicate
  • map - map all items with the closure
  • count - returns number of elements what satisfies the predicate

Ensure all comments are less than 280 chars

all(comments, {.Size < 280})

Ensure there is exactly one private repo

one(repos, {.private})

Closures

  • {...} - closure

Closures allowed only with builtin functions. To access the current item, used the # symbol.

Example

map(0..9, {# / 2})

If the item of array is struct, it's possible to access fields of struct with omitted # symbol (#.Value becomes .Value).

Example

filter(comments, {len(.body) > 280})

Slices

  • myArray[:] - slice

Slices can work with arrays or strings

Example

The variable myArray is [1, 2, 3, 4, 5]

myArray[1:5] == [2, 3, 4]
myArray[3:] == [4, 5]
myArray[:4] == [1, 2, 3]
myArray[:] == myArray