04c. Creating an Index

Last modified by Jeff Nicholson on 06/11/2025, 14:23

Navigate to the Indexes screen by clicking Search Engine, then Indexes. All currently loaded Indexes will be displayed, and clicking New will allow the creation of a new Index.

Details

Enter the Index name in the Name text box.

Use the Tags textbox to add associated search terms to the Index. This allows the components to be accessed using customised strings when using the search bar.

The Description textbox allows text to be entered that describes the Index. This is displayed in the administration screens and on the Home screen to provide users with additional detail.

The Count Description options allows the customisation of the wording displayed on the Home screen. Left blank, the number of documents will be displayed as ‘Matches’. 

Using the Default Realm Access drop-down list, select either System Default, Open or Closed. Please refer to the Realms chapter for more information.

If building an Incremental Index, enabling the Most Recent First option will order the data chronologically. Enabling this option for Complete Indexes will order the rows as they are retrieved from the source system.

To set the column headers to stay in place when scrolling down in the Query screen, enable the Locked Headers Enabled option. This can be automatically enabled for all new Indexes using the Locked Headers Enabled option in the System Settings.

Enable the Limit Drop-Down List Items option to apply a soft limit to the number of values that can be displayed in a drop-down list. When enabled, only the first 1000 values are displayed to reduce system load. This can also be applied globally using Limit Drop List Items setting in the System Settings.

Select the Data Source Group(s) from which the Index will be created using the Groups added to Index drop-down list.

Index Fields

The Index Fields options allow the formatting of the Index fields to be specified.

The Index Field column contains the field names and, for number fields, the Format column will contain a drop-down list where the number display format can be selected.

To alter the field name that is displayed to end-users, amend the automatically populated Display Field textbox with the desired display name.

If an Index field contains free text, wrapping the text will visually improve the field when viewed in the Query screen. To wrap the contents of an Index field, use the drop-down list in the Wrapping column. Select either Words or Characters and specify the wrap value in the associated textbox. To further enhance presentation, use the Colour Palette drop-down list to apply a previously saved Colour Palette to individual fields.

To add a value to a field in the Index, click the icon in the Value column. Enter a value in the textbox and click Add. All added values will appear below. To remove a value, click the X icon next to the relevant entry. Click Close to add the values and return to the previous screen.

To add a field based on an original, click the Add button next to the relevant field. This will create an exact copy of the field below. This option is especially useful if additional views of the same data are required. For example, adding another date field, formatted as dd/MM/yyyy, and using the Format drop-down list to select MM/dd/yyyy would allow two different views of the same field to be added to reports. 

To remove a field, click the Remove button. Alternatively, all Index fields can be removed simultaneously by clicking the Remove All radio button. To reverse this action, click the Add All radio button.

By clicking the Remove Redundant button, unused or empty fields left over from previous builds can be deleted.

System Fields

Click the Show System Fields button to reveal the system fields generated during the build process. Click Hide System Fields to hide the options.

System fields allow the unique items from the Index make-up to be referenced in reporting. Organise and modify these fields using the same options as detailed above.

The following System Fields are available:

Issue ID

Summary

_DataSource

Denotes the Data Source GUID.

_DataSourceDisplay

Denotes the Data Source name.

_DataSourceGroup

Denotes the Data Source Group GUID.

_DataSourceGroupDisplay

Denotes the Data Source Group name.

_EffectiveDate

Virtual field used for querying purposes. Only available for Timeline Indexes.

_EffectiveTo

The current status of the Effective date. Only available for Timeline Indexes. 

_EntryStatus

Denotes whether the record is an Addition, Deletion, Modification (where the record has been updated), Replaced (where a new record with a valid effective date range has been added) or is a Re-Addition (there is a gap in the record's timeline). Only available for Timeline Indexes.

_EntryStatus2

The status of record before it was deleted. Only available for Timeline Indexes.

_Id

The identifier of the Indexed file. Only available for Unstructured Indexes.

_Indexed

Denotes when the records were inserted.

_MD5

The MD5 hash used to identify the Indexed file. Only available for Unstructured Indexes.

_Modified

The current Effective From date. Only available for Timeline Indexes.

_Type

The file type of the Indexed file. Only available for Unstructured Indexes.

_url

Used to identify URL values within an Index.

Directory

The original location of the Indexed file. Only available for Unstructured Indexes.

File

The original filename of the Indexed file, Only available for Unstructured Indexes.

Score

Virtual field used for ranking query results.

Size

The original size on disk of the Indexed file. Only available for Unstructured Indexes.

Display Fields

The Display Fields options are used to set the fields that can be used as totals.

Use the Total drop-down list to individually select the numeric fields that will be available to be used as a Measure, or click All to add every numerical field. To delete individual fields, click the X icon next to the relevant field. Alternatively, click None to remove all fields.

Fields that are selected using the Excluded drop-down list cannot be used as Axes in Crosstabs.

Descriptions

The Descriptions option allows you to enter a description against each field in the index that can be accessed from the query screen. This can be details from the data dictionary, or an explanation of how the value is calculated. In the query screen you will see an 'i' icon under any fields that have a description set against them and when you click this icon a popup box will appear to show the description.

Similar

The Similar options provide the configuration of the Find Similar feature that allows end-users to search for words similar to those typed.

Select the required fields from the relevant drop-down list to activate the feature. Click the All button add all fields, or None to remove all selected fields.

The Double Metaphone option utilises a phonetic encoding algorithm to account for the numerous applications of individual letters, supporting multiple languages.

Using the Soundex option will result in searches based on sound, as pronounced in English. Different words with similar pronunciation will be returned.

The Fuzzy Matching option searches words based on Levenshtein Distance. This measures the difference between words in the minimum number of single-character edits specified.

Targeting

Select the Target Index from the drop-down list to add its configuration settings to the Index.

For each field, select the relevant Index field from the Mapping drop-down list. This allows Targets to be used across multiple Indexes.

Please refer to the Targeting section of the Extensions chapter for more information.

Currencies

Currency conversion can be applied at Index level using the following options once the relevant currencies have been uploaded in the Currency Management screen.

Any field selected from the Total drop-down list in the Display Fields options will appear in the Currencies section in the Index Field column.

In the Type column, specify the field that will be converted using the Currency Index Field drop-down list and select a predefined currency matching the field from the Base Currency drop-down list. The Display Field column denotes how the field is displayed in the Index.

Select a predefined currency from that the field will be converted to from the To Currency drop-down list.

Filter Values

Using the Filter Values option, a list of reference values can be specified that can be used in the filter pane for Crosstab calculations.

To create a new value, click the Add button and type a name in the Name textbox. Use the Type drop-down list to specify the data format and the resulting Format drop-down list to further specify the display format for 

In the Label textbox, type the name that will appear in drop-down lists for users, and enter the required value in the Value textbox. Click Save to add the value and reveal a new row. Repeat the process to add multiple values.

Links

Adding links allows the joining of fields from another Index based on a join key.

To create a link, click the Add button and type a name in the Name textbox. Use the Form Field drop-down list to specify the join key for the currently loaded Index.

For the Index to be joined, select the required Index from the Index drop-down list, and its join key from the To Field drop-down list.

Display Field Layouts

Use this set of options to configure a number of display options for the Index in the Query screen.

The Order drop-down list specifies how the display field layouts will be ordered, with Alphabetic and Creation Date available as options. Select a field using the Default column radio button to apply the specified layout when the Query screen is first accessed with the Index loaded. Click Edit to modify an existing layout, or Remove to delete it.

To create a new layout, click the Add button and type a name in the Name textbox.

Use the Type drop-down list to specify the display format and reveal the applicable options. The Table option provides options to customise the default Query screen layout of the Index, while the JSON option outputs the data in JSON (JavaScript Object Notation) format for each row. Selecting Editable Table will activate the Editable Data functionality in the Query screen and the List and Fixed List options will restrict the view to a configurable number of rows for structured and unstructured data, respectively. Select HTML to design a layout that will be used for each row, or Word Cloud to display an interactive view of the data. The Clustering options reveal algorithm selection options to create display fields based on related data, and the CXFORMS option will read data directly from the CXFORMS Index.

Select individual fields from the Display Fields drop-down list, or click All to add all fields. Delete individual fields by clicking the X icon next to the relevant field, or click None to delete all fields. Re-order fields by dragging and dropping selections into the required order. Enable the Automatically add new columns option to ensure that any new fields that get added to the Index are automatically included in the display field configuration.

Select fields from the Constraint Fields drop-down list to force rows not to display. Use the Constrain radio buttons to select how many have to be blank before they are hidden, with None, All, or Any available as options.

When configuring a List, specify between 1 and 2 in the Columns drop-down list to specify the number of columns that will display the data. For unstructured data, selecting the Fixed List option reveals the Delimiter option. Specify what separates the field values in this textbox.

Selecting the HTML option reveals a text editor with a range of functions. Quickly reference a field using the drop-down list above and click Add Field. Enter required code manually by clicking Tools, then <> Source code. Enter the required code and click OK to add the contents, or click Cancel to return without applying its contents. Use the Page Size option below to specify how many rows of data will be displayed on each page of the Query screen.

When Word Cloud is selected, a number of display options become available. Please refer to the Word Cloud Configuration section for more information.

Selecting the Clustering option reveals a number of options to facilitate the grouping of related display fields via the Carrot2 open source clustering engine. Specify the amount of rows that will be scanned before the algorithm moves onto the next field using the Number of rows to scan textbox. The more rows that are scanned, the more accurate the grouping output at the expense of increased system load. Select the required algorithm from the Clustering drop-down list, with Bisecting K-Means, Lingo and Suffix Tree Clustering available to account for differing data structures. Select a field previously added to the Analysed list when the Index was created from the Display Field Layout drop-down list to specify the field that will be used to determine the resulting clusters.

Dynamic Views

Using Dynamic Views, Distinct Counts can be created for selected fields. Click the arrows in the Order column to re-order created dynamic views, click Edit to modify an existing dynamic view, or Remove to delete it.

To create a new layout, click the Add button and type a name in the Name textbox. Enter additional information in the Description textbox and select an Index field from the Display Field drop-down list to specify the field used to perform the distinct count.

Use the Display Field Layout drop-down list to select the display field layout the dynamic view will use when drilling through to the underlying data and select the required totals from the Total Fields drop-down list. Select individual fields from the Display Fields drop-down list, or click All to add all fields. Delete individual fields by clicking the X icon next to the relevant field, or click None to delete all fields.

Click Save Dynamic View to add it to the above table, or click Cancel to discard any changes.

Maps

The following options allow configured Maps to be setup for an Index. Once set, the Map option will become available in the Pages functionality.

To add a new map, click Add. Enter a name for the configured map settings and select a map type from the Map drop-down list.

Fusion Maps

Fusion Maps provide over 1,500 offline maps without the loading limits associated with the Google Maps API, and each entry can be previewed on the Fusion Maps website.

Maps can either be created separately for each Index, or created in the Fusion Maps screen and then selected from the Map Type drop-down list. To create a map that can be applied to multiple Indexes, navigate to the Fusion Maps screen by clicking Setup, Extensions, then Fusion Maps. Complete the following instructions and click Create Fusion Map to save the configuration that will then be available from the Map Type drop-down list.

When configuring a map at Index level, select a saved map or region applicable to the Index from the Map Type drop-down list and select the field containing the pertinent location information from the Code drop-down list.

To ensure that the selected Map Type matches the granularity of the data, it is recommended that this expansive drop-down list is fully explored.

There are a number of high-level options, such as Europe, Africa and North America and also variations based on a single country of interest. For example, England contains entries for every county while England Region contains entires for pre-defined regions that could encapsulate multiple fields from the Index.

With a Map Type and Code selected, click Data Mapping to map the field values from the Index to the relevant area of the selected map. Should any matches be found, they will be mapped automatically.

With the required fields mapped, click Save Map to complete the configuration process, or Cancel to discard changes.

Google Maps

With the relevant API configured in the System Settings, Google Maps can be used in reporting. Please note that the number of map loads is limited by the Google Account used to make the connection.

Select the relevant location field from the Name drop-down list and specify the two Latitude and Longitude fields in the Index from the applicable drop-down lists.

Under the Layers heading, select the required fields that will be used to add supplementary information to the map, such as county boundaries.

With the configuration options set, click Save Map to complete the configuration process, or Cancel to discard changes.

Google Maps Drivetime

The Google Maps Drivetime option is also available using an API and configuring its use in the System Settings. This functionality can be built into the Index without the use of a third-party API using the Distance Extra Fields option in the Data Source Groups screen.

Select the Index fields from the Venue Name, Venue Latitude, Venue Longitude, Patron Name, Patron Latitude, Patron Longitude drop-down lists that specify the points that will be used to calculate the distance.

Under the Layers heading, select the required fields that will be used to add supplementary information to the map, such as county boundaries.

Click Save Map to complete the process, or click Cancel to discard any changes.

Exporters

Using the following options, the exporting functionality can be limited to a select few file types.

Use the Add Exporter drop-down list to add individual file types, or click All to add all available options. Click the X icon next to the relevant entry to remove individual file types, or click None to remove all selections.

The specified options will apply to all reports built from the Index.

Database

When the Database option is added, click the ... icon to configure the database that the data will be exported to. Please note that a Database Connection must be configured and saved before the database can be specified at Index level. The database instance must be running on the same server as CXAIR.

Use the Database Connection drop-down list to specify the saved Database Connection that will be used to connect to the target database, and select the relevant table from the Table drop-down list.

Click New next to the Field Mappings option to map fields from the Index to those in the database. To create additional field mappings, click New to reveal another row of options.

Enable the Delete All Rows option to clear previously exported rows before adding the new entries. If left disabled, the same records will be amended to the database table after every export.

If operating over a network with a high amount of concurrent traffic, reduce the Rows per Transaction value to export the data in smaller packets, ensuring the export process does not impact network performance.

Twitter

Before exporting to Twitter, a number of options must be configured. Please refer to the Twitter Configuration chapter for more information.

When the Twitter option is added, click the ... icon to reveal the Screen Names drop-down list. Select the Index field containing the screen name to map its value to the export process.

External Files

These options allow uploaded documents relating to the Index to be accessed from the home screen.

Click Add and then Select File. Locate and select the required file to add it to the Index. For added files, click Edit to replace the file, or Remove to delete it.

Languages

To translate column names into a different language, click Choose file to upload the relevant properties file.

Once all the options have been completed, click Create Index to complete the process. To build the Index and navigate directly to the Search Engine or Query screen, click the up arrow next to this text and click Create Index and View Search Engines or Create Index and Search.

Hierarchy

Using the Hierarchy options for an Index, fields can be related to enhance the ad hoc functionality in the Query screen for end users by presenting pre-defined charts for specific fields.

To define a hierarchy, click Add and enter a label in the Name textbox.

Specify the fields that will be used to drive the resulting chart using the Set Hierarchy drop-down list. For example, selecting 'Region', 'Area' then 'Branch' will set the data in the appropriate order to group rows together. Drag the fields into the required order and use the X icon to remove any unwanted entries.

Then select the required chart from the Set Chart Type drop-down list and select the theme from the Colours drop-down list. Leaving the below radio buttons set to Default Colours will display the chart using the predefined colour scheme, or All Same Colour can be selected to specify the colour used by entering the corresponding hex code or using the popup colour picker.

A preview of the resulting chart is drawn from the top ten values and can be used to further configure the styling options. Click an area of the preview and select the area from the subsequent popup to reveal the styling options.

Click Save Hierarchy to save the configuration options, or Cancel to discard any changes.

This functionality must be enabled in the User Preferences for users to activate this feature.