Skip to Content

Wiki source code of 06. User Data Source

Last modified by Tom Jardine on 06/11/2025, 14:24
Show last authors
1 {{box cssClass="floatinginfobox" title="**Contents**"}}
2 {{toc/}}
3 {{/box}}
4
5 The User Data Source option allows users, with the relevant permissions, to upload and configure their own Data Sources from CSV and Excel files.
6
7 To access the User Data Source functionality, click **Data Source**.
8
9 This will display all currently uploaded User Data Sources, with the **Refresh** button available to display any changes made to the underlying structures.
10
11 = Creating a New Data Source =
12
13 Click **New Data Source** to start the process.
14
15 There are multiple steps to complete. When all options for a step have been completed, click the **Next** button. To return to a previous step, click the **Back** button.
16
17 == 1. Select Data Source ==
18
19 In this step, the data will be uploaded and the build process specified.
20
21 Click **Choose File** to open the file browser, where the **CSV** or **XLSX** file can be selected.
22
23 If a CSV file has been selected, specify the **Delimiter** and **Text Qualifier** using the relevant textbox.
24
25 If an XLSX file with multiple sheets has been selected, specify the sheet number using the **Sheet** textbox.
26
27 The **Build Type** options relate to when the Index is refreshed to account for any changes.
28
29 Selecting the **Complete** option will result in all items in a Data Source being re-Indexed. Once complete, a new version number is applied to the Index. Each run completely refreshes the Index.
30
31 Building an **Incremental** Index enables an optimised refresh process when rebuilding Indexes to account for any changes. Rather than rebuilding the entire Index, only data that has been modified or added will be processed.
32
33 To set the contents of the first row as the column headings in the resulting output, enable the **Use First Row as Header** option.
34
35 Use the **Add to Search Engine** drop-down list to select a Search Engine the new Data Source will be added to once configured.
36
37 == 2. Select Columns ==
38
39 Now the columns have been mapped, the unnecessary columns can now be removed from the output.
40
41 Deselect the checkbox above the columns that are not required. Only columns with the above checkbox enabled will be imported.
42
43 == 3. Rename Columns ==
44
45 The imported columns can now be renamed.
46
47 Enter the new name, if required, in the textbox next to the original column name.
48
49 == 4. Field Types ==
50
51 To ensure the data is imported correctly, the data type must be correctly configured for each column.
52
53 Next to each field, a **Type** drop-down list is available. While the data type is detected automatically, the following options are available for conversion:
54
55 === Data Type Conversion ===
56
57 Use the **Data Type Conversion** options to convert individual fields to a number of different formats.
58
59 The **Force Date Field**, **Force Number Field** and **Force Text Field** options will change how the field is recognised in other functionality areas, while the **Timestamp** option will attempt to convert the contents of the field into a time field matching the format selected in the subsequently available format drop-down list.
60
61 If a number or date type is selected, specify the format using the relevant drop-down list under in the **Format** column.
62
63 ==== Strings ====
64
65 The **Strings** options allow string values to be systematically modified to enforce consistency across the source data.
66
67 The **HTML Field** option will convert the field into HTML format, while the **JSON** option will output the data in JavaScript Object Notation format. These options are especially useful when reporting requirements dictate a specific data format.
68
69 Selecting **Literal** will import the field without removing any leading or trailing spaces.
70
71 Modify the presentation using the **Lower Case** option to display contents in lower case, the **Proper Case** option to display fields with the first letter of every word capitalised, the **Remove Spaces** option to delete spaces between words or the **Upper Case** option to display contents in upper case.
72
73 ==== Obfuscation ====
74
75 Using the **Obfuscation** options allow selected fields to be obscured, preventing individual records from being identifiable.
76
77 The **Anonymise** option will number fields to obscure the original field names, while the **Dual MD5**, **Dual SHA-2**, **Dual SHA-3**, **MD5**, **SHA-2** and **SHA-3** options will hash or double hash the fields to suit individual security requirements.
78
79 ==== Third Party ====
80
81 The **Third Party** options allow the conversion to a number of supported data types.
82
83 The **JD Edwards Date CYYDDDD** (Century, Year, Day of Year) and **JD Edwards Date CYYMMDD** (Century, Year, Month, Day of Month) options provide conversion to the J.D Edwards format of recording dates and the **MS Excel Date Serial Number** option will convert date fields to the sequential serial number used in Microsoft Excel to retain compatibility with Excel calculations.
84
85 The **SQL Latin 1 General CP1 CI AS Numeric** option will convert the fields into a SQL collation string using Latin rules, code page 1, case insensitive and accent sensitive string.
86
87 == 5. Advanced ==
88
89 Use the **Options** drop-down list to access the available settings.
90
91 ==== Metadata ====
92
93 Specify a **Description** that will be visible to those who have access to the report, and specify the **Count Description** to customise the wording displayed on the **Home** screen for the number of contained documents.
94
95 ==== Dates ====
96
97 Selecting Dates will present all of the specified date fields and the options to split individual parts of a date field into separate fields.
98
99 Use the **Split Column** drop-down list for the relevant field to select an individual component of the date field. Make note of the **Format** column, where the available components for each date field are detailed.
100
101 Selecting a field from the **Split Column** drop-down list will create a number of fields depending on the original field format and the option selected. The new fields are detailed below, and can be renamed by editing the relevant textbox in the **Field** column. To delete new fields, click the **X** icon next to the first field of the format
102
103 ==== Total Fields ====
104
105 Specify the fields that will be used as totals by enabling the checkbox next to the relevant field in the Total column.
106
107 ==== Crosstab Field ====
108
109 Specify the fields to exclude from the Crosstab and Venn diagram functionality by enabling the checkbox next to the relevant field in the Exclude column.
110
111 = Managing Created Data Sources =
112
113 Created Data Sources are displayed with a number of options used to maintain and modify the data made available to users.
114
115 Below the **Options** column, click the **Advanced Options** icon to reveal the options available in Step 5 of the creation process. Use the **Options** drop-down list to configure the key areas of the Data Source and click **Save** to save the selections made, or click **Cancel** to discard changes.
116
117 To apply any changes made to a Data Source, Click the **Run** icon. This will ensure any further configuration is applied to the underlying data. Click the **Refresh** button to monitor the build progress found in the **Status** column. Note that each build will update the number displayed in the **Version** column.
118
119 To revisit the complete configuration options, click the **Edit** icon. Once every step has been completed, the build will automatically be refreshed to apply the changes made.
120
121 To delete a Data Source, click the **Remove** icon. This will permanently remove all reports and associated data.