Configure Datastreams

The Configure Datastream tab is where sensors and the fields in a table representing them are defined. Below are instructions for how to create and configure them.

Navigate to the Configuration Interface

1. From Stations and the checklist, move down to Tools, Data Setup, Configure Datastreams. Two choices appear.

   Option
   Organization Datastreams	This is for manual measurements taken without a datalogger with no table. These will be associated with a site, not a station.
   Datastreams for a specific station	This is for everything else.

2. Choose the station you have been working on. If you do not see it, check to make sure you are in the correct organization (top, near center) and check under Stations in the menu to make sure you did not hide or disable the station.

3. Choose the time zone. Once a station has been selected, the assumption is that it is within a particular time zone.

4. Three options have been set up to create datastreams:

   • Use Existing Tables
   • Use Equipment
   • Batch Editing

   Currently only one is enabled. Select Use Existing Tables.

Select Fields

1. Once within the configuration interface, the name of your station will be at the top, along with your checklist in red. On the left is a list of database tables associated with this station.

   If you do not see the table you recently uploaded, hit the refresh button. It is represented by the text, Not seeing a table you expect? Click here to re-run table discovery.

   Manually imported files will have imported prefixed to their names.

2. Click on a table you wish to create datastreams from. The left side will show you a list of fields or headers from the CSV or table. It has their datatype described in a two letter icon on the right.

   Datastreams describe a measurement or observation taken at a specific place (site) using a specific method (ThingType or equipment). They usually have more than one field, since the data is aggregated into a time interval.

   Example Fields

   Air Temperature usually has average, min, max, and maybe standard deviation. All of these are associated with the same datastream.

   Note

   All values within one datastream have to have the same units. If you have a sensor that gives you min, max, average, and a text status quality field, the status quality field will go into a separate datastream.

Create a Datastream

I. Initiate and Configure Controlled Vocabulary

1. Select all the field names that should be associated with one datastream, (e.g. AT_C_Avg, AT_C_Max, AT_C_Min).

2. Click the blue button with the text + New Datastream.

3. Controlled Vocabulary: A small wizard will show up. The controlled vocabulary your organization selected as required will show up here as a set of drop down menus (See Set Up a Station - Controlled Vocabularies). It will always include Dendra’s internal Measurement controlled vocabulary.

   Select appropriate vocabulary. This is how the data will be found.

   Tip

   See Set Up a Station - Controlled Vocabularies for more information.

4. Hit Next.

II. Set Output Fields, Data Type, and Units

5. Two things must be done on the Output Fields tab. Output fields need to describe aggregates, such as average, min, max, standard deviation. If your data is not aggregated, select Value, First, or Last.

6. If you remember the order of your fields selected from the table, re-order the aggregates to match.

7. Data Type: On the same tab at the bottom, select what kind of data this is. Most monitoring data is floating point or Double. Counts are integer. Status information is often String.

8. Unit (Geophysical unit of the measurement): Set units. This is critical for the functioning of the datastream.

   Note

   If a geophysical unit is missing, please contact the Dendra team and request an addition.

9. Hit Next.

III. Set Attributes

1. Many instruments have specific attributes. These are most commonly heights, depths (e.g. Soil Moisture 15 cm, sometimes they can be calibration values.

   If your sensor does not have specific attributes, skip to the next step. Otherwise, choose an attribute, a value, and a unit, then hit Save.

   Example Attributes

   An air temperature sensor placed at head height is - Attribute Name: Height - Numeric: 2 - Unit: Meter (m)

2. Hit Next.

IV. Create and Start a New Configuration

Name and Create

1. While the controlled vocabulary allows a datastream to be found, the name gives additional detail once it is found. The station name will be associated with the datastream, so there is no need to include in the datastream name. Also, the name can be identical to others. Dendra will default to “Measurement” “Attribute”.

2. Hit Create & Start a New Config.

   Caution

   Only use second button if you know what you are doing.

New Configuration

1. You are now creating a new configuration.

   Configurations match tables and fields inside Dendra to the metadata about the datastream. Configurations are always bounded by a time range.

2. On Field Mappings: The fields you selected from the table are listed here on the left. The unit is listed next to it and the output field is on the right. You selected the output fields in a previous step.

   Note

   Operation allows you to perform a conversion calculation if you need to. Release 3 has not yet fully implemented this functionality. When it is active, it will no longer be greyed out.

3. On Output Fields: Check that the aggregation matches from input to output. If they don’t, move them around until they do.

4. Operation Code: Not yet functional. Coming soon!

5. Time Period: All configurations have a timeframe. Set the start timestamp on the left.

   Note

   On the top is the time zone of the datastream. The times shown are in that time zone, sans Daylight Savings. If you set your loggers to UTC, please make sure to set your station to UTC as well. Otherwise, there will be conversion issues.

   If the data is ongoing, leave the Ends Before blank.

   If you use the same timestamp on two configurations, set Ends Before on the first, then Begins At on the second. They will not overlap, since one ends before the other.

   Look at the bottom of the window. You should see data from the fields you selected show up with a timestamp. If it does not, you have the wrong start or end time.

6. Hit Save (the blue button on the top right).

   Once you have created a datastream it will be listed on the center-right with a timeline of configurations.

V. Finish, and Repeat

The fields that were entered into that datastream are now checked off with green checks on the field list on the left.

Continue to create datastreams using the yellow exclamation mark fields.

If you want to hide finished fields, switch Any Status to Unmapped.

Note

You have just created a datastream and given the datastream its first configuration. There is more information that should be added to a datastream. Most importantly, you should add the equipment type used.

By adding the equipment type (sensor), you are enabling the option to perform automated out-of-range checks and have access to the operating manuals for the sensor.

Edit an Existing Datastream

To edit an existing datastream, click the blue pencil icon to the right of the unit, to the right of the name. You can also find it under Datastreams in the left navigation bar.