Final Report Testbed Sensordata 2026

For this testbed a FROST-Server instance was deployed on the cloud environment of Brabandse Delta. This cloud environment uses Portainer to manage a Docker environment, and uses NGINX as a reverse proxy to route HTTP requests to the correct docker container.

When running in a container environment, FROST-Server consists of separate containers for the HTTP and MQTT parts of the API. These containers have to state and can be duplicated as required to satisfy the load put on the system. The separate FROST-Server containers communicate with each other using a message bus, so that when an update is made through one container, the other containers are notified so they can send push notifications to any users that have subscriptions on the changed entities.

The data that is managed by the system is stored on a PostgreSQL database server with PostGIS extensions. This database server writes the data to a persistent volume, which is the only part of the system that is persitent over restarts.

In a docker environment the deployment of a composite system with sub-systems is configured in a docker-compose.yaml file. A basic docker-compose file consists of a name, a set of services and a set of persistent volumes. The name is used as a prefix to name containers. The volumes are the file systems that persist across restarts of the services. The services specify which containers are generated, and which settings are used for these containers. The basic structure of a docker-compose file for FROST-Server is shown in the following example:

name: myFROST
services:
  nginx: (Usually defined externally)
    image: nginx:1-alpine
    [...]

  frosthttp:
    image: fraunhoferiosb/frost-server-http:latest
    [...]

  frostmqtt:
    image: fraunhoferiosb/frost-server-mqtt:latest
    [...]

  mosquitto:
    image: eclipse-mosquitto
    [...]

  database:
    image: postgis/postgis:18-3.6-alpine
    [...]

volumes:
    postgis_volume: {}

Each service lists an image that is used as a basis for the containers of that service, the ports that are exposed to the outside, the persistent volumed mapped into the containers and environment variables.

A ports definition is only needed for services that need to be directly accessible from outside of the cluster. All services in the same docker-compose file can access each other directly using the service name. Therefore, the only service that needs a portmapping is our NGINX service.

The configuration of NGINX must be done using a confguration file, that is placed next to the docker-compose.yaml file and mapped into the appropriate place in the container, using the read-only (:ro) mode to ensure it can not be changed.

name: myFROST
services:
  nginx:
    image: nginx:1-alpine
    ports:
      - "443:443"
    volumes:
      - "./nginx.conf:/etc/nginx/conf.d/default.conf:ro"

All configuration of FROST-Server is done using environment variables, listed in the environment section of the service. There are many settings, listed in the documentation. FROST-Server lists each configuration setting in its log files and on the console, when it first uses a setting.

The environment variables can use placeholders that in turn are defined in a separate .env file. This is useful for values that appear multiple times in the file, or for passwords or other secrets that should not appear as literals in the file.

The bus_mqttBroker and persistence_db_url variables reference other services in the same docker-compose file by simply using the names of these services.

  frosthttp:
    image: fraunhoferiosb/frost-server-http:latest
    environment:
      - serviceRootUrl=http://${hostname}/FROST-Server
      - mqtt_exposedEndpoints=ws://${hostname}/mqtt
      - plugins_coreModel_enable=true
      - plugins_actuation_enable=true
      - plugins_openCitySense_enable=true
      - plugins_projects_enable=true
      - bus_busImplementationClass=de.fraunhofer.iosb.ilt.frostserver.messagebus.MqttMessageBus
      - bus_mqttBroker=tcp://mosquitto:1883
      - persistence_db_url=jdbc:postgresql://database:5432/sensorthings
      - persistence_db_username=${db_username}
      - persistence_db_password=${db_password}
      - persistence_autoUpdateDatabase=false
      # [More settings ...]

Some complete docker-compose examples can be found in the FROST-Server repository.

The entire set of services defined in the docker-compose file can be started with the single command:

docker compose -f docker-compose.yaml up

To stop all services in one go, the following command can be used:

docker compose -f docker-compose.yaml down

When no authentication module is enabled, anyone can create, edit and delete all entities on the server. Usually this is not desired, and thus an auth* provider should be enabled. FROST-Server comes with two auth providers out of the box: Basic Auth and KeyCloak auth.

The Basic Auth provider uses internal user and userrole tables, that can be in the same database as the other data, or in a different database. A user uses the HTTP Basic Auth method to authenticate directly to FROST. Users have to be added directly in the database using SQL queries.

The KeyCloak Auth provider uses an external authentication server to handle user authentication using OpenID-Connect.

To enable basic auth, the following set of enviroment variables can be added to both the frosthttp and frostmqtt services in the docker-compose file:

      # [More settings ...]
      - auth_provider=de.fraunhofer.iosb.ilt.frostserver.auth.basic.BasicAuthProvider
      - auth_db_driver=org.postgresql.Driver
      - auth_db_url=jdbc:postgresql://database:5432/sensorthings
      - auth_db_username=${db_username}
      - auth_db_password=${db_password}
      - auth_plainTextPassword=false
      - auth_autoUpdateDatabase=true
      - auth_allowAnonymousRead=false
      # [More settings ...]

The first time FROST-Server is started with an empty database, a few standard users are created with standard passwords:

• read / read: A user that can only read entities.
• write / write: A user that can read, create and update entities, but not delete entities.
• admin / admin: A user that can read, create, update and delete entities, and can access the Database Status page.

These should be changed as soon as possible.

The server at Brabandse Delta is available under the following URLs:

• The REST API is available over HTTPS at: https://sta.wbd-rd.nl/FROST-Server/
• The MQTT API is availale over WebSockets at: wss://sta.wbd-rd.nl/mqtt

The core data model of version 1.1 of the SensorThings API consists of 9 classes, 8 relations between those classes. The following diagram shows the UML representation of the data model:

A detailed description of each class and its properties and relations can be found in the specification: OGC 18-088.

The standard tasking extension OGC 17-079r1 can be used to model Actuators, their tasking capabilities, and the task send to the actuators. The extension adds three extra classes to the SensorThings data model, shown in the figure below in orange.

In FROST-Server the tasking data model extension can be enabled using the environment variable plugins_actuation_enable=true.

To make the complexities of sensor management more manageable, Fraunhofer IOSB started an internal research project to design the concept for a sensor management system, based on the OGC SensorThings API, and create an implementation of this system: OpenCitySense.

The main reason that sensor management is complex is that there are many different types of sensors, from many different vendors, using many different communication protocols. The result is that a sensor management system consists of many different sub-systems, and these sub-systems all store a part of the management data of each sensor that they (partially) manage. The management data of each sensor is thus spread out over many systems, and that makes coordination of this data and control over these subsystems problematic. Since the SensorThings API version 1.1 is extendible by design, and already comes with several extensions, the architecture can be greatly simplified by using the SensorThings API service as the central data store for all sensor related data (figure below). This means that all components communicate through a single service, reducing the number of interconnects between components and reducing the spread of primary information across components.

The standard tasking extension can be used to coordinate management actions between components, such as signalling to a connector that a sensor needs to be on-boarded, that a configuration needs to be changed, or that a sensor needs to be off-boarded. The connector concept can then be extended to not just be a one-way ETL process, but to take an active role in the sensor registration process on the LoRaWAN stack. It can receive information about new or updated sensors from the SensorThings service, and automatically take all required registration actions in the communication infrastructure.

Besides greatly simplifying the architecture, a second major advantage to using the SensorThing API service for all data storage is that it offers a consistent, powerful API for managing relational data. This makes all data relevant for managing sensors and their data available in a unified, consistent way, and management tools or other clients do not need to implement multiple APIs and access data in multiple services. While all publicly relevant sensor data and metadata can be stored in the core data model of the SensorThings API, internal management data can be stored in a custom data model extension. Since this does not alter the core data model of the SensorThings API, clients implementing only the Sensing part will not be affected by this data model extension.

Sensor configuration parameters are modelled using JSON-Schema, and translated by the connector into a form that the sensor understands. This means that regardless of sensor brand or type, the management GUI can offer a consistent interface for changing sensor settings.

2.3.3.1 Data Model

To allow the representation of device management information, a data model extension has been designed for the data models of the SensorThings API and the tasking extension. The extended data mode is depicted in the following image.

Connectors and Devices are modelled as Things. To make it easier to distinguish between different types of Things, a "type" field has been added to the Thing entity type that indicates the type of the thing. Things of type "Connector" are linked to the Things of the devices they manage, through the ControlledDevices <-> ControllingConnector relation. This makes it easy to find all the devices controlled by a certain connector, and to find the connector controlling a certain device.

Each Thing can have a DeviceModel, describing the capabilities of the Device or Connector. A DeviceModel contains the schema for the Configurations of devices of this model, and can link to a Configuration that is the template or default configuration of devices of this model. DeviceModels can link to a Decoder that can be used to decode and encode data coming from and sent to devices of this model.

DeviceModels link to Sensors that describe the Sensors that a device of the model has. In turn, Sensors link to the ObservedProperties that a Sensor of this type observes. Using these two links, a Connector knows which Datastreams to create and which Sensor and ObservedProperty to link, when onboarding a Device.

DeviceModels link to the DeviceModels of the Connectors that they are compatible with. This allows a user interface to find the DeviceModels that work on a chosen Connector, and allows the Connector to specify additional configuration options it requires on a Device and a DeviceModel.

Configurations describe how a device can be, was or is configured. The schema for the config is stored in the DeviceModel of the device. The status field of a configuration indicates the current status of a sensor (Created, Active, Inactive, Removed) or if the Configuration is a Template. Configurations have a time field that indicates when this configuration became active. If a device has multiple configurations there must be only one configuration with status "Active". The other configurations are historical Configurations or templates.

To allow the secure storage of passwords or API keys, the DeviceSecret class was added to the data model. The secrets can be secured, both by only giving certain users read-access to these device secrets, and by encrypting the values of the device secrets. To allow Encryption, a connector has a public/private key pair. The private key of a connector is not stored in the SensorThings data model, but directly passed to the connector, usually using an environment variable. The public key of the connector is available in the SensorThings data model and can be used by clients to encrypt passwords before storing them in a DeviceSecret entity. This way only the connector can decrypt these secrets.

2.3.3.2 Configuration Definitions

A device generally has two types op parameters: Fixed ones that can not be changed during the operation of the device, and dynamic ones that can be changed.

The dynamic parameters are independent of the Connector used to manage the device. These parameters are specified in the section configuration of the configDefinition attribute of the DeviceModel, and the values for these are stored in the config attribute of the active Configuration entity linked to the device.

The static parameters are (can be) connector-specific. They are stored in the properties of the device (Thing), as secret linked to the device or in the properties of the DeviceModel. The definition of these parameters is stored in section configuration of the configDefinition attribute of the DeviceModel of the connector.

The configDefinition of a DeviceModel may thus have three sections:

• configDefinition/configuration: Definitions of fields that can be added to te Configuration of devices of this DeviceModel. These fields should be shown when creating a new device, or when changing the configuration of a device.
• configDefinition/device: Definitions of fields that the connector requires in the properties or a DeviceSecret of a Thing that it manages. These should be shown when creating a new device.
• configDefinition/deviceModel: Definitions of fields that the connector requires in the properties of a DeviceModel of devices that it manages. These should be shown when creating a new DeviceModel, or when linking an existing DeviceModel to the DeviceModel of a Connector.

Using these schema definitions, a user interface can generate all the forms required for onboarding and managing both connectors and devices.

Security is an important aspect of any API, especially when the API is use for both writing and reading. In many cases, a simple all-or-nothing approach is sufficient, but when multiple groups of users use the same service, yet should not be able to change, or even read, each others data, a more fine-grained approach to access control is required. FROST-Server comes with a highly-configurable fine-grained access control engine, but setting up the access-control rules can be a daunting task.

To help with setting up access rules, the Projects extension provides a data-model extension and a set of access-control rules that are sufficient for most use cases. The finer details of the Projects extension can be found in its documentation.

2.3.4.1 Data Model

The image below shows the core STA data model in blue, with the security extension in yellow.

To be able to take user information into account when determining which actions a user can do on individual entities, the user and its roles need to be connected to the data model. The Projects extension does this by introducing a Project entity type. Users can have certain Roles in one or more Projects. The other STA entities can also be connected to one or more Projects, either directly (such as Thing) or indirectly (such as Datasteam and Observation).

In STA version 1.x, the ObservedProperty has an attribute called definition that is meant to point to an external, authoritative source for the ObservedProperty. This use case, where an entity has an external, authoritative definition exists not just for ObservedProperties, but for almost all entities. Therefore, in STA 2.0, almost all classes have a definition attribute.

One of the most asked questions people have when starting to use STA is: "What should I use as Thing, and what as Feature?" Unfortunately, for STA 1.x, there was only the pragmatic answer: "The Thing has to be that which you want to attach a time-series to." This is because a Thing is linked to time series (Datastreams) and FeaturesOfInterest are not. This limitation breaks the usefulness of the Feature, and made it impossible to properly use the FeatureOfInterest in the way it was intended in the conceptual O&M Model.

In the update of the OGC/ISO Observations and Measurements specification, now called Observations Measurements and Samples (OMS) the concept of Feature and its roles in relation to Observations has be clarified further. A Feature can have multiple roles when relating to Observations:

• UltimateFeatureOfInterest: The largest scale Feature that is represented by an Observation.
• ProximateFeatureOfInterest: A smaller, more precise proximate for the ultimate Feature.

An example from the water domain would be that of a River, that has sampling points, where water samples are taken, on which measurements are made_

• The River is the Ultimate Feature of Interest. All measurements made on all samples taken at all sampling points ultimately say something about the water in the river.
• The sampling points are Proximate Featurs of Interest. They are a proxy for the river with a higher spatial detail.
• The samples are Proximate Featurs of Interest for the sampling points. They have a higher temporal detail and can capture processing steps made to the water before Observations are made.

The river, sampling points and samples are just Features. They only become "ofInterest" in the context of one or more Observations.

This more detailed model has been taken over in STA 2.0 as depicted in Figuur 9:

Features are now just named Feature, and no longer FeatureOfInterest. Features can now be directly linked to Datastreams, in the roles of Ultimate or ProximateFeatureOfInterest, and to Observations in the role of ProximateFeatureOfInterest. The relation between Observation and Feature is now optional, since it is only relevant when the Observation was made on a sample. Features now have one or more FeatureTypes, which makes it possible to quicky find all Features of the same type..

The result attribute of an Observation in STA has always been of type ANY, meaning its value can be anything. A major problem with this is that STA version 1.x has no way to indicate what a client can expect. It could be a number, a string, an array, or a complex json object. In version 2.0 this has been improved as depicted in Figuur 10:

The Datastream now has two fields, resultType and resultEncoding that use the OGC SWE-Common standard to describe the exact schema of the values of the result attribute of the Observations in a Datastream. Because a complex Observation can involve multiple ObservedProperties, a Datastream can now be linked to multiple ObservedProperties. The draft specification has examples of Datastreams with simple and complex resultTypes.

This change makes the MultiDatastream extension of STA 1.x obsolete, since the MultiDatastream structure can also be described by the resultType attribute.

The STA v1.1 entity model is itself an encoding of OGC Observations, Measurements and Samples: Things, Sensors, ObservedProperties, Datastreams, Observations and FeaturesOfInterest map one-to-one onto OMS concepts. The server implements this model with a few points worth noting:

• Observation types — Datastreams carry their OMS observationType (OM_Measurement, OM_TruthObservation, OM_CountObservation, …) as OGC-defined URIs, and observation result values are validated against the declared type.
• Unitless streams — for Datastreams without a unit of measurement (e.g. OM_TruthObservation for passive-infrared detectors) the unitOfMeasurement sub-keys are null, exactly as the specification mandates. Our validation was initially stricter than the standard here; see Lessons learned.
• FeatureOfInterest resolution — when an Observation arrives without an explicit FeatureOfInterest, the server auto-resolves it from the Thing's Location per the specification.
• Time-series storage — Observations live in a TimescaleDB hypertable partitioned on phenomenonTime, which keeps large time-range queries fast without deviating from the STA query interface. phenomenonTime supports both instants and ISO 8601 intervals.

Our proposal envisioned linking the OMS FeatureOfInterest to a hierarchical System Breakdown Structure with IFC-based object definitions managed in ANT CDE. During the testbed we deliberately shifted that depth toward the ingest side: full MQTT support (both server-side publish and connector subscriptions) and a multi-protocol connector programme (HTTP, MQTT, Modbus, LoRaWAN via KPN Things) — capabilities the tender lists as optional but which proved to be where the interoperability lessons are. The ANT platform is capable of the sensor-to-object mapping — the path would link each FeatureOfInterest to an object node in the ANT CDE breakdown structure, so observations inherit the asset context (which pump, which bridge deck) of the object they measure — but we have not built or tested that path within the testbed, and we report it here as future work rather than a delivered feature.

The originally proposed Brabantse Delta / Node-RED source was not implemented; as agreed with Geonovum, connectivity was validated with the other sources and testbed connectors instead.

A source is fully described in YAML: its ingest protocol and parameters, its transform, its target datastream and its egress protocol. Defaults (batching, retry schedule) live once and are overridable per source. Secrets never live in YAML — the configuration only names environment variables. Transform graphs (byte decoding, unpacking, unit normalisation) are authored in the portal and executed by the connector. Sources can also be provisioned at runtime through the admin API, which is how the portal onboards a new source without YAML edits. The repository ships an annotated example configuration (config/connector.example.yaml) that doubles as the template for a new source.

The Green Village's indoor climate sensors (temperature, humidity, CO2, air pressure) publish to a Confluent Cloud Kafka topic using Avro serialization with a Schema Registry. Onboarding required: obtaining Confluent Cloud API credentials, understanding the Avro schema, mapping measurement_id values to SensorThings entities via a device mapping configuration, and handling Avro union types in deserialized records. Once connected, the data flows reliably at 5-minute intervals. This was the most technically demanding integration but also the most robust once established.

The Ohnics SamenMeten network exposes outdoor air quality readings (PM2.5 and temperature) for sensors in Delft via a public JSON endpoint. No authentication, no registration, no credentials needed. The connector polls the endpoint every 5 minutes, discovers sensors by name prefix, and dynamically registers SensorThings entities for each discovered sensor. This was by far the easiest onboarding experience and demonstrates the value of publicly accessible sensor APIs.

Groundwater level sensors at TGV installations are managed via the CARS Online platform, which exposes an OData API secured with OAuth2 client credentials. Onboarding required: obtaining OAuth2 credentials, discovering installation UUIDs via a custom discovery script, understanding the proprietary API contract (POST with startDate/endDate/amountOfEntries), and mapping the response to SensorThings entities. Each installation needed to be individually configured with its UUID, name, and coordinates.

Despite being confirmed in our tender as a primary fieldlab location, we have not been able to connect Blijdorp sensors during the testbed period. The challenge was not technical but organizational: coordinating access across institutional boundaries, clarifying data ownership, and obtaining credentials took longer than the testbed timeline allowed. The entity model for Blijdorp was registered as scaffolding in FROST (10 Things with Locations, Sensors, and Datastreams), proving that the SensorThings side was ready, but no live observations have flowed. This is an important lesson for future testbeds: governance and access provisioning can be a larger bottleneck than any technical challenge.

The Erasmus University Rotterdam campus was included in our tender as a fieldlab site with weather stations and soil moisture sensors for climate adaptation monitoring. Like Blijdorp, the connector architecture and entity model are ready to receive the data, but access provisioning and institutional coordination have not been completed within the testbed period. The pattern is the same: the technical side is not the bottleneck.

The table below summarizes the contrast:

The SensorThings API entity model proved to be a good fit for climate adaptation sensor data. The separation between Thing (the installation), Sensor (the instrument), ObservedProperty (what is measured), and Datastream (the specific combination) maps naturally to how fieldlabs organize their monitoring. The OData query capabilities ($filter, $expand, $orderby) were powerful for programmatic lookup of existing entities and for seeding deduplication timestamps on startup.

Most importantly, the standard delivers on its core promise: once data is in a SensorThings server, it is genuinely interoperable. Our observations are queryable by anyone with access to the FROST endpoint, using a standard protocol, without needing to know anything about the source infrastructure. This is exactly the kind of interoperability that climate adaptation practitioners need.

These issues were raised and discussed with other participants and server implementers in the testbed GitHub repository. We opened discussions #22, #23, and #24 ourselves; the others were started by fellow participants and reflect shared experiences across the testbed.

Entity modeling is non-trivial (discussion #22). The standard defines the entity types but provides no guidance on naming conventions, Thing granularity, or how to structure properties. Different implementers will model the same sensor installation in different ways, undermining the interoperability the standard aims to provide. For example: should a weather station with five parameters be one Thing with five Datastreams, or five Things? The standard allows both, but only consistent choices enable meaningful cross-dataset queries.

Observation deduplication is left to the client, and it costs a round-trip per write (discussion #23). The standard has no mechanism for idempotent observation submission. (Datastream, phenomenonTime) is a natural key in every deployment we saw, but nothing enforces it, so re-polling, back-filling, or retrying after a timeout can all create duplicates with no server-side protection. The safe-but-slow pattern is to probe (GET …/Observations?$filter=phenomenonTime eq …) before every write; the fast pattern is to write optimistically and treat an HTTP 409 Conflict as "already delivered", but that only deduplicates if the server actually enforces uniqueness and returns 409, which is undocumented and varies by implementation. We settled on the optimistic path plus three safeguards: startup seeding of the latest timestamp per datastream from FROST, deferring the Kafka offset commit until after the push succeeds, and an existence probe confined to the rare dead-letter replay path. This is a common need that should be addressed at the standard or server level, ideally as a discoverable dedupe-on-natural-key capability, so a client can tell whether the cheap path is safe rather than establishing it by trial. To make that concrete, we implemented the client half of exactly this idea: a capability-discovery step that records each server's advertised write capabilities (data-array batch support, client-specified IDs, and any duplicate-handling behavior). Today that information is incomplete and inconsistent across servers, which is precisely why a discoverable, standardized dedupe capability would remove guesswork rather than add to it.

The v1.1 to v2.0 gap is significant (discussion #17). When we added a FROST v2.0 target alongside our v1.1 servers, we discovered that the payload format differs substantially: @iot.id becomes a different field structure, observationType and unitOfMeasurement are replaced by SWE-Common resultType, and collection count fields use different names (@iot.count vs @count). We had to build a payload adaptation layer to support both versions. This transition path needs clear documentation and tooling support if v2.0 adoption is to succeed.

Semantic interoperability stops where syntactic interoperability ends (discussion #24). ObservedProperty.definition is a free URI and unitOfMeasurement is effectively free text. This openness is deliberate (no standard can enumerate every phenomenon in every domain), but it means a consumer cannot reliably answer "give me all CO2 readings in this area" across servers, or even within a single server, without someone hand-curating a mapping. Avoiding exactly that work is why organizations adopt standards in the first place. We addressed it on our own side by pinning every ObservedProperty.definition to a CF Standard Names URI and every unitOfMeasurement.symbol to a UCUM code (Cel, %, [ppm], deg, hPa, km/h, mm, W/m2), never null or a placeholder. But this only produces interoperability if other parties converge on the same authorities. This is the layer a national profile is best placed to pin down, and it requires no change to the specification.

Testing against multiple FROST server instances revealed inconsistencies that single-server testing would have missed. Many of these were tracked in the testbed's shared bug-report thread (discussion #10):

• $expand on Observations did not work as expected on one server, requiring individual queries instead of batch expansion (discussion #10).
• resultQuality format differed between servers: one accepted a scalar string, another required an array. The same valid SensorThings payload was rejected by one server and accepted by another.
• Entity ID types differed: the Collaborall PHP server rejected string @iot.id values and required native integers, while FROST accepted both. The same reference payload had to be coerced per target (discussion #12).
• FeatureOfInterest auto-resolution and deep-insert semantics differed, and this silently killed an entire target. The CollaborAll PHP server rejected every observation with an HTTP 409 Conflict because its Things had no linked Location, so the server could not auto-generate a FeatureOfInterest for the observation. On the FROST servers the same entities worked. Worse, the standard remedy (a deep-insert of nested Things when POSTing the Location) was silently ignored by the PHP server, so the Thing and Location existed but were never associated. We only diagnosed this weeks in, because the failure was invisible: registration succeeded, the datastreams existed, and the observations returned 409 (which our optimistic dedup logic treats as "already delivered"). The fix was to explicitly PATCH-link each Thing to its Location on the secondary stacks. This is a sharp illustration that a server accepting your entities is not the same as a server accepting your observations, and that implicit FeatureOfInterest generation is under-specified across implementations (discussion #10).
• Batch push behavior (the CreateObservations dataArray extension) varied: a batch of 4,500 observations would succeed on one server but time out on another, requiring us to implement chunking (500 observations per request) and configurable timeouts (discussion #18).
• Error response formats for the same problem (e.g., a duplicate entity name) differed between server implementations, making generic error handling more difficult.

These are not failures of the standard itself, but they demonstrate that interoperability at the standard level does not automatically mean interoperability at the implementation level. Conformance testing for SensorThings servers would help.

The hardest interoperability problem we faced was not on the SensorThings side at all. It was getting access to the data in the first place. Each fieldlab has its own infrastructure, its own governance model, its own authentication mechanism, and its own data format. The SensorThings API standardizes the output, but the input side of the pipeline, where the real complexity lives, remains entirely ad-hoc.

Sensor metadata (make, model, calibration data, installation details) is rarely available in machine-readable form. We often had to discover sensor capabilities empirically by examining the data that arrived, rather than by consulting documentation. This makes entity modeling a trial-and-error process rather than a systematic one.

At the kick-off session we introduced Geo Insights and our co-applicant Building Changes, presented our use case (connecting climate adaptation sensors from urban fieldlabs to the SensorThings API), and outlined our goals for the testbed: building an open-source connector, pushing to multiple FROST servers, and feeding observations into our UrbanAdapt platform. With the ultimate goal to create a national fieldlab sensornetwork and train a geospatial foundation model.

At session 2 the connector existed as scaffolding: 10 Things, 26 Datastreams, and 13 ObservedProperties were registered on the Brabantse Delta FROST server for The Green Village and Diergaarde Blijdorp, but no live sensor data was flowing yet; we were still waiting for real payload samples from the fieldlabs. The presentation introduced the connector architecture (FastAPI, entity registration, dead-letter replay, Node-RED bridge), demonstrated governance via the FROST Projects extension, and showed a working Tasking MVP end-to-end on the live server. Open questions posed to the audience included push-vs-pull per fieldlab location and how to populate resultQuality across sources with different reliability characteristics.

Presentation slides: Public session 2 slides (PDF)

By session 3 the project had moved from scaffolding to live operation. Three real data sources were connected (TGV Office Lab via Kafka with 5 sensors, Ohnics air quality via REST polling with 46 sensors, and Levellog groundwater via OData with OAuth2 with 3 sensors) and observations were flowing to all three FROST targets simultaneously (Brabantse Delta v1.1, CollaborAll v1.1, and the Fraunhofer v2.0 test server). The presentation walked through the engineering problems solved since session 2 (Avro union type handling, polling deduplication, multi-server fan-out with per-target auth and versioning) and gave a live demo of the UrbanAdapt Monitoring Module, which had been expanded with features inspired by another participant's RIME Observatory. The session closed with 101 datastreams live across 2 data sources feeding 2 servers, and a roadmap for onboarding Blijdorp, EUR Campus, Living Lab Binckhaven, and AMS Institute in the coming months.

Presentation slides: Public session 3 slides (PDF)

Topic 2 asks participants to connect one or more sensors to an OGC SensorThings API server, to validate that the connection is reliable, to document an approach others can reproduce, and to report what we learned about the practical effort involved. Each participant picks their own technical route: connect devices directly, or put an intermediary in between.

We built an intermediary translation layer instead of connecting devices directly, because our sensors are not reachable as devices: the waste-container fleet reports into a vendor platform, and what we get is a vendor API. The layer takes any source, maps it to the STA entity model, and writes it to one or more STA servers, and the parts that make those writes reliable sit between the two ends, meaning the stored cursor, the deduplication and the write-log.

We validated the layer against the live Brabantse Delta FROST-Server, not a local mock: five containers with their fill-level datastreams and more than 1,400 observations, written and read back over both HTTP and MQTT, including a multi-day catch-up and a restart.

We then did something that was not in the original scope, and learned more from it than from the planned work. We used the same codebase to read another organisation's STA server and copy it into ours. That gave us two directions, two independent servers and one entity model, and it turned a single-server exercise into a real interoperability test.

Finally we connected the real thing. The SULO waste-container fleet reports into a proprietary vendor platform, and once its documentation reached us we wrote a third adapter against it: 51 container slots across 27 sites in Nieuwpoort, Belgium, four waste fractions, hourly fill-level estimates, published into the same shared FROST server. This is the case most organisations are in: the source is a vendor API that will never speak STA, and it is where the standard's value has to be argued. It also produced the single most serious defect we found, in section 4.11.

Our main finding is where the work goes.

Conforming to the standard was the cheap part. Everything around it, meaning transport, authentication, server-specific behaviour, concurrency, idempotency and semantics, is where the engineering went.

Seven findings shaped our view most.

• Mapping to STA was mechanical. An unfamiliar domain, municipal waste containers, fitted the seven core entities with no extensions and no compromises. A second, unrelated domain then fitted the same mapping layer (see sections 1.2, 1.3 and 5.1).
• Concurrency corrupted data in silence. Resolving datastreams in parallel duplicated the entities they share. STA does not require names to be unique, so the server reported nothing at all (see section 4.4).
• Idempotency is the client's problem, and it costs throughput. Idempotency here means that writing the same reading twice leaves one copy on the server. Nothing in STA enforces it, so we built our own bookkeeping plus a check before every write, and that check, not the write, became the bottleneck (see section 4.5).
• The same payload was accepted by one server and rejected by another. We read the specification. The payload is legal, and the rejecting server enforces a constraint the standard does not state (see section 4.7).
• Well-formed STA data can still be hard to use. The second server is valid and carries four different definitions of "CO₂ level", plus units written as "...". We achieved interoperability of form. We did not achieve interoperability of meaning (see section 4.8).
• Server capabilities are not discoverable. Which extensions exist, which authentication scheme applies, whether the build is standard, even which dialect of the query language is in force: we learned all of it by probing, one request at a time (see sections 4.2, 4.9 and 6.3).
• Two individually correct rules combined to delete data. Our newest source publishes forecasts in the same collection as measurements, distinguished only by a future timestamp. Rejecting a future-dated reading is right; advancing the poll cursor past a definitively rejected reading is also right. Together they moved the cursor into the forecast horizon and discarded every real measurement until that date, with no error logged, while the stream still looked healthy (see section 4.11).

Four clusters account for the effort the standard itself caused: semantic vocabulary, entity and observation identity, capability discovery, and loosely typed fields. Section 6 covers all four, in that order of importance.

Section 4.10 answers a question the findings raise again and again, because the answer decides what to do about them. Is a given problem the FROST-Server implementation, the specific deployment, or the standard? The three have different remedies on different timescales, and confusing them is how integration teams end up absorbing other people's bugs into their own client.

The strongest argument for STA came out of the second integration. We pointed our reader at an unfamiliar organisation's server: 46 observed properties, 460 datastreams, and a domain of pump telemetry, seismics and air quality with no overlap at all with our own waste containers. Working replication took days, with no bilateral agreement, no vendor SDK, no schema exchange, and no conversation with the other party's engineers. Every vendor API integration we have done cost more than that. Section 7 makes that case in full, together with the honest caveats.

Figure 1. The translation layer, its sources and its targets. The Collaborall server appears twice, because we read from it and also write back into it.

The system has three seams: the adapter contract, the ingest core and the FROST writer.

The adapter contract. Every source produces the same canonical Thing, Datastream and Observation types, whether that source is a polled vendor API, a pushed webhook or another STA server. Canonical here means our own internal shape, one step removed from both the vendor and the standard. Nothing downstream knows where the data came from. Adding the Collaborall source needed no change to the ingest core, the validator or the writer, beyond loosening two type constraints (see section 4.6).

The ingest core. It resolves the STA entity chain, meaning the six linked records STA requires before an observation can exist: Thing, Location, Sensor, ObservedProperty, Datastream and FeatureOfInterest. It then writes the observations, records what it wrote, and moves the per-stream cursor forward. The cursor is a stored marker of the last reading we delivered successfully. Cursor and write-log together are what make the pipeline safe to restart and able to fill gaps.

The FROST writer. It upserts entities by name, meaning it looks each entity up by name and creates it only when it is absent. It checks for an existing observation before writing a new one, and it supports two transports for creating observations: HTTP POST, or MQTT publish over WebSockets, while entity upserts always go over HTTP. We can write several targets in parallel, which is what we call dual-write.

Table 3 is the whole mapping: one container arrives from the vendor API as an id, a position and a fill percentage, and becomes six entities plus a reading.

Table 3. Mapping one waste container onto the seven core STA entities.

The shared entities carry weight. Sensor and ObservedProperty are shared across containers on purpose, because there is one "Fill level" concept and not five hundred. That is correct modelling, and it is what raced and duplicated when we resolved datastreams in parallel (see section 4.4).

We carry vendor detail in properties, not in fields of our own invention. Each Thing gets vendor, vendor_native_id, clappform_source_system and first_seen_at, plus whatever the adapter adds. For the synthetic set that means area and synthetic. Nothing we carry forces a non-standard field onto the wire, which keeps the output readable by any STA client (see section 5.2).

Every entity name takes a configurable prefix. On a server we share with other testbed parties, ENTITY_NAME_PREFIX, for example CF_, goes in front of all seven names. Our upsert resolves entities by name, so without a prefix it would bind to someone else's Fill level (see section 4.3).

Each stored Observation carries two more fields, and STA has a place for both, so we did not have to invent one. resultQuality.validated_by and validation_version record that our validation stage ran and which version ran. And parameters.raw_observation_id holds the vendor's own identifier for that reading, which is the key that makes re-polling safe.

Table 3 is the destination, and we reach it differently from each of our two kinds of source. A vendor REST API has a model with nothing to do with STA, while another STA server already has the right model, so the job there is faithful copying.

The shapes below follow the source schemas and our mapping code. The values illustrate rather than reproduce captured traffic.

// GET /api/3/containerSlots
{ "id": 4821, "name": "Dorpsstraat 12 - rest", "customerKey": "SULO-NL",
  "contentType": 3, "site": "915", "container": 7734,
  "fillLevel": 24.0, "observedFillLevel": 27.0,
  "observedFillLevelTime": "2026-07-17T09:15:00Z", "lastModified": "2026-07-17T09:15:04Z" }

// GET /api/3/sites
{ "id": 915, "name": "Dorpsstraat", "areaName": "Centrum", "address": "Dorpsstraat 12",
  "postalCode": "8620", "city": "Nieuwpoort", "country": "BE",
  "latitude": 51.1298, "longitude": 2.7521 }

// GET /api/3/devices/linked
{ "id": 5567, "brand": "Sensoneo", "model": "Single Sensor 4", "serial": "SN-88213",
  "container": 7734, "installed": "2025-03-04T00:00:00Z",
  "status": { "signal": 74, "batteryPercentage": 91 } }

// GET /api/3/contentTypes
{ "id": 3, "name": "Restafval", "englishName": "Residual waste", "categoryName": "Household" }

// GET /api/3/fillLevels/containerSlot/4821?after=2026-07-17T08:00:00Z
{ "time": "2026-07-17T09:15:00Z", "fillLevel": "27", "containerSlot": 4821,
  "site": 915, "contentType": 3, "contentTypeName": "Restafval",
  "confidence": 100, "frozen": true }

// canonical.Thing
{ "VendorID": "sulo", "VendorNativeID": "4821",
  "Description": "SULO waste container slot \"Dorpsstraat 12 - rest\" for Restafval
                  at site \"Dorpsstraat\" (Dorpsstraat 12, 8620, Nieuwpoort)",
  "Location": { "Lon": 2.7521, "Lat": 51.1298 },      // note the swap, see below
  "Properties": {
    "reen_container_slot_id": "4821", "reen_container_slot_name": "Dorpsstraat 12 - rest",
    "reen_customer_key": "SULO-NL", "reen_content_type_id": "3",
    "reen_content_type_name": "Restafval", "reen_site_id": "915",
    "reen_container_id": "7734", "reen_site_name": "Dorpsstraat",
    "reen_site_area": "Centrum", "address": "Dorpsstraat 12",
    "postal_code": "8620", "city": "Nieuwpoort", "country": "BE",
    "reen_device_id": "5567", "device_serial": "SN-88213" } }

// canonical.Datastream
{ "ThingVendorNativeID": "4821", "ObservedProperty": "FILL_LEVEL", "Unit": "PERCENT",
  "Description": "Fill level (% of capacity) of Restafval in SULO container slot
                  Dorpsstraat 12 - rest",
  "ExpectedCadenceSeconds": 3600,
  "SensorMetadata": { "source_system": "reen", "model": "Single Sensor 4",
    "firmware_version": "unknown", "brand": "Sensoneo", "serial": "SN-88213",
    "reen_device_id": "5567", "signal_percent": "74", "battery_percent": "91" } }

// canonical.Observation
{ "ThingVendorNativeID": "4821", "ObservedProperty": "FILL_LEVEL",
  "PhenomenonTime": "2026-07-17T09:15:00Z", "ResultTime": "2026-07-17T09:15:00Z",
  "Result": 27, "RawObservationID": "4821@2026-07-17T09:15:00Z" }

// POST /Things
{ "name": "CF_Sulo Container 4821",
  "description": "SULO waste container slot \"Dorpsstraat 12 - rest\" for Restafval ...",
  "properties": { "vendor": "sulo", "vendor_native_id": "4821",
    "clappform_source_system": "smartsulo", "first_seen_at": "2026-07-17T09:17:40Z",
    "reen_container_slot_id": "4821", "city": "Nieuwpoort", "...": "..." } }

// POST /Things(96)/Locations
{ "name": "CF_Location of Sulo Container 4821 at 2026-07-17",
  "encodingType": "application/geo+json",
  "location": { "type": "Point", "coordinates": [2.7521, 51.1298] } }

// POST /Sensors
{ "name": "CF_Sulo fill-level sensor Single Sensor 4 unknown",
  "encodingType": "application/json",
  "metadata": { "brand": "Sensoneo", "model": "Single Sensor 4",
                "serial": "SN-88213", "...": "..." } }

// POST /ObservedProperties
{ "name": "CF_Fill level", "description": "Container fill level as a percentage of total capacity",
  "definition": "http://qudt.org/vocab/quantitykind/DimensionlessRatio" }

// POST /Datastreams
{ "name": "CF_Fill level — Sulo Container 4821",
  "unitOfMeasurement": { "name": "Percent", "symbol": "%",
    "definition": "http://www.opengis.net/def/uom/UCUM/0/%" },
  "observationType": "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement",
  "Thing": { "@iot.id": 96 }, "Sensor": { "@iot.id": 41 }, "ObservedProperty": { "@iot.id": 12 },
  "properties": { "expected_cadence_seconds": 3600 } }

// POST /Observations
{ "phenomenonTime": "2026-07-17T09:15:00Z", "resultTime": "2026-07-17T09:15:00Z",
  "result": 27,
  "FeatureOfInterest": { "@iot.id": 88 },
  "parameters": { "raw_observation_id": "4821@2026-07-17T09:15:00Z" },
  "resultQuality": { "validated_by": "clappform-translation-layer", "validation_version": "v1" } }

// GET /Things?$expand=Locations
{ "@iot.id": 31, "name": "Netatmo Indoor 24E124707E427318",
  "properties": { "deviceEUI": "24E124707E427318", "floor": 2 },
  "Locations": [ { "@iot.id": 44,
                   "location": { "type": "Point", "coordinates": [4.478, 51.925] } } ] }

// GET /Things(31)/Datastreams?$expand=Sensor,ObservedProperty
{ "@iot.id": 148, "name": "motion",
  "observationType": "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_TruthObservation",
  "unitOfMeasurement": { "name": "...", "symbol": null, "definition": "..." },
  "Sensor": { "@iot.id": 9, "name": "Netatmo module" },
  "ObservedProperty": { "@iot.id": 61, "name": "motion",
    "definition": "https://en.wikipedia.org/wiki/Motion_detection" } }

// GET /Datastreams(148)/Observations
//     ?$filter=phenomenonTime gt 2026-07-21T00:00:00Z&$orderby=phenomenonTime asc
{ "@iot.id": 90211, "phenomenonTime": "2026-07-22T07:31:00Z", "result": true }

{ "things": [ { "thing": {
      "VendorID": "collaborall", "VendorNativeID": "31",
      "Location": { "Lon": 4.478, "Lat": 51.925 },
      // note: 2 becomes "2", JSON-encoded
      "Properties": { "deviceEUI": "24E124707E427318", "floor": "2" } },
    "datastreams": [ {
      // ObservedProperty here is a stream key, not a fixed list member
      "ThingVendorNativeID": "31", "ObservedProperty": "motion",
      "Name": "motion", "ObservedPropertyName": "motion",
      "ObservedPropertyDefinition": "https://en.wikipedia.org/wiki/Motion_detection",
      "SensorName": "Netatmo module",
      "ObservationType": ".../OM_TruthObservation",
      "UnitName": "", "UnitSymbol": "", "UnitDefinition": "" } ] } ],      // "..." blanked
  "observations": [ {
      "ThingVendorNativeID": "31", "ObservedProperty": "motion",
      "PhenomenonTime": "2026-07-22T07:31:00Z",
      "ResultRaw": true } ] }                                              // verbatim, not a float

// POST /Things             -> { "name": "CF_Netatmo Indoor 24E124707E427318", ... }
// POST /Sensors            -> { "name": "CF_Netatmo module", ... }
// POST /ObservedProperties -> { "name": "CF_motion",
//                               "definition": "https://en.wikipedia.org/wiki/Motion_detection" }
// POST /Datastreams        -> { "name": "CF_motion",
//                               "observationType": ".../OM_TruthObservation",
//                        "unitOfMeasurement": { "name": "", "symbol": "",
//                                               "definition": "" } }
// POST /Observations
{ "phenomenonTime": "2026-07-22T07:31:00Z", "resultTime": "2026-07-22T07:31:00Z",
  "result": true,
  "parameters": { "raw_observation_id": "collaborall:148@2026-07-22T07:31:00Z" },
  "resultQuality": { "validated_by": "clappform-translation-layer", "validation_version": "v1" } }

The mapping above is static, whereas the sequence below runs on every poll, and it is where the reliability in section 2 comes from.

1. Poll from a stored cursor. The adapter asks the vendor for everything newer than the last observation we wrote successfully for each stream. The cursor lives in Postgres, so a restart resumes instead of starting over, and a multi-day outage becomes a catch-up rather than a hole.
2. Normalise to canonical types. The adapter emits Things, Datastreams and Observations in our own vocabulary. Everything downstream is vendor-agnostic from here on.
3. Validate. Drop readings with a missing or non-numeric result, drop fill levels outside 0-100, and reject timestamps implausibly far ahead of our clock. Stamp provenance on what survives. We count rejections by reason instead of discarding them without a record.
4. Resolve the entity chain. For each Thing and Datastream pair: upsert the Thing, then the Location under that Thing, then Sensor, ObservedProperty, Datastream and FeatureOfInterest. Each upsert is a cache lookup, then a GET ?$filter=name eq ..., then a POST only when the entity is absent.
5. Write each observation. Check our own write-log for this datastream and timestamp, ask the server for the same key, then either POST the observation or publish it over MQTT.
6. Record and advance. Write the log row, move the cursor. Only now do we treat the observation as delivered.
7. Repeat per target. With more than one target configured, steps 4 to 6 run per server, each with its own entity-id mapping. Server-assigned ids differ per server, so the only thing the two may share is the names.

Steps 4 and 5 are the expensive ones, and they are expensive for the same reason. A client cannot tell the server "create this if it does not already exist", so both steps read before they write. That is one extra round-trip per entity and one per observation, and section 6.2 is the argument for removing them.

Table 4. The three integrations side by side.

The server row is the most important one in that table, and we had it wrong at first. We assumed both targets were FROST-Server builds, and only one of them is: the Collaborall server is a separate, independently written STA implementation. Its maintainer confirmed this in the testbed discussions as the first non-FROST implementation in the programme, and the PHP/Laravel stack traces other participants received from it point the same way. Once we knew that, the work became an interoperability test rather than a single-server exercise. Every difference between the two targets in section 4 is a difference between two independent readings of one specification, not two configurations of one codebase. It is also the reason section 4.10 exists.

The third integration answers a different question from the first two. WBD and Collaborall are both STA servers, so both tested the standard against itself. SULO's platform is not an STA server and never will be: it is a proprietary vendor API, which is the situation most organisations are in when they consider adopting the standard. It therefore tested the claim in section 1.6 that a vendor's concerns can be kept behind one contract. It is also the only one of the three where the sensors are a real operational fleet rather than a test fixture. The fill levels are live municipal waste containers in Nieuwpoort, Belgium, across four waste fractions, reporting hourly. What that integration cost is in section 3; what it taught is in sections 4.1, 4.9 and 4.11.

We built a translation layer instead of connecting sensors to STA directly. Our sources are vendor APIs and ERP systems, not devices we can reprogram. A layer in between keeps vendor concerns behind one contract, meaning authentication, pagination, rate limits and wire formats, and it lets the same validated write path serve any future source.

We hand-rolled an STA client on net/http instead of taking a third-party STA library. This looked like the more expensive option and turned out to be the cheaper one. STA's wire format is plain JSON with OData-style query parameters, so the client is a few hundred lines. In return we kept the upsert and idempotency behaviour explicit and easy to inspect, and that is where the subtle problems in section 4 showed up. A library would have hidden them rather than prevented them. That is a finding about the standard in itself: it is small enough to implement directly.

We built a synthetic dummy adapter first. Deterministic, labelled fake containers, with properties.synthetic = "true" and names prefixed Dummy , let us exercise the whole chain against the live WBD server before any real sensor connector existed. No vendor dependency, no risk of polluting real records, and a one-command run that reviewers can reproduce. It paid for itself several times over, because we discovered every finding in sections 4.1 to 4.5 with synthetic data.

We made the reader a separate binary that pushes through our own ingest endpoint. The Collaborall reader, cmd/collaborall-reader, does not write to FROST directly. It reads the source and posts canonical envelopes to /ingest/collaborall, so copied data passes through the same validation, mapping and dual-write path as everything else. That gives us one code path to trust, and the reader still deploys on its own. It keeps its own cursor file, and a re-run is safe because the service's write-log removes duplicates.

The design set a reproducibility target: a contributor should be able to wire a second, non-SULO vendor adapter and produce valid observations in at most one working day. The SULO/REEN adapter is the honest test of it, because it is the first adapter written against the contract after the contract had stabilised.

The mapping work landed inside that budget and the shared write path needed no changes at all. Two things cost more than the mapping, and neither was the standard's doing:

• Reading the vendor's documentation. The published guide renders as a JavaScript single-page application with no content in the served HTML, so the machine-readable OpenAPI description behind it had to be found and read directly. Once located it was clear and complete, and it is where we found the forecast rule in section 4.11 that no amount of inspecting responses would have revealed.
• Deciding what a Thing is. The vendor models a container slot and a container separately, and attaches fill-level history to the slot, because a physical bin can be swapped out without breaking the measurement series. Mapping the container instead would have fragmented every stream on the first bin replacement. That is a modelling judgement about the vendor's domain, not a translation problem, and it is the kind of decision the standard cannot make for you.

The transferable point: the cost of an additional source sits in understanding the source, and it does not scale with the number of targets. That is the economic argument of section 7.3, measured once.

Two separate problems, both cheap to fix and expensive to diagnose.

Reads are authenticated too. We first attached Basic credentials to writes only. But our upsert starts with a GET ?$filter=name eq ... probe, and WBD authenticates every request, so entity resolution failed before we ever attempted a write. We now attach credentials to every request.

The wider point is that our upsert reads before it writes. Anything that authenticates reads differently from writes therefore breaks entity resolution before the write path is exercised at all. A credential scheme that looks fine for a write-only client is not necessarily fine for an upserting one, and every STA client that resolves entities by name is an upserting one.

An empty password returns HTTP 500, not 401. FROST-Server's BasicAuthFilter splits the decoded user:pass string on the colon and assumes two fields. An empty password produces a one-element array, an ArrayIndexOutOfBoundsException and a 500. A useful rule of thumb for operations: a 500 from this endpoint usually means a missing or empty credential rather than an outage.

This one cost far more than it should have. A 500 sends you to look at the server, and a 401 sends you to look at your own configuration. We spent time on the former when the answer was the latter. Section 6.4 returns to the same theme. A server whose error response carries the wrong meaning sends the client to the wrong place, and whoever is debugging pays the difference.

Status: closed. Both are fixed on our side. The 500 is a robustness issue in the FROST implementation rather than anything to do with the standard, which is verdict A territory in section 4.10, and worth reporting upstream.

We designed for an API key the vendor does not have. On the source side, our design assumed the SULO platform authenticated with a static API key, and specified a zero-downtime rotation procedure around two simultaneously valid keys, with the config to match. The vendor documentation, when it arrived, settled it differently: REEN has no API-key scheme at all. Credentials are the account's username and password, exchanged at a /session endpoint for a short-lived token sent as an X-Token header. These are the credentials used to sign in to the web interface.

The dead configuration was easy to delete. The more useful observation is what replaced the requirement. We wanted rotation without an outage; what we got was better, and from a different mechanism. Because the adapter holds a token rather than the credential, and re-authenticates automatically on any 401, a password can be rotated with no coordination at all: change it at the vendor, update the secret, and in-flight polling recovers by itself. The capability we had written a runbook for fell out of ordinary session handling.

The first lesson: do not specify a credential lifecycle before you have the vendor's authentication documentation. We wrote a procedure, a config surface and an operational runbook for a scheme that never existed, and unwound all three. The second: treat "session token, refreshed on 401" as the default assumption for a vendor REST API, because it is both more common than a static key and easier to operate. The one design consequence worth carrying: a token refresh must be single-flight. Datastreams are polled concurrently, so an expired token is discovered by many goroutines at the same instant, and without that guard every one of them opens its own session.

The WBD deployment is not a standard build. Things expose non-standard navigation properties, namely Configurations, ControlledDevices, DeviceSecrets and Projects, plus a restricted boolean on each entity. Together these form a project-scoped authorisation and device-management extension.

Our live concern was whether new entities would need an explicit Project association, or would be blocked by the restricted model, and neither turned out to apply: a standard STA POST of a new Thing was accepted and created with restricted: false, with no Project association required.

We only established that by probing: create a Thing, read it back, see what the server added. The extension is not announced anywhere a client can find it, which is the same gap section 6.3 asks to close, and it is not a hypothetical inconvenience. Another participant in the programme registered a batch of Things on this server before learning about the Project model, then had to go back and patch every one of them to attach a Project. That is the cost of an undiscoverable data model: work that has to be redone, not work that fails cleanly up front.

Status: closed for the testbed, still open as an integration-contract item. The lesson is that an integration contract must not assume a standard server, and that a deployment's authorisation model stays invisible until you probe it. Note also that customised is not a criticism here. The extensions this server carries, project scoping and device management, address real needs the core standard does not cover, and their maintainer has said they would like to see them standardised. The gap is discoverability, because a client cannot find out that they exist.

We write into a FROST-Server we share with other testbed parties. Our name-keyed upsert, a GET ?$filter=name eq 'X' followed by a create when absent, will bind to someone else's entity that happens to be named X, and our entities equally pollute their namespace. STA has no concept of ownership, tenancy or namespacing, so nothing prevents this.

The failure mode is worse than a naming clash. Our upsert treats a name match as identity. If another party has already created an ObservedProperty called Fill level, our datastreams attach themselves to their entity in silence, and inherit their definition, their units and their intent. No error appears, because from the server's point of view nothing unusual happened. Two organisations' data then share a concept neither of them agreed to share, and the only symptom is that someone's units look wrong much later.

Resolution. A configurable ENTITY_NAME_PREFIX, for example CF_, goes in front of every entity name the layer creates. The mapper exposes a single set of *EntityName methods, so the upsert probe and the created payload can never disagree about the name. Status: closed, though it is a workaround for a missing concept rather than a fix.

It is also a workaround with a real cost, which is why section 6.2 asks for the underlying capability. A prefix marks our entities as ours by making them match nobody else's. So if two parties should converge on one shared Fill level concept, prefixing stops that. We have traded accidental collision for guaranteed fragmentation. That is the right trade for a testbed and the wrong one for a national ecosystem, and it illustrates why namespacing belongs in a profile rather than in each participant's config file.

What we saw. On WBD, the ObservedProperty Fill level and the Sensor Dummy fill-level sensor DUMMY-1 1.0.0 each existed five times.

Why. The scheduler resolves Datastreams concurrently, one goroutine each, sharing one Writer. Entity upsert is check-then-act, meaning it looks first and creates second, with a gap in between where another request can slip through: cache miss, then FindByName, then POST when absent. Entities with per-container unique names, Thing, Location, Datastream and FeatureOfInterest, never collide. But Sensor and ObservedProperty are shared across all five datastreams, so five goroutines each saw the entity as absent and each posted a copy. Two defects sat underneath that: the in-memory entity cache was an unsynchronised map, which is a genuine data race, and check-create was not atomic per key.

Why the standard is implicated. FROST does not reject duplicate names, and it is right not to, because STA does not require entity names to be unique. So there is no 409, no error and no signal. The race writes wrong data, and our conflict-refetch branch never fired. This is the clearest example we found of a gap in the standard causing an implementation bug, and section 6.2 sets out what we would change.

Resolution. A mutex-guarded cache plus a single-flight group keyed on entity and name, so concurrent first-sight creates collapse into one FindByName and one POST. Single-flight means one shared lookup that every waiting caller reuses. This sits in the writer layer, independent of scheduler concurrency, and a race-detector regression test backs it. Status: closed. Cleaning up the redundant entities already created on WBD is still outstanding.

How we found it. No error appeared; what surfaced it was a routine log line reporting sta duplicate entity ... count=5 when a later run's name lookup returned five hits where it expected one. The duplicates had been created cleanly and accepted cleanly, and they would have stayed invisible indefinitely if the lookup had not been written to notice multiplicity. Had we written FindByName to take the first result, which is the obvious implementation, this defect would still be in production and we would not know.

We caught a silent data-integrity bug only because of a defensive check we had written for unrelated reasons. Any pipeline doing name-keyed upserts against STA without such a check has this bug and no way to see it. A server-side uniqueness constraint would have turned invisible corruption into a loud, immediate 409 on the very first run (see section 6.2).

Impact. Low functional risk, because name lookups resolve to the lowest @iot.id and the stored observations are correct. But it is real data-quality debt, and the map race could have caused a panic under load. It also does not grow after the first race, because later runs find the existing copies and reuse the lowest id. That is why it is easy to miss, because the symptom shows up once, early, and then stops.

A synchronous HTTP probe, GET .../Observations?$filter=phenomenonTime eq ..., gates every observation on both transports before we write it. During a multi-day catch-up over a high-latency path, that probe rather than the write became the bottleneck. It repeatedly hit the client timeout while MQTT publish kept pace.

Resolution. The FROST HTTP timeout is now configurable, FROST_HTTP_TIMEOUT_SECONDS, default 15 seconds, so it can absorb slow paths. Status: closed.

On the MQTT path the write itself is asynchronous and fast, while the safety check in front of it is synchronous and slow. The slower protocol throttles the faster transport, and adding MQTT capacity changes nothing, because requests queue in front of the probe. Under burst load the probe hit the client timeout while publishes kept pace, which looks like a transport failure and is not one.

The optimisation, and a better version of it than we had planned. We already knew that when the stored cursor guarantees an observation is new, the probe is redundant and we could skip it. Hylke van der Schaaf described a fuller pattern in discussion #22: run one thread for non-observation entities and one thread per datastream for observations, ask once per datastream what its latest observation is, and from then on treat anything with a later timestamp as new and create it directly. Then upload in batches using data arrays rather than one request per reading. That replaces one probe per observation with one probe per datastream per run.

It also explains something we had recorded as unexplained. Inserting an observation takes a lock on its datastream, so running several threads at one datastream buys nothing, which is why adding concurrency did not move our throughput. Single-threading per datastream costs us nothing and removes the contention, so we are adopting the pattern.

One case still needs our own bookkeeping. That invariant holds for a source that only moves forward in time. Ours does not, because the vendor API mixes settled estimates with forecasts and can re-issue a corrected value for a timestamp we have already passed. For a back-dated correction, the newest stored reading tells us nothing, so the write-log stays, which is part of what section 6.2 asks the standard to remove.

A second implementer measured the same cost. After we posted this finding, the team running the TGV/BDJ connector reported their own numbers from three weeks of continuous operation across three STA targets in discussion #23. During a backlog replay of roughly 4,500 observations they measured the per-observation probe at 150 to 300 ms on a good connection, which made it the bottleneck by a wide margin, exactly as it did for us. They also report the one mitigation we had not tried: the dataArray batch extension cuts the write side, 500 observations per request instead of one POST each, but the probe cannot be batched, so on a replay path it still dominates. Their working arrangement is three-layered rather than one mechanism: seed the cursor from the server's latest observation at startup, write optimistically and treat a 409 as "already delivered", and fall back to the probe only when replaying a previously failed write.

The third layer carries the wider lesson. Optimistic 409 is the cheapest correct approach, and it only works if the server enforces uniqueness and says so with that status. Nothing in the specification requires either, and no server documents which it does, so a client cannot adopt the cheap path without testing each target by hand. The probe is one cost, and the other is that the efficient alternative is undiscoverable.

This one belongs to nobody's implementation. The probe exists because STA defines no idempotent write, and both servers behave correctly, so section 4.10 gives it neither verdict A nor verdict B. The specification is silent here rather than ambiguous, and that silence costs a round-trip per observation, paid by each producer separately.

What happened. We scoped the Collaborall integration expecting fill-level data and found none of it. What the server held instead was 460 datastreams of pump pressure and flow, indoor and outdoor air quality, CO₂, particulates, motion, noise, light, ground acceleration on three axes, and a full weather station. Our fill-level-only filter would have mapped zero of them.

The shapes did not fit our model either, in three ways.

• Non-numeric results. OM_TruthObservation results are booleans, such as motion, and OM_CountObservation results are integers, such as light_level. Our canonical Observation.Result was a float64, and our FROST payload typed result as float64.
• Arbitrary phenomena. Our ObservedProperty was a closed list with FillLevel as its only v1 member, and both the push endpoint and the mapper filtered on it.
• Placeholder and null units. Some unitOfMeasurement fields hold the string "...".

Resolution: generalise instead of special-casing. We added an optional passthrough mode alongside the existing fill-level mode.

• canonical.Observation gained ResultRaw json.RawMessage, which carries the verbatim STA result, and frost.Observation.Result became any. Numeric vendors are unaffected.
• The validator skips numeric range checks when ResultRaw is set, because a boolean has no range.
• canonical.Datastream gained optional Passthrough* fields: source names, the unit triple, observationType and sensor name. When they are present the mapper reproduces the source entity faithfully. When they are absent it falls back to the fill-level defaults, so existing adapters stay untouched.
• The push endpoint no longer filters on fill level. The observed-property string is now the per-Thing stream key.
• We blank placeholder units, "...", at the source boundary instead of passing them on.

Status: closed. We changed nothing in the ingest core, the state store, or the dual-write and MQTT paths, which is the adapter seam paying off.

Lesson. We over-specialised our model where STA had not. STA already had OM_TruthObservation, OM_CountObservation and an untyped result. We narrowed it and then had to widen it back. When you map onto a general standard, resist encoding your first use case's assumptions into the canonical layer. Read the places the specification leaves open as a list of the variation you have not met yet.

What we saw. POST /Sensors against Collaborall:

frost http 422: {"error":{"code":"STA-422",
                 "message":"The metadata field must be a string.",
                 "target":"metadata"}}

The identical Sensor payload succeeds against WBD.

Why. We serialise Sensor.metadata as an inline JSON object:

{ "encodingType": "application/json",
  "metadata": { "model": "DUMMY-1", "firmware_version": "1.0.0" } }

What the specification says. We read the normative text, because it determines whose defect this is. OGC 18-088, STA 1.1, Table 13, types metadata as "Any (depending on the value of the encodingType)", mandatory. Table 15 lists three encodingTypes, application/pdf, SensorML and text/html, and the accompanying prose states that "Other encodingTypes are permitted". It adds that the metadata property may contain either a URL to metadata content, or the metadata content itself, in the case of text/plain or other encodingTypes that can be represented as valid JSON.

An inline JSON object under encodingType: application/json therefore sits inside the specification, and nothing in the standard requires metadata to be a string. A server that rejects a JSON object with "The metadata field must be a string" enforces a constraint the specification does not state, which makes the rejection a server-side defect.

Where the constraint comes from. The rejecting server is not a FROST build at all, but an independent PHP/Laravel implementation (see section 1.5), and the message gives it away. "The metadata field must be a string." is the verbatim output of Laravel's built-in string validation rule. The constraint is almost certainly one line in a request-validation schema, 'metadata' => 'string', written by someone reading the three encodingTypes the specification enumerates, all of which do carry string-shaped content. That is an understandable place to land, and it is still narrower than what the standard permits. Section 4.10 works through what follows from that.

Where the standard still contributes. The same passage closes by saying that clients are responsible for whatever string parsing is needed to handle metadata content, and all three enumerated encodingTypes carry string-shaped content. An implementer reading that could reasonably type the column as a string. The declared type is Any, and the surrounding prose assumes a string. The mixed signal makes the mistake easy, and section 6.4 asks for the loose typing to be pinned down as well as the bug fixed.

Impact. Sensor registration fails, so the whole Datastream chain that references it fails. Our error classifier also treats the 422 as temporary and retries a payload the server will never accept.

Status: open. The plan of record is to serialise metadata as a JSON-encoded string, which satisfies "must be a string" while preserving the structure, to make the serialisation per-target since one server accepts objects and the other does not, to classify STA-422 as permanent rather than retryable, and to report the rejection to the operator.

This remains the most instructive interoperability finding in the testbed. A server can present itself as STA-conformant and still reject a payload the specification allows. You cannot take a conformance claim on trust, which is the argument both for testing against several implementations (see section 7.7) and for publishing a test suite that a candidate server must pass (see recommendation 3 in section 8.1). See also section 6.4 for the part of this the standard should tighten.

The Collaborall server is a well-formed STA server carrying data that is hard to consume.

• Duplicate ObservedProperty names with different @iot.ids and different definitions. Several distinct co2_levels, gauge_pressure, battery_level and temperature_indoor entries, with definitions drawn variously from QUDT, Wikipedia, DBpedia and CF conventions.
• Placeholder or null unitOfMeasurement values, "..." and null.
• One node maps to many datastreams, one ObservedProperty maps to many sensors, with no consistent naming convention across them.

None of this breaks STA, and all of it means a consumer cannot answer "give me all CO₂ readings in this area" without a human-curated mapping. We achieved interoperability of form. We did not achieve interoperability of meaning. This is the gap a national profile is best placed to close (see section 6.1), and in our view it is the highest-value thing Geonovum could add on top of the standard.

• phenomenonTime is polymorphic. The same field is either an instant, 2026-01-02T03:04:05Z, or an interval, start/end. Every consumer has to handle both. Ours parses it and takes the interval start.
• @iot.id order is not input order. Concurrent creation means server ids reflect completion order: 96 for DUMMY-0001, 97 for DUMMY-0003, 99 for DUMMY-0002, and so on. Harmless, since we resolve entities by name, but confusing when you reconcile by hand.
• MQTT publish is fire-and-forget. There is no acknowledgement and no server response, so we can only verify persistence with a REST read-back. That is correct behaviour, and it means the MQTT path cannot report its own success. Monitoring has to be built separately.
• Which OData functions a server implements is not discoverable. STA borrows its query syntax from OData, but not from one version of it. On the WBD server substringof('Sulo', name), the OData v3 spelling, works; contains(name, 'Sulo'), the v4 spelling that replaced it, returns 400. Nothing in the service document or the $metadata response says which dialect is in force, so a client either probes or hard-codes one server's answer. This is a concrete instance of the gap section 6.3 argues about: the query surface is uniform in shape but not in vocabulary, and the difference is only visible by trial.
• Auditing our own footprint on a shared server is harder than it should be, and that is our fault. Our entity names follow a different template per entity type, so no single name prefix finds all of one vendor's entities: a Thing is CF_Sulo Container 514419 but its Datastream is CF_Fill level — Sulo Container 514419, with the phenomenon first, a lowercase "level", and an em dash. Verifying a run meant six differently-shaped queries and produced two false negatives before we noticed. The name templates are fine for upsert, which is what we designed them for, and poor for audit, which we did not. We already write vendor and clappform_source_system on Things but on nothing else, and had we put that marker on every entity type, one query would answer "what did we write?". Section 5.2 makes the case for properties; this is the case for using it consistently.
• We recorded the server's observation ids but not its entity ids. Our state store had columns for the FROST @iot.id of each Thing and Datastream, and setters to populate them, and nothing ever called those setters. Only sta_observation_id was written, so a local row could not be joined to the entity it created without re-resolving it by name, which is the fragile path the previous point describes. Nobody noticed because name-keyed resolution works, which is the point: a reconciliation gap stays invisible until you need to reconcile, and we only needed to when auditing a partial run. Status: fixed. The ids are now written back when the entity chain resolves. Where several targets are configured the first one wins and a single id is kept, matching the convention already used for sta_observation_id; per-target bookkeeping stays deferred under ADR-004.

Almost every problem above arrived looking like "the standard is unclear" or "their server is broken", and those two diagnoses lead to different actions on different timescales. One team can fix a server bug in a week. A specification change takes years and needs a working group. A client workaround takes an afternoon and makes the problem permanent, so choosing wrongly is expensive. The Sensor.metadata rejection in section 4.7 serves as the worked example.

This finding arrived with the third integration, after we wrote section 4.10, so we get to use that section's framework on a case it did not consider. The verdict is verdict A, our implementation, and it is the most consequential defect we found, because unlike everything above it destroys data rather than duplicating or rejecting it.

The SULO waste-container fleet reports into the REEN platform, whose fill-level endpoint returns a mixture of three kinds of value. The vendor's own documentation is explicit about them: values that are frozen, meaning analysis is complete and the number will never change; values that are pending, still being refined as more data arrives; and values that are predicted, which are forecasts of the near future derived from historical trend. The documentation also states the rule that makes the three separable: a predicted value never carries a past timestamp, so a forecast is always dated in the future at the moment you read it.

So a single request for a container's fill-level history returns both measurements and forecasts, and the only thing separating them is that the forecasts are dated ahead of now.

Why that is dangerous rather than merely untidy. Our write path already rejects a future-dated observation: the validator flags it in_future against a clock-skew tolerance, and it never reaches the server, so that part was fine. The damage came from a second rule interacting with the first. Our poll cursor advances to the newest observation the cycle covered, and "covered" includes, by design, observations that were definitively rejected. A permanently invalid reading must not be re-fetched forever, so the cursor moves past it. in_future was classified as definitive.

Compose the two and the failure is silent and total:

1. REEN returns a forecast dated three days ahead, alongside today's real readings.
2. The validator rejects the forecast as in_future. Correct.
3. Because that rejection is definitive, the cursor advances to the forecast's timestamp.
4. Every genuine measurement arriving over the next three days is now older than the cursor, so it is discarded as before_cursor.
5. Three days of real data for that container are gone, no error is logged, and the stream looks healthy because the cursor is moving.

Each rule is defensible alone, and together they convert a vendor's forecast horizon into a data-loss window that the freshness watchdog does not catch, because the datastream is being polled successfully the whole time.

The fix, and where it belongs. The adapter discards any row newer than the current time before handing the batch to the shared write path. We put it in the vendor adapter rather than in the validator on purpose: "my source publishes forecasts in the same collection as measurements" is a fact about REEN, not about SensorThings, and the adapter layer exists to stop vendor quirks from becoming core behaviour. A regression test asserts that a future-dated row is dropped and that the cursor stays on the last real measurement.

What generalises. Two things, and neither is specific to this vendor.

• A cursor that advances past rejected records is a data-loss risk whenever the rejection reason can be caused by data you have not seen yet. The rule is safe for "malformed" and "out of range", which are properties of the record itself. It is unsafe for in_future, which is a property of the record relative to now, and time moves. Any implementation that batches "skip it" and "advance past it" into one decision should re-check it against that distinction.
• Ask what a historical endpoint contains before trusting its ordering. We would not have found this from the response alone, because a forecast and a measurement are the same JSON shape. It was in the prose of the vendor documentation, two sections away from the endpoint reference. A status or frozen field that distinguishes matured from projected values should be read as a filter you must apply, not as metadata you may keep.

For STA specifically this is an argument for section 6.2: had the observations carried a vendor-side identity and maturity marker that survived into the entity model, the distinction would have been machine-checkable at the boundary instead of inferred from a timestamp.

Four hours into the SULO run, 48 of 51 container slots had reached the server. We expected the remaining three to be the last stragglers of the proxy problem in section 4.1. They were not. The transport had fully recovered, and checking the three directly against the vendor API showed two unrelated causes, neither of them a transport fault:

The first two are correct behaviour: no sensor, no observations, nothing to publish. The third is a defect, and the mechanism behind it is more general than the one slot it caught.

Where it comes from. Our ingest path returns early when a cycle produces no accepted observations, before it resolves the entity chain. That is a deliberate efficiency: do not create a Thing, Location, Sensor and Datastream on a server for a stream that has nothing to say. The consequence is that a container is created in STA by its first observation, not by being discovered. Combine that with a first-sight cursor that looks back one hour, and any container whose sensor stopped reporting more than an hour before we first saw it is never published at all.

Why this is the wrong failure mode. The container that most needs attention is the one whose sensor has died, and that is the one that disappears. A consumer reading the server sees 48 containers and has no way to learn that three more exist: absence is indistinguishable from non-existence. Our own monitoring is fine, because the freshness watchdog counts a datastream that has been polled but never written as stale, so internally we know. The published data is incomplete, and nothing on the server says so. An asset inventory taken from the STA server would be missing the broken containers.

What we would change. Create the entity chain at discovery rather than on first observation. A Thing, its Location and an empty Datastream describe a real asset whether or not a reading has arrived, an empty Datastream being how a server says "this container exists and has told us nothing". Widening the first-sight lookback only moves the boundary and would still lose a sensor that died a week ago. We have not made the change yet, because it trades a silent omission for entity churn on sources with many dormant assets, and that trade should be a deployment choice rather than a default.

The transferable point, and it is not about this vendor. Publishing driven by data arrival, rather than by asset discovery, reshapes what your dataset means: it becomes "things that are working" rather than "things that exist". If a consumer is going to use the server as an inventory, and section 5.1's argument that the entity model fits a domain invites that, then the set of Things has to be complete independently of whether observations are flowing.

Once the third source was running steadily we set its per-datastream freshness expectation from the cadence we had measured: REEN publishes an estimate for each container every hour, so one hour went into the configuration. Our watchdog turns that into an alarm threshold of three times the cadence, giving three hours. Within a day it was reporting thirteen of fifty-one streams as stale.

None of the thirteen were stale. Checking each against the vendor API showed we held every reading the source exposed, and the flagged containers had not yet had a new estimate published. The lag between the hour a reading describes and the moment it becomes readable varies per container, and we measured it spread across the fleet like this:

The mistake is a category error, and it is easy to make. Two different quantities were both being called "how often does this update":

• Cadence is the interval between consecutive readings. It is a property of the sensor and its duty cycle, and for this fleet it is one hour.
• Publication lag is the delay before a reading becomes visible to a client. It is a property of the vendor's processing pipeline, and here it runs from minutes to five hours depending on the container.

A staleness alarm is asking "should I have heard something by now?", which is bounded by cadence plus lag. We fed it cadence alone, so it alarmed on the vendor's normal processing delay. The failure mode is worse than a wrong number: an alarm that fires on healthy streams gets muted, and then it is not there for the container that has stopped reporting.

What we changed. The expectation is now derived from the observed lag rather than the cadence, so the threshold clears the slowest normal publisher while still catching a dead sensor inside half a day. The measurement matters more than the value, since both quantities are per-deployment: another REEN customer, or another vendor, will have a different lag profile and copying our number would move the error.

Why this belongs in a report about a standard. STA carries phenomenonTime and resultTime, which describe when a thing was measured and when its result was produced. Neither says when the record became retrievable, and that third timestamp is the one a monitoring system needs. We could only recover it by polling and watching what appeared. A consumer reading the server after the fact cannot compute it at all. This is the same gap section 6.2 describes from the identity angle, seen from the freshness angle: the entity model records the measurement well and records almost nothing about the pipeline that delivered it.

A note on how we got this wrong twice. The one-hour figure came from sampling three containers and reading their median gap. That was right about cadence and silent about lag. We made the same shape of mistake earlier in the same integration, concluding from three slots that the fleet ran a single sensor model when a survey of all fifty-one found two. Both times a three-item sample produced a confident and incomplete answer, on an estate of fifty-one that one request enumerates in full. Where the population is that small, count it rather than sampling it.

We mapped a domain the standard's authors did not have in mind, municipal waste containers, onto Thing, Location, Sensor, ObservedProperty, Datastream, Observation and FeatureOfInterest, with no extensions and no compromises. A container is a Thing, its position is a Location, the fill sensor is a Sensor, "fill level" is an ObservedProperty, and the pairing is a Datastream. We then mapped pump telemetry, seismics and weather onto the same seven entities without touching the mapping layer. The model absorbed both without strain.

Every entity carries a free-form properties bag. We used it for vendor native ids, source-system tags, an area label, a synthetic marker, expected cadence and first-seen timestamps. STA models none of that, and we needed all of it. Nothing we carry forces a non-standard field onto the wire, so our data stays readable by any STA client while losing nothing from the source. This is the feature that makes adopting the standard a non-destructive decision. You do not have to throw away your vendor-specific data to conform.

$filter, $count, $expand, $orderby and $top over plain HTTP meant that:

• our entire upsert is GET ?$filter=name eq '<escaped>' followed by a POST, with no vendor API involved;
• verification is curl, reproducible by any reviewer with no tooling, for example GET /Things?$count=true&$filter=startswith(name,'Dummy');
• $expand let the reader inline Locations onto Things, and Sensor plus ObservedProperty onto Datastreams, so rebuilding the entity graph costs one request per level instead of one per related entity;
• $filter=phenomenonTime gt <cursor> with $orderby and $top is an incremental-sync primitive we got for free. It is the query a cursor-based reader needs, and we did not have to negotiate it with anyone;
• our client is a few hundred lines of net/http.

Compare that with a typical vendor telemetry API: bespoke pagination, bespoke filtering, a proprietary SDK, and no way to express "count the things whose name starts with X".

POST /Things(96)/Locations creates the Location and the association in one request, so there is no separate link call and no risk of an orphaned entity between two writes. It is a small feature that makes the write path simpler.

STA specifies a REST interface and an MQTT interface over the same entities. We switched observation writes from HTTP POST to MQTT publish over WebSockets with no change to the mapping layer, because the payload stays the same and only the pipe changes. Changing transport characteristics without touching your data model is unusual and valuable. Our MQTT run sustained a multi-day catch-up that HTTP was struggling with.

This is the strongest property of the lot. @iot.selfLink and navigation links on every entity mean a client can walk a server it has never seen, with no prior knowledge of its contents. Our collaborall-reader enumerates Things, follows each to its Datastreams with Sensor and ObservedProperty inlined, reads Observations per stream, and rebuilds the entity graph. It did that against a server whose domain we did not know, whose operators we never spoke to, and for which no documentation exists.

This is what turns STA from a data format into an integration strategy, and section 7 works through the consequences.

Because the same model describes both directions, the same client code serves both. Our internal/frost package holds a reader.go next to writer.go, sharing the same entity types. Copying one STA server into another therefore takes a small program. Federation, mirroring, aggregation and archival follow from the same code.

Observation.resultQuality and Observation.parameters gave us standard places for validated_by and validation_version provenance, and for the vendor idempotency key parameters.raw_observation_id, without polluting result or inventing a side channel. Our observations describe how we processed them, and any STA client can read that.

OM_Measurement, OM_TruthObservation, OM_CountObservation and friends, all with an untyped result. As section 4.6 recounts, the standard was more general than our canonical model, and that generality was correct.

Problem. ObservedProperty.definition is a free URI and unitOfMeasurement is free text. In real data (see section 4.8) that produces four different co2_levels ObservedProperties with definitions from four different vocabularies, and units written as "...". A consumer cannot reliably ask for "all CO₂ measurements in this region" across servers, or even within one server.

Why this comes first. It is the only item on the list that limits the standard even when every implementation is correct. Everything else here makes STA harder to implement. This one makes the result less useful however well you implement it. We achieved interoperability of form, and not of meaning, and meaning is what a data consumer wants.

It is also the only item anyone can fix without changing the specification, which gives it the most realistic route to being fixed at all.

Another team has now done it, and reported the cost. In discussion #24 the TGV/BDJ connector team pin every ObservedProperty.definition to a CF Standard Names URI and every unitOfMeasurement.symbol to a UCUM code, publishing the mapping they use for temperature, humidity, CO₂, air pressure, PM2.5, groundwater level, wind speed, precipitation and solar radiation. Their assessment of the effort was "maybe two hours to set up and zero maintenance since". That is a second party confirming the high-value, low-effort claim, and it makes the recommendation concrete: the work is picking the authority, not building machinery.

The same thread carries a caution worth passing on. A fixed parameter-to-unit table holds for atmospheric and hydrological quantities but breaks once engineering units enter, mass flow being the example given, and the durable form is a conversion table anchored on base units rather than a list of pairs. The related point is that an STA server is not the right place to hold that truth: what the standard should settle is how a unit is written, so that two servers spell the same unit the same way, leaving conversion to the client or a shared library. That is an argument for our proposal rather than against it, and it sharpens what to mandate.

Proposed change. At the standard level: require definition to resolve to a term in a declared vocabulary, and require UCUM for unitOfMeasurement.symbol. At the national-profile level, which is our recommendation to Geonovum: mandate a specific vocabulary per domain, require UCUM, and publish a profile that can be checked for conformance. This is the single highest-value thing a Dutch STA profile could specify, because the base standard leaves that layer open, and gemeenten, waterschappen and their suppliers gain from agreeing on it together.

These were two findings for us, at two different altitudes, and they share one root cause. STA defines no natural-key identity, so every writer has to implement check-then-act. At entity level that races. At observation level it forces a check before every write.

Problem, at entity level. STA specifies no uniqueness constraint on name or any other natural key, no server-side upsert, and no conditional create. So every integrator reinvents check-then-act: GET ?$filter=name eq ..., then POST. That is an extra round-trip per entity, and it is racy. We hit this and produced five copies of two entities (see section 4.4).

Problem, at observation level. For our source, a datastream and a timestamp together identify one reading, and nothing enforces that pairing. Re-polling, catching up or retrying after a timeout can all duplicate observations, and MQTT publish is fire-and-forget with no acknowledgement at all, so you cannot even tell whether your write landed. We built a Postgres write-log and a synchronous per-observation GET probe to compensate, and that probe became our throughput bottleneck during catch-up (see section 4.5).

We first wrote this up as though that pairing were a natural key in any deployment. It is not, and we did not know the counter-example until Hylke van der Schaaf, who maintains FROST-Server and sits on the STA standards working group, set it out in discussion #22 on the testbed board. Several observations on one datastream at the same phenomenonTime are common in the water domain, where a laboratory repeats an analysis on one sample with different parameters. He gives two further cases: two Things may each carry a Datastream of the same name, and two Things may share a name while sitting at different Locations.

That changes the ask, and it lands in the domain of the server we write to, since WBD is a waterschap. Uniqueness is a property of a use case rather than of the standard, so a specification that mandated it would break valid shapes rather than protect them. What we ask for below is therefore narrower than our first draft: not enforcement, but a way for a client to create without reading first, and a way to find out what a given server does enforce.

Why it matters. Two reasons, and the second is the serious one.

The first reason is cost, and it is the largest piece of infrastructure the standard pushes onto its clients. A write-log, a stored cursor and a check before every write is not a small amount of code, and every reliable STA producer has to build it.

The second is worse: the entity-level failure is silent. FROST does not reject duplicate names, and it is right not to, because STA does not require them to be unique. So there is no 409, no error and no signal, only duplicated data. The standard makes the naive implementation the incorrect one and gives the implementer no feedback that they got it wrong. Anyone writing to STA concurrently either has this bug, or has found and solved it themselves.

What the standard is already doing about it. STA v2 addresses the entity half directly. As Hylke van der Schaaf quotes in discussion #22, v2 states that the service usually generates the id of an entity, and that a service may also let the client specify one. The same capability came up from the other direction in discussion #12, where a participant asked for external identifiers as a first-class field and hylkevds described both the new definition attribute in v2 and client-defined ids as answers. That is the capability we needed and did not have. If we can supply the id, our upsert stops being check-then-act altogether. We derive an id from the vendor key, write at that id, and the primary key deduplicates, which client-side locking cannot guarantee. That removes the race and a round-trip per entity together.

Proposed change. Three asks remain, and they are about portability rather than enforcement.

1. Make client-specified ids discoverable. The v2 text says a service may allow them, which is the kind of optional behaviour a portable client has to establish by trial. A conformance class, or a statement in the landing document, turns that into something a client can read (see section 6.3).
2. Pair it with create-or-replace at a known id, so a client can be idempotent in one request rather than treating an already-exists error as a normal path.
3. For observations, keep any dedupe on a datastream and timestamp pair strictly opt-in and discoverable, because the water-domain case above shows a mandatory version would break valid data. Give the MQTT path an acknowledgement mechanism as well, so a publisher can tell a delivered write from a dropped one without a REST read-back.

And one route that needs no specification change at all. Hylke van der Schaaf points out in the same thread that a server administrator can add uniqueness constraints in the database, after talking to local domain experts, including constraints on fields inside the properties object, and that this project could have done so. That is the fastest remedy available to us. On the shared testbed instance a unique constraint on Sensor.name and ObservedProperty.name would have turned our silent duplication into a 409 on the first run.

That reframes the finding: on a server several parties write into, the constraints are a deployment decision that every writer depends on and none of them can see. A national profile is where that belongs, alongside the vocabulary work in section 6.1, and it is a second argument for the capabilities document in section 6.3, because an administrator's added constraints are the kind of local truth a client cannot guess.

Problem. We discovered everything by probing, one request at a time: which extensions exist, whether MQTT is available, which authentication scheme is required, whether duplicate names are rejected, what the page size is, and whether the server is a standard build or carries custom entities and an authorisation model (see section 4.2). None of that is discoverable from the service root, which lists collections and little else. Every one of those probes was a manual investigation.

Why it matters. It makes onboarding a human task, and it makes generic clients defensive and slow. OGC API - Features solved this with a required /conformance endpoint. It is also the cheapest item on this list to specify, which is why it sits above section 6.4 despite costing us less in absolute terms.

A second implementer reached the same conclusion from three servers at once. The TGV/BDJ team run FROST v1.1, Collaborall v1.1 and a Fraunhofer v2.0 server concurrently, and report in discussion #22 that they had to build an explicit write-capability discovery step: does this target support dataArray batching, does it accept client-specified ids, what does it do when handed a duplicate. All three behave differently and, in their words, there is currently no way to discover that programmatically. Our own list of probes came from two servers; theirs comes from three, including a v2.0 implementation, and it produced the same finding. Two independent teams building the same missing feature by hand is the argument for specifying it.

Proposed change. A required machine-readable capabilities document that declares supported conformance classes and extensions, transports, authentication schemes, limits, and any non-core entities.

Problem. Sensor.metadata is declared as Any, its shape is said to be governed by encodingType, only three encodingTypes are enumerated, and the same passage that permits others tells clients to perform string parsing. As section 4.7 records, one server took that as licence to require a string and rejected a payload the specification allows. The server carried the defect, and the standard carried the ambiguity behind it, and this item asks only for the second half.

The rejection then compounds the ambiguity, because STA-422 is a FROST-specific code and nothing in the standard tells a client whether a rejection is permanent or temporary. Ours classified the 422 as temporary and retried, indefinitely, a payload that server was never going to accept.

Why it matters. Implementers read a field declared Any alongside prose that assumes a string, and they diverge. Divergence in the core entity set breaks the assumption most adopters start with, that STA-conformant implies portable. Without a classifiable error model, each divergence becomes a retry bug as well as a compatibility bug.

Proposed change. State normatively how metadata is encoded when encodingType is a JSON media type, and reconcile the string-parsing sentence with the declared Any type. Either answer works, and the specification should not imply both. More broadly, audit the fields declared Any and give each a normative encoding per encodingType. Separately, define normative error codes for the common failure classes, with an explicit permanent-versus-temporary distinction, so clients can implement correct retry behaviour without per-server heuristics.

Three further gaps cost us less and are still worth reporting.

Bulk observation insert sits in an extension rather than in the core, so a portable client cannot assume it, and 1,400 observations meant 1,400 posts plus a check each. We first wrote this up as a capability gap. Hylke van der Schaaf uses data arrays and batch processing as his normal import path in discussion #22, so the facility works. The problem is the one running through this section, that a client cannot discover whether a given server offers it (see section 6.3).

There is no multi-tenancy or ownership model (see section 4.3). And phenomenonTime polymorphism forces every consumer to write the same defensive parser (see section 4.9).

We pointed our reader at sta-server.collaborall.net, an organisation we had no integration agreement with, whose data we had never seen, in a domain of pump telemetry, seismics and air quality with no overlap with our own. We had a base URL and a set of credentials.

We did not have documentation, a schema, an SDK, a data dictionary, a sample payload, an API changelog, or a single conversation with their engineers.

Working replication took days, and most of that was our own model being too narrow (see section 4.6) rather than the source being hard to read. For comparison, a typical vendor telemetry integration in our experience takes weeks: read the docs, get a sandbox key, reverse-engineer pagination, discover the rate limits the hard way, write bespoke parsing, then repeat all of it for the next vendor.

That gap, days against weeks, is the argument, and you can measure it.

A bespoke API means N consumers times M producers, so N times M integrations, each needing its own bilateral agreement, and STA collapses that to one. Publish once, and every existing STA client can consume you with no work on your side and no negotiation. The marginal cost of your second data consumer is close to zero, and you never have to know who they are.

For a gemeente, waterschap or provincie this is the decisive argument. You cannot list your future data consumers, so you cannot negotiate with them in advance, and a standard lets you serve the ones you have not met.

Section 3 answers anyone who argues that standards are expensive. The effort went into network access, authentication, server behaviour, concurrency and idempotency, and every one of those costs is identical whether you expose a bespoke API or an STA endpoint. Ingestion reliability is hard whatever you publish at the far end. The STA-specific part was the cheapest line in the table.

So the incremental cost of conforming is small and the interoperability upside is large. Bespoke costs the same and returns less.

Mature open-source implementations exist, and FROST-Server, which one of our targets runs, is one. Adopting STA as a publisher can mean deploying software rather than writing it. The build-versus-adopt calculation weighs deploying an existing server against designing, building, documenting, versioning and supporting an API forever.

The properties bag (see section 5.2) carried every vendor-specific field we had: native ids, source system, cadence, provenance and synthetic markers, with no loss and no cost to conformance. That answers the common objection, that the standard does not model our domain. Model the bulk in standard entities and carry the rest in properties. You lose nothing, and the bulk becomes interoperable.

Our production STA client is a few hundred lines of plain net/http (see section 5.3). No SDK dependency, no vendor lock-in, no version-upgrade treadmill. A standard you can implement in an afternoon from the specification is a standard with low adoption risk, and that is not true of most enterprise integration standards.

Adopters should go in knowing section 6. STA-conformant does not guarantee portability between servers. Test against more than one server implementation before you call a client done. We found a hard portability failure (see section 4.7) on the second server we touched, and we would have shipped it undetected had we tested against one only. Note also that the payload was one the specification allows, so the server, and not our client, was at fault. Expect to build idempotency yourself as well (see section 6.2).

Neither of those changes the conclusion, though both change your project plan, and day one is the cheapest time to learn them.

1. Specify a Dutch STA profile that closes the semantic gap (see section 6.1): a mandated vocabulary per domain for ObservedProperty.definition, UCUM for units, and a conformance checker. This is the highest value per unit of effort of anything on the list. The base standard leaves that layer open, and a national ecosystem gains from agreeing on it.
2. Add namespacing and constraint guidance for shared, multi-party servers (see sections 4.3 and 6.2): entity-name conventions, an ownership convention in properties, and which uniqueness constraints a deployment should set in its database and publish to the parties writing into it. Uniqueness cannot come from the standard, because it depends on the domain, so on a shared server it has to come from an agreement. A profile is where that agreement belongs.
3. Publish an interoperability test suite that a candidate server must pass, including the ambiguous cases: Sensor.metadata encoding, duplicate names, non-numeric result types and interval phenomenonTime. Section 4.7 argues for this more directly than anything else in this report. A server there rejected a payload the specification permits, and a published suite would have caught that before we did. Over-strict servers damage interoperability as much as permissive ones, and nothing currently tests for them.
4. Take section 6 to OGC as implementation feedback from a real multi-server deployment. Each item is concrete, cheap, and backed by an incident in this report.

1. Test against at least two independent server implementations before you call a client done (see section 7.7).
2. Budget for the operational envelope, not the mapping (see section 3), and note that you would pay that cost anyway (see section 7.3).
3. Build idempotency on the datastream and timestamp pair from day one, and store a cursor. The standard will not do it for you, and it is what makes catch-up and restart safe (see section 6.2).
4. Prefix your entity names on shared servers (see section 4.3).

Table 7. Standards referenced in this report.

All on the programme board at https://github.com/Geonovum/testbed-sensordata-2026/discussions.

Table 8. Testbed discussion threads referenced in this report.

These record the raw evidence behind this report and are contributed alongside it.

Table 9. Companion documents.

Two packages in rime were used during this testbed:

• rime-ingest — the STA-centric ingestion pipeline that connects to upstream providers, transforms payloads, and writes to STA servers; and
• rime-client — a browser-based STA client for browsing, visualising, and downloading data from any STA endpoint.

For each connected device, rime-ingest performs the following steps in sequence:

1. Connect to the provider — manage communication with an upstream service (e.g. poll an HTTP API or subscribe to an MQTT topic).
2. Decode the wire format — decode applied by the provider or transport.
3. Deserialize the payload — parse the message into an in-memory structure.
4. Decapsulate to the Thing message — extract the device-level payload from any provider-specific wrapper. The same physical Thing should yield equivalent messages regardless of which provider carries them (e.g. a different MQTT broker).
5. Decapsulate to Sensor messages — split the Thing payload into one or more logical sensor readings where a single device exposes multiple quantities.
6. Transform to STA entities — map each reading to STA Observation fields, assign canonical ObservedProperty and Datastream names, and attach the correct entity links (Thing, Sensor, Datastream).
7. Push to STA server(s) — write the resulting observations to one or more STA endpoints (in this testbed, both v1.1 and v2 servers).

Following a brief description of the sensor setup, this report focuses on the tensions encountered in applying the SensorThings API, the benefits of using STA. Altough this testbed required the deployment of STA v1.1, STA v2 is on the horizon and the implications of this revision are explored. Finally, the findings are condensed as recommendations.

The message that flows downstream from an IoT device is rarely ready to fit neatly into STA — or into any other system or standard, for that matter. IoT heterogeneity goes beyond the shape of the decoded payload: network managers must contend with variety along the entire network stack.

This challenge is not considered a consequence of applying STA, but rather a general practical reality of the IoT domain. TU Delft's approach was to develop an ingestion and transformation pipeline abstract enough to handle the variety of sensors, while focusing entirely on STA compatibility.

Implementors should be aware that STA by no means solves the wider heterogeneity problem. It does, however, offer a stable baseline, a robust data model — one that is, in our view, further strengthened in v2. That baseline makes it possible for developers to build and manage software products that handle back-end ingestion and management while remaining out-of-the-box STA compliant.

Recommendation: Treat STA as the integration target and an interoperability contract, not as a substitute for provider and model-specific management. Budget explicitly for an ingestion layer (whether rime-ingest, Node-RED, or any other middleware) that owns transport, identity, and payload transformation.

This issue was discussed on the testbed GitHub repository (#12).

This issue is partially addressed in STA v2.

STA assigns every entity a service-local identifier (@iot.id in v1.1, id in v2). That value uniquely identifies a record within one server, but it is not a durable external identity: it is typically server-generated, and the same physical device will receive a different @iot.id if recreated or published to another STA instance. The standard does not require a Universally Unique Identifier (UUID), DevEUI, MAC address, or similar key on Thing or Sensor. Without a project convention, it is therefore easy to create two Things (or Sensors) that represent the same physical device — with the confusion, metadata drift, and fragmented timeseries that follow.

In STA v1.1 the usual place for a durable key, if required, is an unconstrained entry in properties, for example on the Thing:

"properties": {
  "mac_address": "70:ee:50:7f:11:11"
}

or:

"properties": {
  "dev_eui": "24E124707E421111"
}

or:

"properties": {
  "network_station": "RO.TEB11"
}

In the TU Delft deployment, however, sensor identity mattered more than thing identity for ingestion. rime-ingest chooses decoders, deserializers, and parsers from the identity of the upstream sensing device — the component that produces the payload — not from the platform that hosts it. The implemented convention was therefore to place a stable, unique external key in the name field of the Sensor (MAC address for Netatmo stations, DevEUI for LoRa devices, station code for seismic instruments).

That choice has a modelling consequence. In STA, Thing and Sensor are not linked directly: each Datastream points to exactly one of each, and a single Sensor may be reused across many Datastreams (and thus, indirectly, across many Things). That reuse is how STA supports treating Sensor as a type or procedure — for example one shared “DHT22” Sensor linked to many devices. By putting a per-device UUID in Sensor.name, rime instead treated each Sensor as an instance identity. The pipeline can then resolve an incoming message to a unique Sensor (and from there to its Datastreams) without relying on a shared type catalogue. The trade-off is that Sensor no longer behaves as a reusable abstract type; it carries the same kind of instance-level uniqueness assignable to a Thing. That is a valid modelling choice, but it is a deliberate departure from the Sensor-as-type pattern that STA also permits.

The lack of standardised key names (dev_eui versus deveui versus DevEUI, and whether the key lives on Thing.properties, Sensor.properties, or Sensor.name) still undermines cross-system interoperability. STA v2 partially helps by adding an optional definition URI to most entities, including Thing and Sensor, intended as a link to an external authoritative identifier — but it still does not mandate a particular scheme.

Recommendation: Decide early whether durable identity lives primarily on the Thing, the Sensor, or both, and document a project-wide key convention (e.g. always dev_eui, mac_address, or network_station). Prefer keys that are already unique in the upstream network. If middleware routes on sensor identity — as rime-ingest does — put that key where the pipeline can query it reliably (Sensor.name, Sensor.properties, or v2 definition). Be explicit whether Sensor is modelled as a reusable type or as a per-device instance; mixing the two silently is a common source of duplicates. Track STA v2 definition for cross-service identity, but do not expect it alone to replace a documented convention.

STA v1.1 is a generic standard that imposes few limits on the user. That is likely a feature, not a bug — but users should be aware that, within certain domains, it may be desirable to further restrict the inputs of certain entity fields.

Consider, for example, the entity ObservedProperty, which specifies the phenomenon of an Observation and has the following fields:

name: CharacterString
definition: URI
description: CharacterString
properties: JSON Object [0..1]

Without enforced categories for any of these fields, a database can easily accumulate semantically identical objects under different names. STA is extensible, and additional enumerations can be imposed. The approach taken in rime-ingest was to define a set of canonical Datastreams that the transformation pipeline always refers to. Transformation therefore happens at object creation and is limited to canonical Datastream and ObservedProperty names.

Recommendation: Define a closed vocabulary of ObservedProperty and Datastream names (and, where possible, definition URIs) before ingestion begins. Enforce that vocabulary in the pipeline rather than relying on operators to choose consistent free-text labels.

STA does not impose restrictions on the creation of duplicate entities. This includes "partial" duplicates, where all fields except @iot.id are identical but entity linkages differ — for example, two ObservedProperty entities with identical fields linked to different Datastreams. It also includes full duplicates, such as identical Observations linked to the same Datastream.

Pushing duplicate Observations is a common problem when polling a server for new readings, and was encountered with the Netatmo HTTP API. The solution implemented in rime-ingest was to check consistently for object existence before writing to a STA server. To reduce the overhead of an existence check for polling type transports, the last message from a given Sensor is kept cached in memory and and the comparsion happens locally, reducing the need for a redunant round trip.

Recommendation: The implementor should be aware that duplicate prevention remains their responsibility: STA continues to permit duplicates in both v1 and v2. Prefer idempotent writes keyed on external identity and phenomenonTime (or an equivalent natural key), or accept duplication risk and carry out periodic de-duplication prunes.

Further to the previous issue, STA does not define what constitutes equivalence (equal to) or identity (is) between entities. Consider, for example, two Location entities with identical fields, linked to different Things. It is at the discretion of the implementor to decide whether those Locations are the same place reused, or two coincidentally identical records — and whether the ingestion layer should merge, reuse, or always create anew.

This decision cascades: equivalence rules for Sensor, ObservedProperty, Location, and FeatureOfInterest determine how aggressively the pipeline deduplicates, and how portable the resulting graph is across STA servers.

Recommendation: Document explicit equivalence rules per entity type before go-live (e.g. "Locations are equal if coordinates and name match"; "ObservedProperties are equal if definition URI matches"). Implement those rules in middleware; do not leave them implicit in operator behaviour.

This issue is resolved in STA v2.

The FeatureOfInterest entity represents the feature being observed by a sensing entity. For an indoor thermometer, this would be the air inside a given room. Each Observation is linked to this common FeatureOfInterest. STA v1.1 links FeatureOfInterest exclusively to Observations, which creates a problem for the ingestion pipeline: there is no way to infer which FeatureOfInterest should be assigned to incoming Observations.

Consider a typical multi-sensor setup, where a server receives messages from several sensors. Each message usually contains some identifying key (e.g. the MAC address of the device) that can be used to resolve the Sensor in the database — provided the identity issue in discussed earlier is resolved. From the Sensor, the system can trace the Datastreams, which generally gives enough information to parse a message and assign an Observation to the appropriate Datastream. When creating that Observation, however, there is no way to query the STA system for the appropriate FeatureOfInterest, because STA v1.1 defines no relationship between FeatureOfInterest and any non-Observation entity.

In this testbed, when applying STA v1.1, there was no practical solution other than relying on server-side convenience behaviour, which auto-generates a FeatureOfInterest from the Location of the Thing when none is supplied with the Observation. Note that this auto-generation is a server feature, concindentally implemented by both Topic 1 participants, not a requirement of the STA standard itself.

This issue is resolved in the upcoming STA v2, which modifies the FeatureOfInterest entity extensively: renaming it to Feature and linking it to Datastream as well as to Observation.

Recommendation: On v1.1 deployments, decide deliberately whether to accept auto-generated FeaturesOfInterest or to maintain an out-of-band mapping (Thing/Datastream → FeatureOfInterest) in middleware. Prefer STA v2 where Feature–Datastream linkage is desirable.

All Things in TU Delft's testbed contribution are single physical objects that contain multiple sensors and are, for practical purposes, indivisible. This includes, for example, the consumer-grade Netatmo, which measures temperature, humidity, and pressure in one sealed unit. That differs from arrangements where sensors are wired to a transmitter or data logger and may be removed, added, or swapped.

From a modelling perspective, for the sealed multi-sensor case it is acceptable to treat the device as a single sensing object with multiple Datastreams (cardinality 1..*).

Recommendation: Model sealed multi-parameter devices as one Thing / one Sensor with many Datastreams. Reserve a finer Thing/Sensor split for deployments where sensing elements are physically swappable or independently calibrated.

Things need not be physical objects; they include entities "that are capable of being identified and integrated into communication networks". This raises the question of whether a virtual Thing may have a virtual Location, or whether it is rather the absence of a Location that makes a Thing virtual. Irrespective of that question, it remains unclear in which situations modelling a virtual Thing is beneficial and if the differentation should be made explicitly or implicitly.

Recommendation: Introduce virtual Things only when they clarify a real operational role (e.g. an algorithm, aggregator, or connector — see here). Avoid virtual entities that exist solely to fill schema slots; prefer documenting absence (no Location) over inventing placeholder Locations.

This issue was discussed on the testbed GitHub repository (#3).

It is common for sensors to transmit data that is not ready for consumption — for example, readings in voltages or counts. In the simplest cases, linear arithmetic converts such "raw" figures into "real" observations that represent the actual ObservedProperty. In other cases, complex non-linear functions may be applied to a timeseries with extensive post-processing to remove noise, producing an one or more entirely new timeseries.

In both cases, it can be reasonable to treat the initial measurement as the "real" measurement and the produced observations as derived. A counter- argument, made in the same discussion, is that all observations are themselves derivations. Irrespective of that position, it is not recommended to subclass the Observation entity (or any entity) with a convoluted type such as DerivedObservation.

Implementors are reminded that virtual Sensors can represent algorithms or procedures that produce new Datastreams. Such a virtual Sensor may be associated with the "raw" Datastream and produce "derived" Datastreams.

Recommendation: Decide early whether clients need both raw and derived series. If both are required, model derivation as a virtual Sensor (or documented procedure) producing separate Datastreams — not as Observation subclasses. Record the processing relationship in properties or provenance metadata so consumers can trace lineage.

This issue was discussed on the testbed GitHub repository (#15).

This issue is largely addressed in STA v2.

The Netatmo, Milesight, and Dragino sensors used in TU Delft's contribution observe every five to ten minutes. That regime lends itself to modelling Observations as point-in-time measurements, where result and phenomenonTime are scalar values. Conversely, the Kinemetrics sensor samples at 100 Hz. Creating one Observation per sample is both contrary to waveform practice and a significant load on the STA and database servers.

STA does allow the result of an Observation to be of type ANY. Each Kinemetrics Observation therefore uses an array-typed result, with each result holding five minutes of samples, down-sampled to 2 Hz for storage.

Different result types created issues for rime-client, which initially expected scalar results and could not unpack arrays. The client fix was relatively straightforward but incurs a small performance cost: the client must first probe the Datastream endpoint, infer the datatype from the response, and then choose the correct plotting function. That approach is incomplete — any other result shape (e.g. JSON objects) is also valid in STA.

Recommendation: For complex result types, adopt a fixed object structure with predictable keys, especially where plotting or export is required. Note that STA v2 introduces a resultType on a revised Datastream entity, which should become the preferred way to advertise result structure to clients.

Throughout this testbed, the term Connectors appeared in several discussions (e.g. #16). The term is taken from Fraunhofer's OpenCitySense extension, and represents a Thing that manages another Thing — usually a device. Besides the Connector entity, the OpenCitySense data model also embeds other virtual components of the IoT lifecycle, such as configurations and credentials.

The rime-ingest package takes a different approach: it does not extend the STA data model at all. Connectivity, credentials, and provider lifecycle are managed exclusively by the application layer.

Either way, the discussion reflects a desire to model aspects of the "IoT back-end", so that an extended STA could participate in more of the IoT lifecycle. As noted in earlier, middleware such as Node-RED is commonly involved in managing IoT systems. STA has no native connector concept, and does not appear designed to own that layer directly; the connector question therefore remains open.

Recommendation: Implementors should treat this as an open design choice. Keeping credentials and transport configuration outside the STA graph is an option, unless there is a clear requirement for STA-visible management of devices. If Connectors (or equivalent) are modelled, document the boundary between STA entities and operational secrets carefully.

This issue was discussed on the testbed GitHub repository (#16, #10).

Presently STA v1 includes the STAplus and the WebSub Asynchronous Messaging Standard extensions. The former extends the data model to include concepts of ownership (see STAplus 1.0), especially with respect to multi-user contributions such as citizen science projects; while the latter is a technical extension allowing users to subscribe to a STA endpoint and receive notification when the result of an arbitrary query changes. These extensions were not used by TU Delft during this testbed.

In this testbed, the Fraunhofer STA implementation (FROST) came loaded with several non-standard management and quality-of-life extensions implemented as plugins. While TU Delft did not make direct use of any of these plugins due to time constraints, it was apparent during public discussions that some issues encountered during this testbed are at least partially addressed by them.

Of note was the Projects plugin, which introduces Users, Roles, and Projects to the data model, facilitating the management of boundaries between various groups pushing to a common STA server — as was the case in this testbed. The OpenCitySense plugin on the other hand extended the data-model to allow for management of various aspects of the IoT such as sensor configurations and Connectors. OpenCitySense differs from the approach used by rime, which opted not to extend the data model in favour of a predominantly file-based, "GitOps" approach to management.

The absence of standardised management extensions is considered noteworthy, and throughout this testbed we perceived that each Topic 2 participant was developing their own management approaches. The approaches implemented by several partners appeared to have enough similarities to merit the idea that some common standard could serve the various deployments.

Recommendation: Standardised management extensions to the core STA could allow for the development of management systems, portals and interfaces that do away with the need for middleware and reduce deployment tensions. This issue is a SWG issue rather than an implementor's issue.