> ## Documentation Index
> Fetch the complete documentation index at: https://documentation.kodelabs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Collection

> How KODE OS collects sensor and non-sensor data from connected systems

> How KODE OS collects sensor and non-sensor data from your connected systems

Data collection is the continuous process of gathering data from connected external systems. Once devices and points are discovered and registered, KODE OS begins collecting their data according to configured schedules and protocols.

## Collection Methods

KODE OS supports two primary collection methods, determined by the integration's communication protocol and the external system's capabilities.

### Scheduled Pull (Polling)

KODE OS periodically connects to the external system to request the latest data. The polling interval is configurable per connector and per entity type.

```mermaid theme={null}
sequenceDiagram
    participant KODE as KODE OS
    participant Vendor as External System
    loop Every polling interval
        KODE->>Vendor: Request latest data
        Vendor-->>KODE: Return data payload
        KODE->>KODE: Process and store
    end
```

**Protocols used**: REST, SOAP, SDK, OpenSSL/TCP, SFTP/FTP, Modbus/BACnet

Each integration defines default, minimum, and maximum polling intervals for its entity types. You can adjust the interval within the allowed range from the connector's entity configuration.

### Real-Time Push

The external system sends data to KODE OS immediately when events occur, without KODE OS having to ask for it.

```mermaid theme={null}
sequenceDiagram
    participant Vendor as External System
    participant KODE as KODE OS
    Vendor->>KODE: Event occurs, push data
    KODE->>KODE: Process and store
    Vendor->>KODE: Another event, push data
    KODE->>KODE: Process and store
```

**Protocols used**: Webhooks, MQTT, WebSocket

Push-based integrations receive data as soon as it's available, resulting in lower latency compared to polling. However, the external system must be configured to send data to the correct KODE OS endpoint.

### Hybrid

Some integrations support both pull and push methods. For example, an integration might use webhooks for real-time point history data but polling for periodic alarm collection.

## Collection Modes

The data collection mode determines how KODE OS interprets the data it receives. Each integration's entities are configured with one of three modes:

### Snapshot

A single view of the current state at a specific moment in time. Each collection cycle captures the full current value of all points.

**Use case**: Occupancy counts, current temperature readings, door status

### Delta

Only the changes in value since the last collection are reported. KODE OS tracks what has changed between polls and stores only the differences.

**Use case**: Controller data from BMS systems like Niagara, where only changed values need to be transmitted to reduce data volume

### Historical

A complete log of past values over a time range. The external system provides timestamped historical data, which KODE OS ingests in bulk.

**Use case**: Historical backfill, energy meter readings, audit logs

## Data Types

### Sensor Data

Time-series data collected from physical sensors on devices. These are measurable, often timestamped values pulled from device points.

| Entity            | Description                 | Example                      |
| ----------------- | --------------------------- | ---------------------------- |
| **Point History** | Time-series sensor readings | Temperature: 22.5°C at 14:30 |

### Non-Sensor Data

Structured data from external systems that is not generated by a physical sensor but is important for building operations.

| Entity            | Description                       | Example                                         |
| ----------------- | --------------------------------- | ----------------------------------------------- |
| **Alarm**         | Alert or fault notifications      | "High temperature alarm on AHU-1"               |
| **Audit Log**     | Access and security event records | "Badge swipe at Main Entrance at 08:15"         |
| **Work Order**    | Maintenance task records          | "Replace filter on RTU-3, due 2026-04-01"       |
| **Booking**       | Space reservation records         | "Conference Room A booked 10:00-11:00"          |
| **EV Charging**   | Charging session records          | "Station 5: 45 kWh delivered, session complete" |
| **Schedule Sync** | Schedule state synchronization    | "Weekday schedule active on AHU-2"              |

## Polling Intervals

Each integration defines its polling schedule with three parameters:

| Parameter            | Description                                                            |
| -------------------- | ---------------------------------------------------------------------- |
| **Default Interval** | The recommended polling frequency for the entity                       |
| **Minimum Interval** | The fastest allowed polling frequency (prevents API rate limit issues) |
| **Maximum Interval** | The slowest allowed polling frequency                                  |

Polling intervals are configured per entity type within a connector. For example, a Niagara connector might poll point history every 30 seconds but alarms every 5 minutes.

<Warning>
  Setting polling intervals below the minimum can cause rate limiting or authentication failures with the external system. Always stay within the allowed range.
</Warning>

## Collection Status

You can monitor the collection status of each connector from the Data Sources page. The status indicators show:

* **Active**: Data is being collected successfully on schedule
* **Error**: The last collection attempt failed -- check the connector logs for details
* **Paused**: Collection has been manually paused
* **Disconnected**: The connector cannot reach the external system

## Troubleshooting

<AccordionGroup>
  <Accordion title="Data is not being collected">
    First, verify the connector shows a successful test connection. Then check that the entities are enabled and the polling interval is configured. If the external system requires re-authentication (e.g., expired OAuth tokens), you may need to update the connector credentials.
  </Accordion>

  <Accordion title="Data appears delayed">
    For pull-based integrations, data freshness depends on the polling interval. If you need more frequent updates, reduce the polling interval (within the allowed minimum). For push-based integrations, verify that the external system's webhook/MQTT configuration is pointing to the correct KODE OS endpoint.
  </Accordion>

  <Accordion title="Duplicate data points">
    Some integrations may deliver the same data across overlapping collection cycles. KODE OS deduplicates data based on the point identifier and timestamp. If you see duplicates, check whether the connector's collection mode matches the external system's behavior (snapshot vs. delta).
  </Accordion>
</AccordionGroup>