Payload Format

Measured data from sensors are send and received as Chirp packets and immediately send to server via MQTT protocol. In case of connection lost the packets are buffered to internal memory and send in bulk as soon as connection with server is established again.

This format is used for trap and broadcast Chirp packets.

Overview

Format is designed with these requirements in mind:

  • Suitable for HW and FW with limited resources.

  • Flexibility for future and 3rd party HW and FW.

  • Sensors may measure multiple mediums.

  • Mediums may have multiple values - items (e.g. three-phase electric power).

  • It’s not possible to send everything independently. We have to put all measurements from one device into one message/request.

We need to transfer these informations in each message/request:

  • measured value or values,

  • time of measurement,

  • device ID (sensor type),

  • one or more attributes (for example medium).

Basic structure

Currently there are five supported formats. In all formats the messages are formatted in JSON. Messages can be sent individually or as an array of JSON objects.

{
    ...
}

[
    {
        ...
    },
    {
        ...
    }
]

Format 1

This format is implemented in all Homebase firmwares prior to January 2017 and should not be used in new devices.

 {
     "t": 1450435969,
     "f": 1,
     "a": {"d":101,"ch":3},
     "s": [
         ...
     ]
 }

where:

  • t is time when data were measured (Unix timestamp in seconds),

  • f is format version (1),

  • a are global attributes,

  • d is device ID (each sensor type has unique ID),

  • ch is channel to which sensor is paired (number 00-29),

  • s is subset, list of dictionaries with data itself.

Subset dictionary has format: {"a":{"m":1,"i":1},"v":54} where:

  • a is attribute for this data set,

  • m is medium number (i.e. water, gas, electricity, …),

  • i is item number (in case of three phase electricity, these are phase numbers),

  • v value itself.

Format 2

More compact version suitable for HW with limited memory. This version is obsolete and not used in new devices.

 {
     "t": 1485510949,
     "f": 2,
     "d": 1,
     "ch": 3,
     "s": [
         ...
     ]
 }

where:

  • t is time when data were measured (Unix timestamp in seconds),

  • f is format version (2),

  • d is device ID (each sensor type has unique ID),

  • ch is channel to which sensor is paired (number 00-29),

  • s is subset, list of dictionaries with data itself.

Subset dictionary has format: {"m": 1, "v": [1, 2, 3]} where:

  • m is medium number (i.e. water, gas, electricity, …),

  • v list of measured values (length of this list can be 1 to 15).

Format 3

This variant extends format 2 and adds new key u. This new key is mandatory.

 {
     "t": 1485510949,
     "f": 3,
     "d": 1,
     "ch": 3,
     "u": 23455,
     "s": [
         ...
     ]
 }

 {
     "t": 1485510949,
     "f": 3,
     "d": 1,
     "ch": null,
     "u": 23455,
     "s": [
         ...
     ]
 }

where:

  • u is sensor’s UNIQ (or unique serial number), 32b number or null.

  • ch is channel to which sensor is paired, number 00-29 or null.

  • all other keys have the same meaning as used in format 2.

Format 4

Format version 4 was created to also accommodate sensors directly connected to the Internet. These new devices can have following features:

  • Not paired to any Homebase.

  • Connected directly to Energomonitor’s (or customer’s) server.

  • Can work with OBIS codes (part of IEC 62056 standard protocol) instead of mediums.

  • Constrained connectivity, i.e. only UDP protocol is allowed, small maximum packet length, etc.

 {
     "t": 1524554310,
     "f": 4,
     "d": 13,
     "ch": null,
     "u": null,
     "s": [
         ...
     ],
     "o": [
         ...
     ]
 }

where t, f, and d and s have the same meaning as in format 3:

  • t [mandatory] is time when data were measured (Unix timestamp in seconds),

  • f [mandatory] is format version (4),

  • d [mandatory] is device ID (each sensor type has unique ID),

  • u [optional] is sensor’s UNIQ (or unique serial number), 32b number. Can be null (this has the same meaning as if the key was not present).

  • ch [optional] is channel to which sensor is paired, number 00-29. Can be null (this has the same meaning as if the key was not present)

  • s [optional] is subset, list of dictionaries with decoded Chirp packets.

Meaning of the new keys:

  • o [optional] is a dictionary with OBIS codes and values.

Subset s can contain dictionaries in following format:

{
    "m": 25,
    "v": [1, 2, null]
}

where m is medium (see below) and v is a list of values. Value can be number, string or null.

Subset o has following format:

{
    "1-0:41.8.0": "text-value",
    "1-0:61.8.0": 99,
    "1-0:61.8.0*1": 123,
    "1-0:61.8.0*2": 456,
    "1-0:77.3.0": null
}

where dictionary keys are valid OBIS codes encoded as a string in full form A-B:C.D.E*F or in shorter variant A-B:C.D.E and value is number, string or null.

Format 5

This variant extends format 4 and adds new keys e and v. The e key is mandatory. In addition, value of key u is now a string in HEX format, not a number.

 {
     "t": 1485510949,
     "f": 5,
     "d": 1,
     "ch": 3,
     "u": "00005B9F",
     "e": 0,
     "v": 0,
     "s": [
         ...
     ],
     "o": [
         ...
     ]
 }

 {
     "t": 1485510949,
     "f": 5,
     "d": 1,
     "ch": 3,
     "u": "00005B9F",
     "e": 0,
     "s": [
         ...
     ],
     "o": [
         ...
     ]
 }

where:

  • e [mandatory] is sensor’s ACC, number 0-255 incremented on every sensor transmittion

  • v [optional] is sensors’s REG, communication interface version 0-255. Can be null (this has the same meaning as if the key was not present).

  • u [optional] is sensor’s UNIQ (or unique serial number), 32b number in HEX format. Can be null (this has the same meaning as if the key was not present).

  • all other keys have the same meaning as used in format 4.

Additional note about ACC key

There are two ranges 0-15 or 0-255 of ACC number (key e), depending on sensor type.

Additional notes about channel and UNIQ keys

Formats 4 and 5 allow for channel ch and UNIQ u keys to be omitted from message. In this case the server should assume their values are set to null.

There are four possible scenarios:

  1. Both UNIQ and channel are set = data from new sensor with UNIQ and paired to some Homebase’s channel.

  2. UNIQ is null, channel is set = data from older sensor without UNIQ and paired to some channel.

  3. UNIQ is set, channel is null = broadcast data from new sensor with UNIQ, not paired to any Homebase.

  4. Both UNIQ and channel are null = data were send directly from measuring device (directly connected to network).

List of mediums and device IDs

See the list of medium IDs and the list of device IDs.

Examples for real sensors

Each example shows number of bytes (without white spaces) used by JSON message. The white spaces were added to examples for better readability.

Section Sensors supported by application shows all sensors and their mediums.

EOS-WM (Relaysense Water)

  • Channel: 22

  • Device number: 3

  • Time: Fri, 18 Dec 2015 11:00:41 GMT (timestamp 1450436441)

  • Data:

    • Medium 15 (pulse count): 39846

    • Medium 25 (coeficient imp/Kunit, ipu): 2000

    • Medium 14 (water, L/h): 1

    • Medium 11 (RSSI, dBm): -65

Format 1, 141 bytes

{
    "t": 1450436441,
    "f": 1,
    "a": {"ch": 22, "d": 3},
    "s": [
        {"a": {"m": 15}, "v": 39846},
        {"a": {"m": 25}, "v": 2000},
        {"a": {"m": 14}, "v": 1},
        {"a": {"m": 11}, "v": -65}
    ]
}

Format 2, 119 bytes

{
    "t": 1450436441,
    "f": 2,
    "ch": 22,
    "d": 3,
    "s": [
        {"m": 15, "v": [39846]},
        {"m": 25, "v": [2000]},
        {"m": 14, "v": [1]},
        {"m": 11, "v": [-65]}
    ]
}

Format 3, 128 bytes

{
    "t": 1450436441,
    "f": 3,
    "ch": 22,
    "u": null,
    "d": 3,
    "s": [
        {"m": 15, "v": [39846]},
        {"m": 25, "v": [2000]},
        {"m": 14, "v": [1]},
        {"m": 11, "v": [-65]}
    ]
}

Format 4, 128 bytes

{
    "t": 1450436441,
    "f": 4,
    "ch": 22,
    "u": null,
    "d": 3,
    "s": [
        {"m": 15, "v": [39846]},
        {"m": 25, "v": [2000]},
        {"m": 14, "v": [1]},
        {"m": 11, "v": [-65]}
    ]
}

Format 5, 144 bytes

{
    "t": 1450436441,
    "f": 5,
    "ch": 22,
    "u": null,
    "d": 3,
    "e": 0,
    "v": 0,
    "s": [
        {"m": 15, "v": [39846]},
        {"m": 25, "v": [2000]},
        {"m": 14, "v": [1]},
        {"m": 11, "v": [-65]}
    ]
}

EPM3 (Pulse Meter)

Example for pulse meter in broadcast mode. The sensor sends broadcast Chirp packets and all nearby Homebase will receive them. This sensors are not paired to any specific Homebase so there is no asigned channel number ("ch": null). The UNIQ can be used to identify these sensors.

  • Device number: 4

  • UNIQ: 83820545 or 04-FF0001

  • Time: Tue, 5 Sep 2017 12:09:53 GMP (timestamp 1504613393)

  • Data:

    • Medium 38 (battery voltage): 3.1 V

    • Medium 15 (pulse count): 150 and 1309

    • Medium 39 (pulse flow): 1231 and 98724

    • Medium 11 (RSSI, dBm): -49

Format 3, broadcast packet, 145 bytes

{
    "t": 1504613393,
    "f": 3,
    "ch": null,
    "u": 83820545,
    "d": 4,
    "s": [
        {"m": 38, "v": [3.1]},
        {"m": 15, "v": [150, 1309]},
        {"m": 39, "v": [1231, 98724]},
        {"m": 11, "v": [-49]}
    ]
}

Format 4, broadcast packet, 135 bytes

{
    "t": 1504613393,
    "f": 4,
    "u": 83820545,
    "d": 4,
    "s": [
        {"m": 38, "v": [3.1]},
        {"m": 15, "v": [150, 1309]},
        {"m": 39, "v": [1231, 98724]},
        {"m": 11, "v": [-49]}
    ]
}

Format 5, broadcast packet, 153 bytes

{
    "t": 1504613393,
    "f": 5,
    "u": "04FF0001",
    "d": 4,
    "e": 0,
    "s": [
        {"m": 38, "v": [3.1]},
        {"m": 15, "v": [150, 1309]},
        {"m": 39, "v": [1231, 98724]},
        {"m": 11, "v": [-49]}
    ]
}

ETM (Thermosense)

  • Channel: 5

  • Device number: 8

  • Time: Fri, 18 Dec 2015 10:52:50 GMT (timestamp 1450435970)

  • Data:

    • Medium 16 (temperature, degrees C): 17.50

    • Medium 11 (RSSI, dBm): -35

Format 1, 94 bytes

{
    "t": 1450435970,
    "f": 1,
    "a": {"ch": 5, "d": 8},
    "s": [
        {"a": {"m": 16}, "v": 17.5},
        {"a": {"m": 11}, "v": -35}
    ]
}

Format 2, 80 bytes

{
    "t": 1450435970,
    "f": 2,
    "ch": 5,
    "d": 8,
    "s": [
        {"m": 16, "v": [17.5]},
        {"m": 11, "v": [-35]}
    ]
}

Format 3, 89 bytes

{
    "t": 1450435970,
    "f": 3,
    "ch": 5,
    "u": null,
    "d": 8,
    "s": [
        {"m": 16, "v": [17.5]},
        {"m": 11, "v": [-35]}
    ]
}

Format 4, 89 bytes

{
    "t": 1450435970,
    "f": 4,
    "ch": 5,
    "u": null,
    "d": 8,
    "s": [
        {"m": 16, "v": [17.5]},
        {"m": 11, "v": [-35]}
    ]
}

Format 5, 105 bytes

{
    "t": 1450435970,
    "f": 5,
    "ch": 5,
    "u": null,
    "d": 8,
    "e": 0,
    "v": 0,
    "s": [
        {"m": 16, "v": [17.5]},
        {"m": 11, "v": [-35]}
    ]
}

ESO (Plugsense)

  • Channel: 4

  • Device number: 3

  • Time: Fri, 18 Dec 2015 11:00:47 GMT (timestamp 1450436441)

  • Data:

    • Medium 12 (power, Watts): 0

    • Medium 11 (RSSI, dBm): -57

Format 1, 91 bytes

{
    "t": 1450436447,
    "f": 1,
    "a": {"ch": 4, "d": 7},
    "s": [
        {"a": {"m": 12}, "v": 0},
        {"a": {"m": 11}, "v": -57}
    ]
}

Format 2, 77 bytes

{
    "t": 1450436447,
    "f": 2,
    "ch": 4,
    "d": 7,
    "s": [
        {"m": 12, "v": [0]},
        {"m": 11, "v": [-57]}
    ]
}

Format 3, 86 bytes

{
    "t": 1450436447,
    "f": 3,
    "ch": 4,
    "u": null,
    "d": 7,
    "s": [
        {"m": 12, "v": [0]},
        {"m": 11, "v": [-57]}
    ]
}

Format 4, 86 bytes

{
    "t": 1450436447,
    "f": 4,
    "ch": 4,
    "u": null,
    "d": 7,
    "s": [
        {"m": 12, "v": [0]},
        {"m": 11, "v": [-57]}
    ]
}

Format 5, 102 bytes

{
    "t": 1450436447,
    "f": 5,
    "ch": 4,
    "u": null,
    "d": 7,
    "e": 0,
    "v": 0,
    "s": [
        {"m": 12, "v": [0]},
        {"m": 11, "v": [-57]}
    ]
}

EOS-PS (Powersense)

  • Channel: 0

  • Device ID: 12

  • Time: Fri, 27 Jan 2017 10:16:03 GMT (timestamp 1485512163)

  • Data:

    • Medium 17 (voltage): 230 V

    • Medium 12 (power): 102, 98 W, third CT is not connected.

    • Medium 11 (RSSI): -55 dBm

Format 1, 179 bytes

{
    "t": 1485512163,
    "f": 1,
    "a": {"d": 12, "ch": 0},
    "s": [
        {"a": {"m": 17}, "v": 230},
        {"a": {"m": 12, "i": 1}, "v": 102},
        {"a": {"m": 12, "i": 2}, "v": 98},
        {"a": {"m": 12, "i": 3}, "v": 0},
        {"a": {"m": 11}, "v": -55}
    ]
}

Format 2, 105 bytes

{
    "t": 1485512163,
    "f": 2,
    "d": 12,
    "ch": 0,
    "s": [
        {"m": 17, "v": [230]},
        {"m": 12, "v": [102, 98, 0]},
        {"m": 11, "v": [-55]}
    ]
}

Format 3, 116 bytes

{
    "t": 1485512163,
    "f": 3,
    "d": 12,
    "ch": 0,
    "u": null,
    "s": [
        {"m": 17, "v": [230]},
        {"m": 12, "v": [102, 98, null]},
        {"m": 11, "v": [-55]}
    ]
}

Format 4, 116 bytes

{
    "t": 1485512163,
    "f": 4,
    "d": 12,
    "ch": 0,
    "u": null,
    "s": [
        {"m": 17, "v": [230]},
        {"m": 12, "v": [102, 98, null]},
        {"m": 11, "v": [-55]}
    ]
}

Format 5, 132 bytes

{
    "t": 1485512163,
    "f": 5,
    "d": 12,
    "ch": 0,
    "u": null,
    "e": 0,
    "v": 0,
    "s": [
        {"m": 17, "v": [230]},
        {"m": 12, "v": [102, 98, null]},
        {"m": 11, "v": [-55]}
    ]
}

Full Chirp Packet

Homebase can receive Chirp packet with maximum 30 bytes of payload (measured values without header, CRC and other part of Chirp packet). This payload can be divided into 2 subsets with 15 8bit unsigned integers, 1 subset with 14 16bit unsigned integers, or 1 subset with 7 32bit unsigned integers.

This example shows JSON message with maximum possible length. Original Chirp packet had payload with 2x15 8bit unsigned integer and 1 RSSI value.

Format 1, 983 bytes

{
    "t": 1234567890,
    "f": 1,
    "a": {"ch": 123, "d": 123},
    "s": [
        {"a": {"m": 123, "i": 1}, "v": 123},
        {"a": {"m": 123, "i": 2}, "v": 123},
        {"a": {"m": 123, "i": 3}, "v": 123},
        {"a": {"m": 123, "i": 4}, "v": 123},
        {"a": {"m": 123, "i": 5}, "v": 123},
        {"a": {"m": 123, "i": 6}, "v": 123},
        {"a": {"m": 123, "i": 7}, "v": 123},
        {"a": {"m": 123, "i": 8}, "v": 123},
        {"a": {"m": 123, "i": 9}, "v": 123},
        {"a": {"m": 123, "i": 10}, "v": 123},
        {"a": {"m": 123, "i": 11}, "v": 123},
        {"a": {"m": 123, "i": 12}, "v": 123},
        {"a": {"m": 123, "i": 13}, "v": 123},
        {"a": {"m": 123, "i": 14}, "v": 123},
        {"a": {"m": 123, "i": 15}, "v": 123},
        {"a": {"m": 123, "i": 1}, "v": 123},
        {"a": {"m": 123, "i": 2}, "v": 123},
        {"a": {"m": 123, "i": 3}, "v": 123},
        {"a": {"m": 123, "i": 4}, "v": 123},
        {"a": {"m": 123, "i": 5}, "v": 123},
        {"a": {"m": 123, "i": 6}, "v": 123},
        {"a": {"m": 123, "i": 7}, "v": 123},
        {"a": {"m": 123, "i": 8}, "v": 123},
        {"a": {"m": 123, "i": 9}, "v": 123},
        {"a": {"m": 123, "i": 10}, "v": 123},
        {"a": {"m": 123, "i": 11}, "v": 123},
        {"a": {"m": 123, "i": 12}, "v": 123},
        {"a": {"m": 123, "i": 13}, "v": 123},
        {"a": {"m": 123, "i": 14}, "v": 123},
        {"a": {"m": 123, "i": 15}, "v": 123},
        {"m": 123, "v": [-12]}
    ]
}

Format 2, 217 bytes

{
    "t": 1234567890,
    "f": 2,
    "ch": 123,
    "d": 123,
    "s": [
        {"m": 123, "v": [123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123]},
        {"m": 123, "v": [123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123]},
        {"m": 123, "v": [-12]}
    ]
}

These two examples above are theoretical but it demonstrates benefits of second format. Full Chirp packet can be encoded into text-oriented JSON with length of 217 bytes. Format 1 requires 983 bytes for the same packet.

Examples of Format 3 and 4 with JSON message with maximum possible length.

Format 3, 230 bytes

{
    "t": 1234567890,
    "f": 3,
    "ch": 123,
    "u": 12345678,
    "d": 123,
    "s": [
        {"m": 123, "v": [123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123]},
        {"m": 123, "v": [123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123]},
        {"m": 123, "v": [-12]}
    ]
}

Format 4, 234 bytes

{
    "t": 1234567890,
    "f": 4,
    "ch": 123,
    "u": 12345678,
    "d": 123,
    "s": [
        {"m": 123, "v": [123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123]},
        {"m": 123, "v": [123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123]},
        {"m": 123, "v": [-12]}
    ]
}

Format 5, 252 bytes

{
    "t": 1234567890,
    "f": 5,
    "ch": 123,
    "u": "00BC614E",
    "d": 123,
    "e": 123,
    "v": 123,
    "s": [
        {"m": 123, "v": [123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123]},
        {"m": 123, "v": [123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123]},
        {"m": 123, "v": [-12]}
    ]
}