CoAP Overview

An Overview of Constrained Application Protocol

Get Started Developing Today Create an Account

CoAP

This page is an overview of the Constrained Application Protocol (CoAP) and how Silver Spring Networks is using it to strengthen its presence in the Internet of Things (IoT) world. CoAP is a simple, RESTful web transfer protocol designed specifically for IoT and resource constrained equipment. It is for machine-to-machine applications such as smart energy and building automation.

CoAP has the advantage of being low cost and secure. It can work with 10KiB of RAM and 100 KiB of code space. At the same time its ease of use and platform independence make CoAP more attractive to users. CoAP supports different data types like text, HTML, XML, JSON among others. It can ask for messages to be returned in different formats too.

Silver Spring Networks’ platform supports the implementation of the CoAP protocol end to end through the NIC, or the CoAP library support on the Core Snappy Linux environment on the IoT Edge Router. The CoAP implementation includes a CoAP Gateway software that serves as a proxy between the client and the end device. A CoAP API completes the solution, providing CoAP interface for client applications.

One use case has a NIC as the CoAP proxy for sensors that are attached to it. The NIC also acts as a CoAP server for data that it provides, such as date, time, battery level, signal strength, or any other sensor data built into it. The intelligent endpoints and sensors are implemented as CoAP servers.

Sensors

Sensor devices communicate using CoAP through the NIC based developer kits. The NIC routes CoAP messages through the mesh via an access point (AP) and a CoAP proxy that is Gateway. A sensor device acts as a CoAP server and the application that communicates with it is a CoAP Client.

A CoAP Client can reach the sensor by sending CoAP requests to the CoAP Gateway using CoAP Gateway APIs. The NIC contains a CoAP proxy that communicates to an attached sensor via a UART interface using CoAP over HDLC. Embedded in the HDLC are CoAP calls that are handled by the sensor attached to the NIC.

A user CoAP client is required to communicate with an end device. A CoAP protocol command is sent by the client to the Gateway which acts as a CoAP proxy. This communication with end devices is facilitated through the CoAP Gateway API.

 


    Sensors and CoAP

     

    Gateway

    The CoAP Gateway’s (GW) job is to control traffic flow from Silver Spring Networks’ back office to the mesh network. Some of that traffic is CoAP, so GW acts as a CoAP proxy. Users establish a session with GW and can send proxy requests to GW. GW converts these requests and sends them out to the mesh so that they can reach devices. When a device responds, the response comes back through GW and is passed back to the original client.  

    Gateway as a CoAP Proxy

    Since the Gateway functions as a standard CoAP-to-CoAP proxy, usage mostly follows the CoAP standard

    Gateway's CoAP Proxy supports the following features:

    • Mesh-side security - CoAP Gateway encrypts traffic bound for SSN NIC-equipped devices using our proprietary encryption.
    • Address resolution - clients access devices by host name, abstracting away the dynamic routing on SSN's mesh
    • Proxy of basic CON/NON CoAP requests using the Proxy-Uri and Proxy-Scheme options. Responses to these requests will be proxied back through Gateway to the client that made the request.
    • Reliable retransmission for CON messages. Gateway tracks outbound (sent to the client) confirmable messages and expects to receive an ACK/RST response from the client. If the ACK/RST is not received Gateway will attempt to resend the confirmable message as described in the CoAP RFC.
    • Proxy CoAP observations of mesh devices that expose observable resources. Gateway allows the client to initiate an observation via a proxy request and will forward subsequent update notifications from the mesh CoAP server to the observing clients.
    • Aggregation of CoAP observations as described in Section 5 of the CoAP observation RFC. Gateway handles observe requests by starting an observation "on behalf of" a client, but Gateway "owns" this observation. This allows Gateway to use a single observation of a mesh resource to service multiple observation requests for the same device/resource combination. This reduces redundant network traffic and prevents burdening constrained network devices with multiple observers.
    • Response caching based on the indicated Max-Age option in the CoAP response.

     

    Proxy features NOT supported by Gateway:

    • Proxied CoAP ping requests to mesh devices. This is actually not possible based on the CoAP spec. A "CoAP ping" is achieved by sending an empty CON message to a server, which is an invalid combination and should result in an RST message from the server that indicates the server received the message. However, it is not possible to proxy an empty message because it cannot include the Proxy-Uri or Proxy-Scheme options (because it is empty). Instead, clients should send a proxied GET request for a mesh device's /.well-known/core resource to determine if it is responsive to CoAP requests.
    • Block-wise transfer of CoAP proxy messages. At the time of this writing no SSN mesh devices support CoAP blockwise transfer. Block-wise responses may also not work correctly with the Gateway's response caching mechanism.
    • DTLS security.  Gateway currently does not support DTLS either between the client and Gateway or between Gateway and the mesh device.

     

    At this time no Silver Spring mesh devices support CoAP blockwise transfer. Gateway does support block-wise transfer for its local resources.

    Client applications must delegate address resolution to the Gateway for devices on the SSN mesh, hence device-CoAP server URIs must specify only the device MAC ID.Gateway exposes an API that supports the CoAP-to-CoAP forward proxy for access to devices on the SSN mesh. Its features include:

    • Transparent forwarding of CoAP protocol messages to devices
    • Address resolution - clients access devices by host name, abstracting away the dynamic routing on SSNʹs mesh
    • Mesh-side security - CoAP Gateway encrypts traffic bound for Silver Spring Networks’ NIC-equipped devices using our proprietary encryption. It is planned to support end-to-end DTLS in a future release.

     

    Gateway Local CoAP Resources

    Gateway directly exposes a few local CoAP resources:

    • /.well-known/core - Allows discovery of local resources.
    • /echo - Gateway will send a response containing the same body as the request.
    • /help - Provides some information about using the Gateway's CoAP proxy, particularly how session management works.
    • /sessions - Used by Gateway CoAP Proxy clients to initiate an authenticated session with the Gateway. A session must be established prior to sending proxy requests. 

     

    Immediate Ack Note

    Gateway immediately ACKs requests to avoid a flood of retries.

     

    Security

    On the mesh side ("downstream”) of Silver Spring's proxy implementation of CoAP, which is in the Gateway:

    • 5683 is the unsecured port  – data is exchanged in clear text on the mesh
    • 4849 is the secure port - encrypted data is exchanged on the mesh

    On the interface between the CoAP client and the Gateway (“upstream”)

    • Firewall port 5683 is open as a UDP port because the CoAP proxy servers always listen on port 5683

     

    Session Management

    Gateway uses the concept of sessions to track client connections under a particular account.  The CoAP standard has no concept of sessions, so this is an additional requirement specific to SSN's CoAP Gateway. A CoAP session has the following lifecycle:

    • create a session
    • do proxy requests that keep the session active
    • close the session

    The following diagram shows the expected CoAP session lifecycle.

     

      Users can capture this CoAP usage pattern in a shell script, allowing them to build complex functionality. The following resembles a typical pattern:

      • Get JWT token
      • Establish a Gateway session with JWT token
      • One or more CoAP commands (such as a GET on a device)
      • Delete the Gateway session

      Gateway requires CoAP clients to establish a Gateway session before sending any proxy requests to mesh devices. Clients manage their sessions by posting account credentials, in the form of a JWT token, to the Gatewayʹs sessions resource.  Refer to the Security section of the API Overview for security, authentication, and how to obtain a token.

      In order to establish a session, you must POST your credentials to an endpoint on the Gateway itself. Session options are passed as query string parameters.

      Parameter Description
      security This is the security protocol used between the Gateway and the devices. The default will use SSN network (net_mgr V2) security. It can be disabled by setting it to 'none'.
      sticky If this is set to true, the account will only be allowed to create on session at a time on the Gateway.

       

      Once the session is established, the Gateway uses the source address and port of the client to track the session. This has the advantage of not requiring clients to provide a session token with every request. But, it has the disadvantage of causing the Gateway to lose track of sessions if the client's ip address changes (say they restart their laptop and get a new address) or if the client's port changes (because their client uses a new socket for a request, or because operations put a misconfigured UDP proxy in front of the Gateway). In other words, CoAP clients must use the same internet address and port used to establish the session for all subsequent requests.

      If you do not need multiple concurrent sessions on the Gateway (and you probably don't) then consider setting the query parameter sticky=true. If your client's socket address changes, you will have to re-authenticate, but the Gateway will re-user your previous session.

       

      Note in the diagram that when a CoAP proxy request is sent to the Gateway it is queued to be sent to the mesh. Gateway immediately responds to the client with a CoAP ACK, indicating that the message was received and processed and that the client does not need to retry sending the request. Eventually the message is removed from the queue and sent to the mesh, which will result in one of the following:

      • Success - the request makes it to the mesh device, the response comes back to Gateway and Gateway proxies the response to the client. This could include an error response from the server, but the communication was successful.
      • Timeout - Gateway does not receive a response from the mesh device in a timely manner. In this case Gateway will retry sending the request to the device on behalf of the client. Gateway implements the CoAP reliable retransmission scheme and will make four attempts to send the message, each with an increasing timeout. The base timeout for the request is 20 seconds and each subsequent retry roughly doubles the timeout period (with a random factor). If all four retries fail, then Gateway will report an error to the client.
        Important: Silver Spring strongly discourages retrying requests to Milli devices. Milli requests can be queued on the mesh network, and this can lead to very high latency, with negative impacts to communication pathways and unnecessary battery drain on the Milli devices as a result. Gateway's CoAP proxy respects this by never retrying requests to these devices. Milli CoAP requests are sent once with a higher than normal timeout (default: 60 seconds). If no response has been received by the time this timeout expires, then Gateway responds to the client with a GATEWAY_TIMEOUT_504 error code. Clients can retry, but you should be very careful not to be too aggressive when communicating with Milli devices.
      • Error - Gateway may report an error to the client, possibly from a timeout, or failure to resolve the target host, or policy limits preventing sending the request on behalf of the session.

      In any case, the eventual CoAP response indicates the completion of the request transaction, the ACK just means Gateway received the request.

      Since the Gateway functions as a standard CoAP-to-CoAP proxy, usage mostly follows the CoAP standard. Therefore it can be exercised with standard tools, for example the libcoap library.

       

      The Gateway's concept of a CoAP session is based on tracking the trusted IP address and port of a client that has authenticated. This means that if the client sends traffic from a different port or address Gateway will not recognize the client as having a session and will reject the request with an UNAUTHORIZED_401 (message code 129) response. When this occurs Gateway will log something like the following:

      2017-04-28T01:37:18.981+0000 [gateway-nio-2-3] WARN  SessionAwareHandler  - Could not retrieve session for: /73.158.59.40:42278. Anonymous sessions are not allowed, please supply account Id

       

      The implication is that it is very important for the client to consistently use the port it established its session with. Some common problems with this include:
      • Using the popular Firefox Copper CoAP plugin will not work. The plugin uses a different ephemeral port for every request and there's no way to configure it to use a constant port. This means Gateway will never recognize the client as having a session.
      • coap-client from libcoap must be used with the "-p" flag for every command. This flag specifies the local port that the client will use to send and receive traffic from. If it is not specified then coap-client also uses an ephemeral port for every command.
      • Firewalls that do UDP port forwarding cause subtle problems. The client can think it is sending on a stable port but the firewall may be changing that port behind the client's back to forward the request to Gateway. When this happens, Gateway is receiving traffic from whatever port the firewall decides and cannot match the port to one that the session was established on, which result in all requests being rejected.

       

       

      coap-client uses a tiny buffer to read the CoAP URL from the console. If you are having trouble creating a session, you may need to specify the query options separately. For example:

       

        CoAP Tools & Reference Applications

         

        CoAP API

        Gateway exposes a CoAP API to act as a CoAP-to-CoAP forward proxy  to allow clients to exchange CoAP messages with devices on the SSN mesh network. The API allows clients to issue CoAP requests to mesh devices that expose CoAP servers. These requests are proxied by the GW.

         

        Java CoAP Client

        A component of the Silver Sprint Networks developer program, the Java CoAP client is a standalone command line utility that can be used to communicate with any Silver Spring Networks based CoAP, IoT device; any of the Hardware Developer Kits available for  purchase on the Starfish Developer Portal.

        The JAVA CoAP Client is packaged as a jar and takes a configuration file, that defines network parameters and the IoT device, as well as command line options. Users can run the client on Linux, Windows, and MacOS.

         

        Windows CoAP Client

        A sample Windows CoAP Client application available in the SDK provides the capability to communicate with a CoAP sensor device on the SSN network. The CoAP client automatically retrieves devices available to the user and enables communication with those devices by issuing CoAP commands. Source code for the application is available in the Document Center.

         

        Windows CoAP Server

        A sample Windows CoAP Server application available in the SDK provides the capability for a third party to simulate a sensor device connected to the Developers Kit. Source code for the application is available in the Document Center.

        A Windows CoAP Server application can assume the role of a sensor device by connecting to a NIC based Dev Kit over UART. This is a useful test tool for simulating a sensor and developing an application that consumes sensor data through the Starfish Data Platform or the CoAP Gateway.

        CoAP Sensor Emulation

        The Windows CoAP Server is written in open-source C# .NET, which, in turn, leverages other open source components. Some of the underlying CoAP implementation is taken from CoAPSharp, a CoAP implementation provided by EXILANT Technologies.

        CoAP Server Functionality:

        • Resource simulation
        • Native NIC/Milli operations, including firmware image management and NIC/Milli-specific operations
        • Diagnostic operations, such as plugin validation and review of recent CoAP requests and responses
        • Tools for generating plugin shell code in the correct simulator format, and for extracting the core code for insertion into the final sensor server implementation