One use case has a NIC as the CoAP proxy for sensors that are attached to it. The NIC also acts as a CoAP server for data that it provides, such as date, time, battery level, signal strength, or any other sensor data built into it. The intelligent endpoints and sensors are implemented as CoAP servers.
Sensor devices communicate using CoAP through the NIC based developer kits. The NIC routes CoAP messages through the mesh via an access point (AP) and a CoAP proxy that is Gateway. A sensor device acts as a CoAP server and the application that communicates with it is a CoAP Client.
A CoAP Client can reach the sensor by sending CoAP requests to the CoAP Gateway using CoAP Gateway APIs. The NIC contains a CoAP proxy that communicates to an attached sensor via a UART interface using CoAP over HDLC. Embedded in the HDLC are CoAP calls that are handled by the sensor attached to the NIC.
A user CoAP client is required to communicate with an end device. A CoAP protocol command is sent by the client to the Gateway which acts as a CoAP proxy. This communication with end devices is facilitated through the CoAP Gateway API.
The CoAP Gateway’s (GW) job is to control traffic flow from Itron Networked Solutions’ back office to the mesh network. Some of that traffic is CoAP, so GW acts as a CoAP proxy. Users establish a session with GW and can send proxy requests to GW. GW converts these requests and sends them out to the mesh so that they can reach devices. When a device responds, the response comes back through GW and is passed back to the original client.
Gateway as a CoAP Proxy
Since the Gateway functions as a standard CoAP-to-CoAP proxy, usage mostly follows the CoAP standard.
Gateway's CoAP Proxy supports the following features:
- Mesh-side security - CoAP Gateway encrypts traffic bound for Itron Networked Solutions NIC-equipped devices using our proprietary encryption.
- Address resolution - clients access devices by host name, abstracting away the dynamic routing on Itron Networked Solutions' mesh
- Proxy of basic CON/NON CoAP requests using the Proxy-Uri and Proxy-Scheme options - responses to these requests will be proxied back through Gateway to the client that made the request.
- Reliable retransmission for CON messages - gateway tracks outbound (sent to the client) confirmable messages and expects to receive an ACK/RST response from the client. If the ACK/RST is not received Gateway will attempt to resend the confirmable message as described in the CoAP RFC.
- Proxy CoAP observations of mesh devices that expose observable resources - Gateway allows the client to initiate an observation via a proxy request and will forward subsequent update notifications from the mesh CoAP server to the observing clients.
- Aggregation of CoAP observations as described in Section 5 of the CoAP observation RFC - Gateway handles observe requests by starting an observation "on behalf of" a client, but Gateway "owns" this observation. This allows Gateway to use a single observation of a mesh resource to service multiple observation requests for the same device/resource combination. This reduces redundant network traffic and prevents burdening constrained network devices with multiple observers.
- Response caching based on the indicated Max-Age option in the CoAP response.
Proxy features NOT supported by Gateway:
- Proxied CoAP ping requests to mesh devices - this is actually not possible based on the CoAP spec. A "CoAP ping" is achieved by sending an empty CON message to a server, which is an invalid combination and should result in an RST message from the server that indicates the server received the message. However, it is not possible to proxy an empty message because it cannot include the Proxy-Uri or Proxy-Scheme options (because it is empty). Instead, clients should send a proxied GET request for a mesh device's /.well-known/core resource to determine if it is responsive to CoAP requests.
- Block-wise transfer of CoAP proxy messages - at the time of this writing no Itron Networked Solutions mesh devices support CoAP blockwise transfer. Block-wise responses may also not work correctly with the Gateway's response caching mechanism.
- DTLS security - Gateway currently does not support DTLS either between the client and Gateway or between Gateway and the mesh device.
At this time no Itron Networked Solutions mesh devices support CoAP blockwise transfer. Gateway does support block-wise transfer for its local resources.
Client applications must delegate address resolution to the Gateway for devices on the Itron Networked Solutions mesh, hence device-CoAP server URIs must specify only the device MAC ID.Gateway exposes an API that supports the CoAP-to-CoAP forward proxy for access to devices on the Itron Networked Solutions mesh. Its features include:
- Transparent forwarding of CoAP protocol messages to devices
- Address resolution - clients access devices by host name, abstracting away the dynamic routing on Itron Networked Solutions' mesh
- Mesh-side security - CoAP Gateway encrypts traffic bound for Itron Networked Solutions’ NIC-equipped devices using our proprietary encryption. It is planned to support end-to-end DTLS in a future release.
Gateway Local CoAP Resources
Gateway directly exposes a few local CoAP resources:
- /.well-known/core - Allows discovery of local resources.
- /echo - Gateway will send a response containing the same body as the request.
- /help - Provides some information about using the Gateway's CoAP proxy, particularly how session management works.
- /sessions - Used by Gateway CoAP Proxy clients to initiate an authenticated session with the Gateway. A session must be established prior to sending proxy requests.
Immediate ACK Note
Gateway immediately acknowledges requests to avoid a flood of retries.
On the mesh side ("downstream”) of Itron Networked Solutions' proxy implementation of CoAP, which is in the Gateway:
- 5683 is the unsecured port – data is exchanged in clear text on the mesh
- 4849 is the secure port - encrypted data is exchanged on the mesh
On the interface between the CoAP client and the Gateway (“upstream”)
- Firewall port 5683 is open as a UDP port because the CoAP proxy servers always listen on port 5683
Gateway uses the concept of sessions to track client connections under a particular account. The CoAP standard has no concept of sessions, so this is an additional requirement specific to Itron Networked Solutions' CoAP Gateway. A CoAP session has the following lifecycle:
- Create a session
- Do proxy requests that keep the session active
- Close the session
The following diagram shows the expected CoAP session lifecycle.
Users can capture this CoAP usage pattern in a shell script, allowing them to build complex functionality. The following resembles a typical pattern:
- Get JWT token
- Establish a Gateway session with JWT token
- One or more CoAP commands (such as a GET on a device)
- Delete the Gateway session
Gateway requires CoAP clients to establish a Gateway session before sending any proxy requests to mesh devices. Clients manage their sessions by posting account credentials, in the form of a JWT token, to the Gatewayʹs sessions resource. Refer to the Security section of the API Overview for security, authentication, and how to obtain a token.
In order to establish a session, you must
POST your credentials to an endpoint on the Gateway itself. Session options are passed as query string parameters.
This is the security protocol used between the Gateway and the devices. The default will use Itron Networked Solutions network (net_mgr V2) security. It can be disabled by setting it to 'none'.
If this is set to true, the account will only be allowed to create on session at a time on the Gateway.
Once the session is established, the Gateway uses the source address and port of the client to track the session. This has the advantage of not requiring clients to provide a session token with every request. But, it has the disadvantage of causing the Gateway to lose track of sessions if the client's ip address changes (say they restart their laptop and get a new address) or if the client's port changes (because their client uses a new socket for a request, or because operations put a misconfigured UDP proxy in front of the Gateway). In other words, CoAP clients must use the same internet address and port used to establish the session for all subsequent requests.
Gateway has a concept called "sticky sessions". When a client makes a second session request with the same account ID and the
sticky=true query parameter Gateway will see if any other session for that account already exists and will update it to use the IP address/port of the new session request. This is intended to allow the session to "follow" a client's IP address and also to keep the client from accidentally opening multiple sessions when it doesn't really need them.
However, this does not transfer existing CoAP operations such as observations or reliability re-transmissions. The CoAP protocol has a number of dependencies on client IP address for doing request/response matching and it's likely that changing client address in the middle of a CoAP operation will cause confusion.
If you do not need multiple concurrent sessions on the Gateway (and you probably don't) then consider setting the query parameter sticky=true. If your client's socket address changes, you will have to re-authenticate, but the Gateway will re-use your previous session.
Message Response, Retry, and Timeout Handling
Note in the diagram that when a CoAP proxy request is sent to the Gateway it is queued to be sent to the mesh. Gateway immediately responds to the client with a CoAP ACK, indicating that the message was received and processed and that the client does not need to retry sending the request. Eventually the message is removed from the queue and sent to the mesh, which will result in one of the following:
- Success - the request makes it to the mesh device, the response comes back to Gateway and Gateway proxies the response to the client. This could include an error response from the server, but the communication was successful.
- Timeout & Retries - Gateway does not receive a response from the mesh device in a timely manner. In this case Gateway will retry sending the request to the device on behalf of the client. Gateway implements the CoAP reliable retransmission scheme and will make four attempts to send the message, each with an increasing timeout. The base timeout for the request is 20 seconds and each subsequent retry roughly doubles the timeout period (with a random factor). If all four retries fail, then Gateway will report an error to the client.
Important for Milli: Itron Networked Solutions strongly discourages retrying requests to Milli devices. Milli requests are be queued on the mesh network, and this can lead to very high latency, with negative impacts to communication pathways and unnecessary battery drain on the Milli devices as a result. Gateway's CoAP proxy respects this by never retrying requests to these devices. Milli CoAP requests are sent once with a higher than normal timeout (default: 60 seconds). If no response has been received by the time this timeout expires, then Gateway responds to the client with a
GATEWAY_TIMEOUT_504 error code. Clients can retry, but you should be very careful not to be too aggressive when communicating with Milli devices.
The general guidance is that clients need to send one request at a time to milli based devices, and wait for a response or timeout from that command before sending again. Clients need to tolerate high latency when communicating with milli devices.
- Milli Request Serialization: The very long milli timeout is driven by the BAP retrying a request queued on the mesh. That timeout is multiplied for each pending request in the BAP queue - i.e. if there are 3 requests in the queue, the third one potentially has to wait for the other two to time out before it is processed. This results in an unpredictable timeout if more than one client is putting requests into the BAP queue. Gateway addresses this issue by strictly serializing requests to each milli to ensure that the BAP queue depth is predictable. Gateway will only allow one request per milli device at a time and will reject subsequent requests to that device until the current one completes. Requests made to "busy" milli devices will be rejected with a 5.03 SERVICE_UNAVAILABLE message with a payload indicating 'Request blocked by device serialization requirements, another request is enqueued or in process for this device.'
- Error - Gateway may report an error to the client, possibly from a timeout, or failure to resolve the target host, or policy limits preventing sending the request on behalf of the session.
In any case, the eventual CoAP response indicates the completion of the request transaction, the ACK just means Gateway received the request.
Since the Gateway functions as a standard CoAP-to-CoAP proxy, usage mostly follows the CoAP standard. Therefore it can be exercised with standard tools, for example the libcoap library.
The Gateway's concept of a CoAP session is based on tracking the trusted IP address and port of a client that has authenticated. This means that if the client sends traffic from a different port or address Gateway will not recognize the client as having a session and will reject the request with an
UNAUTHORIZED_401 (message code 129) response. When this occurs Gateway will log something like the following:
2017-04-28T01:37:18.981+0000 [gateway-nio-2-3] WARN SessionAwareHandler - Could not retrieve session for: /184.108.40.206:42278. Anonymous sessions are not allowed, please supply account Id
The implication is that it is very important for the client to consistently use the port it established its session with
. Some common problems with this include:
- Using the popular Firefox Copper CoAP plugin will not work. The plugin uses a different ephemeral port for every request and there's no way to configure it to use a constant port. This means Gateway will never recognize the client as having a session.
- coap-client from libcoap must be used with the "-p" flag for every command. This flag specifies the local port that the client will use to send and receive traffic from. If it is not specified then coap-client also uses an ephemeral port for every command.
- Firewalls that do UDP port forwarding cause subtle problems. The client can think it is sending on a stable port but the firewall may be changing that port behind the client's back to forward the request to Gateway. When this happens, Gateway is receiving traffic from whatever port the firewall decides and cannot match the port to one that the session was established on, which result in all requests being rejected.
CoAP-client uses a tiny buffer to read the CoAP URL from the console. If you are having trouble creating a session, you may need to specify the query options separately. For example:
Observation Aggregation by Gateway
The CoAP Gateway aggregates observations on behalf of multiple clients as described in section 5 of the CoAP observe RFC:
If two or more clients have registered their interest in a resource
with an intermediary, the intermediary MUST register itself only once
with the next hop and fan out the notifications it receives to all
registered clients. This relieves