Gateway WebSockets Network Architecture

Client-Server Terminology

(Adapted from Client Server Model)

The client–server model of computing is a distributed application structure that partitions tasks or workloads between the providers of a resource or service, called servers, and service requesters, called clients. Clients and servers typically run on separate hardware and communicate over a computer network. A server host runs one or more server programs which share their resources with clients. A client does not share any of its resources, but requests a server's content or service function. Clients therefore initiate communication sessions with servers which await incoming requests.

In the client–server model, the server is often designed to be a centralized system that serves many clients.

Clients and servers exchange messages in a request–response messaging pattern: The client sends a request, and the server returns a response.

WebSocket Protocol Introduction

(Adapted from WebSocket)

WebSocket is a protocol providing full-duplex communication channels over a single TCP connection. The WebSocket Protocol was standardized by the IETF as RFC 6455 in 2011, and the WebSocket API in Web IDL is being standardized by the W3C.

The intent of the WebSocket feature is to provide a relatively simple protocol that can coexist with HTTP and deploy HTTP infrastructure (such as proxies), and that is as close to TCP as is safe for use with such infrastructure, given security considerations. (From RFC6455: 2011)

The WebSocket Protocol is designed to be implemented in web browsers and web servers, but it can be used by any client or server application. The WebSocket Protocol is an independent TCP-based protocol. Its only relationship to HTTP is that its handshake is interpreted by HTTP servers as an Upgrade request. The WebSocket Protocol makes more interaction between a client (e.g., browser) and a server (e.g., website) possible, facilitating the real-time data transfer from and to the server. This is made possible by providing a standardized way for the server to send content to the browser without being solicited by the client, and allowing for messages to be passed back and forth while keeping the connection open. In this way a two-way (bi-directional) ongoing conversation can take place between a client and a server. The communications are done over TCP port numbers 80 or 443, which is of benefit to those environments which block non-web Internet connections using a firewall.

Unlike HTTP, the WebSocket protocol provides full-duplex communication. Additionally, WebSocket enables streams of messages on top of TCP. TCP alone deals with streams of bytes with no inherent concept of a message.

The WebSocket Protocol specification defines ws and wss as two new uniform resource identifier (URI) schemes that are used for unencrypted and encrypted connections, respectively.

The WebSocket Protocol is designed to supersede existing bidirectional communication technologies that use HTTP as a transport layer to benefit from existing infrastructure (proxies / filtering / authentication). Such technologies were implemented as trade-offs between efficiency and reliability because HTTP was not initially meant to be used for bidirectional communication. (See RFC6202: 2011)

The WebSocket Protocol attempts to address the goals of existing bidirectional HTTP technologies in the context of the existing HTTP infrastructure; as such, it is designed to work over HTTP ports 80 and 443 as well as to support HTTP proxies and intermediaries.

Typical Network Configuration

The IP Host server is located in the cloud, with a publicly addressable IP address, and is reachable from any client having either a public address or private LAN address. See Figure 2 for an illustration of a typical network configuration.

Details of Implementation

Successful Implementation Results

Each gateway establishes a secure long-lived (hours) WebSocket connection with the IP Host. Once established, communication is full duplex, with either side initiating messages to the other side. Application-level messaging uses a request-response paradigm in which JSON-formatted payloads emulate the ENGAGE Gateway’s request-response RESTful API originally developed for use over HTTP.

IP Host Requirements

The IP Host server must meet the following requirements:

• The IP Host server must be located at a publicly addressable IP address.

  • This requirement exists only for Alliance Partners wishing to host their WebSocket server on a separate subnet from the Gateway.

• The IP Host server must support a relatively large number of Gateway WebSocket clients communicating simultaneously.

• The IP Host must host a HTTPS server providing a limited number of URLs for use by the gateway.

• The IP Host server must contain a unique X.509 certificate and private key used to authenticate itself to gateway clients during TLS session establishment.

• The IP Host server must provide a X.509 root certificate which is signed by the root certificate authorities’ HMAC-SHA1.

  • The HMAC must use the Site Key which is established with the ENGAGE server during initial set-up of the site.

• The IP Host server must provide the ability for the root CA to be re-downloaded upon authentication failure.

• The IP Host must be able to authenticate Gateways using its Site Key and is able to maintain unique HTTP Basic Authentication credentials on a per-Gateway basis.

• The IP Host server must support WebSocket protocol version 13.

• The IP Host server must support the Allegion WebSocket sub-protocol.

Proof-of-Concept - WebSocket Server Provided

In another section is a Proof-of-Concept - WebSocket Server, developed by Allegion as a sample implementation method for Allegion Alliance Partner Integrators use. The source code and a short implementation tutorial have been made available for Alliance Partner Integrators to gain experience with, and understand the basic implementation of a WebSocket server as well as the Allegion WebSocket sub-protocol. While operating WebSockets over SSL/TLS (wss) is encrypted, it is the responsibility of the Alliance Partners to verify and validate any server implementation which they plan to use including the attached Proof-of-Concept Server.

Authentication Process

For Authentication, the IP Host shall host a HTTP server that serves at least the following fixed resource path:

• /engage/newCredentials

Gateways will attempt to connect to the “Server URL” path which is specified during Gateway Commissioning and authenticate themselves over HTTPS by issuing a POST to this resource path in order to establish credentials prior to any WebSocket connection with the host.

If the request is valid, the Host must respond with a 200 OK to this request and must contain a unique temporary 32 byte Basic Authentication password to be used during the WebSocket connection upgrade request. The IP Host must decline all WebSocket connection requests by HTTP clients not having the correct Basic Auth credentials.

Once credentials are established the Gateway opens a WebSocket connection. The Gateway only accepts TLS encrypted WebSocket server connections, and validates the IP Host server based on the TLS certificates supplied at the time of connection. The TLS certificate is validated against a root signing certificate which the Gateway attempts to fetch from the “CA Server URL” address supplied during Gateway commissioning.

NOTE: While all WebSocket communication with the server must be over TLS/SSL, the CA server must provide the root CA over unsecured HTTP.

The root signing certificate is retrieved with the same method as other ENGAGE devices. Please refer to the Schlage ENGAGE – Lock Root Certificate Update App Note documentation for more details.

Gateways will authenticate every 24 hours, or whenever the connection is (re)established. This authentication mimics that of any other ENGAGE edge device which communicates directly with a server and is documented in the Schlage ENGAGE – Alliance Partner Integration document (Lock Authentication (210 mode)).

Refer to the ENGAGE – Lock Root Certificate Update App Note documentation for details of when the Gateway attempts to re-retrieve the root CA.

WebSocket Connection Establishment

Each Gateway initiates a WebSocket connection with the IP Host (Server URL) using an HTTP request containing an Upgrade request header. Reference RFC6455: 2011 for specifics, or see Example of Connection Establishment below for an example of what the connection establishment looks like.

Example of Connection Establishment

For this example assume the IP Host HTTP server has IP address 137.112.45.2. Refer to the ENGAGE – Alliance Partner Integration document (Lock Authentication (210 mode)) for authentication specifics.

Client Request:

GET HTTP/1.1
Host: 137.112.45.2
Upgrade: WebSocket
Connection: Upgrade
Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw==
Sec-WebSocket-Protocol: engage.v1.gateway.allegion.com
Sec-WebSocket-Version: 13
Authorization: Basic QmFzZTY0IDMyYnl0ZSBSYW5kb20gUGFzc3dvcmQuLi4=

Server Response (success):

HTTP/1.1 101 Switching Protocols
Upgrade: WebSocket
Connection: Upgrade
Sec-WebSocket-Accept: AXmrc0sM1YLm3Gmm5OPpG2HaG23=
Sec-WebSocket-Protocol: engage.v1.gateway.allegion.com

If the IP Host is unable to authenticate to the Gateway, the IP Host must send an error message in response, at which time the Gateway tries to re-authenticate.

Server Response (failure):

HTTP/1.1
401 Unauthorized

WebSocket Protocol

Reference RFC6455: 2011 for specifics of the WebSocket Protocol.

ENGAGE WebSocket Sub-Protocol

Sub-protocols are application-level protocols, layered over the base WebSocket protocol that defines the structure and contents of a frame’s application data payload. The Gateway indicates which sub-protocols are acceptable to the client during connection establishment, at which time the sub-protocol is negotiated and selected by the server who echoes that value in its handshake to indicate which sub-protocol, if any, was selected.

The ENGAGE WebSocket Sub-Protocol is indicated in the connection request by:

Sec-WebSocket-Protocol: engage.v1.gateway.allegion.com

The ENGAGE sub-protocol defines two JSON message types to emulate the ENGAGE IP Mode request / response RESTful API originally developed for use over HTTP. The two defined message types are:

• ENGAGE Request Message

• ENGAGE Response Message

Since WebSocket communication is asynchronous, the IP Host can send an ENGAGE Request Message at any time. The receiver responds with an ENGAGE Response Message.

Additionally, the ENGAGE sub-protocol defines two additional message types that capitalize on the full-duplex, asynchronous communication capabilities of WebSockets:

• ENGAGE Event Subscription Message

• ENGAGE Event Message

The IP Host sends ENGAGE Event Subscription Messages to the Gateway to subscribe to Gateway and/or edge device events. If enabled, the Gateway will send ENGAGE Event Messages to the IP Host in real time.

ENGAGE Request Message Format

Request Messages generally emulate the request content previously conveyed by HTTP in the ENGAGE IP Mode Gateway’s RESTful API. A requestId tag has been added to permit the requestor to associate responses with previously sent requests. The requestId numbers are 32-bit values and are incremented with each new message request to prevent re-use.

The ENGAGE WebSocket Sub-Protocol for an ENGAGE Request Message is described below:

A JSON Schema is provided against which all ENGAGE Request Messages must be validated:

{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "engage.v1.gateway.allegion.com Request",
"description": "A request sent by the server to the client",
"type": "object",
"properties": {
    "requestId": {
        "description": "Unique id between all requests",
        "type": "integer"
    },
    "request": {
        "description": "Object that represents the contents of a request",
        "type": "object",
        "properties": {
            "method": {
                "description": "The HTTP-style method of the request",
                "type": "string"
            },
            "path": {
                "description": "The path which the request is targeted at",
                "type": "string"
            },
            "messageBody": {
                "description": "The additional body information associated with the request",
                "type": "string"
            },
            "optionalQueryStrings": {
                "description": "Optional strings that may be required",
                "type": "string"
            }
        },
        "required": [
            "method",
            "path",
            "messageBody"
        ]
    }
},
"required": [
    "requestId",
    "request"
]
}

ENGAGE Response Message Format

Response Messages generally emulate the response content previously conveyed by HTTP in the ENGAGE IP Mode Gateway’s RESTful API. The returned requestId tag value must match the requestId tag from original request. The status tag must contain a valid HTTP response code.

The ENGAGE WebSocket Sub-Protocol for an ENGAGE Response Message is described below:

A JSON Schema is provided against which all ENGAGE Response Messages must be validated:

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "title": "engage.v1.gateway.allegion.com Response",
    "description": "A response sent by the client to the server associated with a request",
    "type": "object",
    "properties": {
        "requestId": {
            "description": "Unique id matching the associated request's id",
            "type": "integer"
        },
        "response": {
            "description": "Object that represents the contents of a response",
            "type": "object",
            "properties": {
                "status": {
                    "description": "The HTTP-style status code of the response",
                    "type": "string"
                },
                "messageBody": {
                    "description": "The additional body information associated with the response",
                    "type": "string"
                }
            },
            "required": [
                "status",
                "messageBody"
            ]
        }
    },
    "required": [
        "requestId",
        "response"
    ]
}

ENGAGE Event Subscription Message Format

Event Subscription Messages are unacknowledged messages send from the IP Host to the Gateway to enable or disable real-time Gateway or edge device event messages. If the IP Host does not subscribe then the Gateway will default to not “eventing”.

The ENGAGE WebSocket Sub-Protocol for an ENGAGE Event Subscription Message is described below:

A JSON Schema is provided against which all ENGAGE Event Subscription Messages must be validated:

{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "engage.v1.gateway.allegion.com Event Subscription",
"description": "A message sent from the server to the client indicating the desired events to be received",
"type": "object",
"properties": {
    "subscriptionId": {
        "description": "Unique id between all event subscriptions",
        "type": "integer"
    },
    "subscription": {
        "description": "Array that represents the contents of a subscription",
        "type": "array",
        "items": [
            {
                "type": "object",
                "description": "Object that represents the Gateway specific subscription infomation",
                "properties": {
                    "source": {
                        "description": "Indicator of the type of subscription contents",
                        "type": "string",
                        "enum": [
                            "gateway"
                        ]
                    },
                    "eventingEnabled": {
                        "description": "Indicates whether events of this type are desired",
                        "type": "boolean"
                    },
                    "subscriptionBody": {
                        "description": "TDB. Will eventually contain more specific subscription info",
                        "type": "object"
                    }
                },
                "required" : [
                    "source",
                    "eventingEnabled",
                    "subscriptionBody"
                ]
            },
            {
                "type": "object",
                "description": "Object that represents the edgeDevice specific subscription infomation",
                "properties": {
                    "source": {
                        "description": "Indicator of the type of subscription contents",
                        "type": "string",
                        "enum": [
                            "edgeDevice"
                        ]
                    },
                    "eventingEnabled": {
                        "description": "Indicates whether events of this type are desired",
                        "type": "boolean"
                    },
                    "subscriptionBody": {
                        "description": "TDB. Will eventually contain more specific subscription info",
                        "type": "object"
                    }
                },
                "required" : [
                    "source",
                    "eventingEnabled",
                    "subscriptionBody"
                ]
            }
        ]
    }
},
"required": [
    "subscriptionId",
    "subscription"
]
}

ENGAGE Event Message Format

Event Messages are unacknowledged asynchronous messages sent from the Gateway to the IP Host that convey real-time Gateway or edge device status(s) or audit(s).

The ENGAGE WebSocket Sub-Protocol for an ENGAGE Event Message is described below:

A JSON Schema is provided against which all ENGAGE Event Messages must be validated:

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "title": "engage.v1.gateway.allegion.com Event",
    "description": "An event sent from the client asynchronously to the server",
    "type": "object",
    "properties": {
        "eventId": {
            "description": "Unique id between all events",
            "type": "integer"
        },
        "event": {
            "description": "Object that represents the contents of a event",
            "type": "object",
            "properties": {
                "eventType": {
                    "description": "Indicator of type of body contents",
                    "type": "string"
                },
                "source": {
                    "description": "Identifies the source of the event",
                    "type": "string"
                },
                "deviceId": {
                    "description": "Serial number of the source device",
                    "type": "string"
                },
                "eventBody": {
                    "description": "Contains the event message",
                    "type": "string"
                }
            },
            "required": [
                "eventType",
                "source",
                "deviceId",
                "eventBody"
            ]
        }
    },
    "required": [
        "eventId",
        "event"
    ]
}

Connection Timeout and Error Handling

The ENGAGE WebSocket connection is re-established automatically when a connection is dropped or does not exist. Either the IP Host or the Gateway may drop the WebSocket connection. The Gateway drops the connection if it has existed for more than 24 hours. The Gateway re-authenticates itself to the IP Host, and must receive a new temporary password, at least once every 24 hours.

TCP Keep Alive

Gateways acting as WebSocket clients will have a periodic TCP keep alive message enabled in order to preserve potential routing restrictions. This does not have any impact on the implementation of the WebSocket server because TCP keep alive messages are handled by the receiving TCP stack, not the application layer code. The rate of the TCP keep alive message can be configured in the Gateway as necessary.

WebSocket Ping Pong

Due to the nature of TCP connections, it may be desirable to know if a connection to a WebSocket client from a server is no longer routable. A server may optionally implement WebSocket application layer ping pong messages to solve this problem. All ENGAGE Gateways which are configured to act as a WebSocket client respond with a pong message when they receive a ping message as defined in the WebSocket specification. A WebSocket server can then force close a connection that fails to receive enough pong messages.

Broker Architecture

Other architectures based on WebSocket protocols are possible. For example, adding an HTTP server back-end to the WebSocket server results in a cloud based “broker” model that permits existing ENGAGE Alliance Partner IP Hosts acting as HTTP clients to monitor and control ENGAGE WebSocket enabled Gateways. In this configuration, both IP Host and Gateways can reside on their own private LANs. This architecture supports existing IP Host equipment that already has been configured to inter-operate with the ENGAGE Gateways RESTful APIs.

Figure 3: ENGAGE WebSocket Broker Concept

In the configuration shown in Figure 3, the IP Host issues HTTP client requests to the broker, which translates them into the appropriate ENGAGE WebSocket Sub-Protocol messages that can be forwarded to the specified Gateways. Since the WebSocket server is located on a public IP, large numbers of Gateways can connect to it. The IP Host appends a simple query string to each HTTP request for use by the broker to determine which Gateway to forward the request to. If no query string is provided the request is forwarded to all connected Gateways. The broker translates the ENGAGE WebSocket Sub-Protocol responses back into HTTP responses and returns them to the IP Host.

FAQ

WebSocket Scalability

• How does a single server scale to handle multiple WebSocket connections?

  • WebSocket runs on top of a TCP connection that is persistent to support the initial request and then is retained after the protocol switches to WebSocket. 

  • The TCP protocol defines five elements for each connection: local IP, local port, remote IP, remote port, and protocol; and inherently supports multiple incoming connections.

  • The WebSockets design philosophy (Section 1.5 of the RFC) specifically calls out “adds an addressing and protocol naming mechanism to support multiple services on one port and multiple host names on one IP address.”

  • The only HTTP connections are for the handshake and upgrade request – after that, it falls back to the WebSocket protocol running on the underlying TCP connection.

  • References:

    • https://tools.ietf.org/html/rfc6455

    • https://stackoverflow.com/questions/28516962/how-websocket-server-handles-multiple-incoming-connection-requests

    • https://stackoverflow.com/questions/32317848/multiple-tcp-connection-on-same-ip-and-port

• Since WebSocket connections are stateful, how do you scale server support?  (i.e. how do you load balance, etc.?)

  • While this is outside the scope of the Proof-of-Concept implementation provided in the the section: Implementing a Proof-of-Concept WebSocket Server (it is more complex than stateless connections), this can be accomplished. Below is a link to a blog discussing this issue, and one approach to resolve it:

    • https://hackernoon.com/scaling-websockets-9a31497af051

• How does the Proof-of-Concept Python WebSocket server handle multiple connections?

  • For each new incoming WebSocket connection, the server factory (./EngageWS/EngageWsServer.py) creates a new instance of the EngageWsServerProtocol class to encapsulate the connection.  This is part of the underlying library WebSocket library implementation.

  • In the EngageEsServerProtocol class (./EngageWS/EngageWsProtocol.py), a connection is recorded as a valid connection (connection_made method) during the onOpen callback.  The onOpen callback is called when all prior states in the WebSocket setup are completed. (For additional information, see: latest WebSocket programming.

  • Finally, this set of valid_connections is then used throughout the EngageWsServer Class to allow the server to know what connection is to each Gateway.

  • Example: ./examples/example-server-user-application.py, only shows a single connection. However, if a list were used that tracked multiple connections, it could send messages to multiples. This was done to keep the example as simple as possible.