How to Manually Test WebSocket APIs

Sometimes, as a test engineer, you have to deal with advanced technology, such as WebSockets position itself.

8 min readJul 12, 2022

--

If you have ever tested HTTP API, then you will not encounter very difficulties with testing bidirectional messages. However, there are a few features inherent in WebSockets that should be taken into account before starting testing.

Theory

A WebSocket is a persistent connection between client and server. WebSockets provide a full-duplex communication channel (allows communication in both directions simultaneously) through a single TCP/IP socket connection.

Thanks to WebSocket connection, client-server communication can happen in real-time (unlike HTTP with timings on each request/response), similar to UDP, but with the reliability of TCP. So it is broadly used in chat, gaming, and stock market browser-based applications.

WebSocket starts from a handshake, followed by basic message framing:

A WebSocket connection is established by upgrading an HTTP request. A client (in most cases, the client means a web browser) that supports WebSocket API and wants to establish a connection should send an HTTP request with WebSocket-specific headers:

Connection: keep-alive, upgrade — a common value for this header is «keep-alive» to make the connection persistent, which allows subsequent requests to the same server. «Upgrade» value pushes to upgrade a connection to a different protocol;
Upgrade: websocket — specifies a protocol to updated;
Sec-WebSocket-Key: key — it is a one-time random value generated by the client to confirm that it is entitled to request an upgrade to WebSocket;
Sec-WebSocket-Version: number — specifies the WebSocket protocol version the client wishes to use.

The minimum required request looks like this:

GET /socket HTTP/1.1
Host: polkadot.webapi.subscan.io
Connection: keep-alive, upgrade
Upgrade: websocket
Sec-WebSocket-Key: N509hQQwSFGfanhbxP3F6g==
Sec-WebSocket-Version: 13

If everything goes well, the server will answer with «HTTP 101 Switching Protocols» response code with the additional headers:

Connection: upgrade
Upgrade: websocket
Sec-WebSocket-Accept: key — informs that the server is willing to initiate a WebSocket connection.

In real life, the handshake looks like this (for the following examples, I will use WebSocket API from Subscan.io):

*Fig. 2. Network request details of WebSocket in Firefox DevTools*

After the client gets the server response, the WebSocket connection is open to start transmitting message-based data according to WebSocket Protocol. Without going into details, the «message-based data» means a payload/body (in most cases, it is a text) inside messages between client and server:

The server constantly sends messages to the client — Fig. 3. Here, the server constantly sends messages to the client. In this case, text messages are parsed in JSON format, but it does not involve transmission in any particular format

As soon as both sides acknowledge that the WebSocket connection should be closed, the TCP connection is terminated.

Tools

Browsers

As follows from the documentation, WebSockets are designed for browser-based applications that need two-way communication with servers. Thus, all modern browsers support WebSockets.

You can explore WebSocket resources in DevTools (as shown in the screenshots above) and even interact with them in the Console tab through WebSocket API.

All the basic functionality fits into several commands:

1. Open the WebSocket connection:

const ws = new WebSocket('wss://polkadot.webapi.subscan.io/socket');

2. Log messages from the server:

ws.onmessage = function (e) { console.log(e.data) };

3. Send messages to the server:

ws.send('hello world');

4. Close connection:

ws.close();

*Fig. 4. Work with WebSockets in the Console through WebSocket API*

Using the browser’s console is very handy for smoke testing.

Wscat

The fans of CLI will be disappointed with a limited set of tools available to work with WebSockets from the terminal (unfortunately, even libcurl does not support WebSockets). Actually, there is only one usable CLI tool — wscat.

It installs as an npm package:

npm install -g wscat

And immediately ready to work:

wscat -c 'wss://polkadot.webapi.subscan.io/socket'

Postman

This «tester’s swiss knife» got WebSockets support just a year ago, but it is available only after logging into a «workspace». Despite the fact that WebSockets in Postman are still in beta, the current functionality is completely sufficient for testing API.

Click [New], and on the «Create New» screen, choose «WebSocket Request»:

*Fig. 6. New «WebSocket Request» in Postman*

On the following interface (it is a bit different from the usual form for HTTP requests), you can easily make WebSocket requests, and send and receive WebSocket messages.

WebSocket request tab in Postman — *Fig. 7. WebSocket request interface in Postman*

At this time, there is no «Tests» tab for automation tests, as HTTP requests interface does. So testing WebSockets in Postam is really manual. A description of the rest of the features is beyond the scope of this article, moreover, the documentation is very detailed.

Practice

WebSocket API is an API; therefore, all the same, testing strategies can be applied to WebSockets as to REST API testing. It means that the basic checks will not differ much from the usual testing of HTTP API:

Verify correct methods and status codes;
Verify request and response payloads;
Verify non-functional requirements: response timings, performance, security, etc.

Furthermore, the functional testing process could be divided into two principal parts:

Testing correct implementation of WebSockets;
Testing business logic.

Disclaimer: I used different tools to demonstrate the execution of various cases. There are no particular reasons why I use one for the other except presentability. The final choice of a testing tool is up to you.

Testing correct implementation

As described in the theoretical part of the article, the initial stage of connection to WebSocket is a handshake, and all available tools do it automatically. It is very convenient, but it does not leave much space for manual testing because familiar HTTP tools do not work with WebSockets — even browsers themselves do not do WebSocket requests outside the origin web page:

*Fig. 8. Nothing will happen if you try to open a WebSockets URL on a new page*

At least you can do:

Positive check: connection should be established through the frontend in a browser.

Reload a web page and verify HTTP status code = 101 and ensure that the browser started exchanging messages with the server:

Negative check: connection should not be established through invalid protocol.

Request a WebSocket URL as an ordinary HTTPS request instead of WSS.

*Fig. 1.. «400 Bad Request» is the expected result*

The other face of this case can be trying to establish an unsecured connection. There are two types of WebSocket protocol:

ws connects on HTTP;
wss connects on HTTPS only (secured).

*Fig. 12. Could not connect to ws://polkadot.webapi.subscan.io/socket is the expected result.* This time I changed the default handshake request timeout in order not to wait indefinitely if there is no connection

Positive check: connection should be established through the API request.

Request WebSocket URL through CLI tool (wscat) or Postman. That will confirm that the server can establish connections with other clients (besides browsers) that support WebSockets.

*Fig. 13. Connected to wss://polkadot.webapi.subscan.io/socket*

Exploratory check: server should respond with correct status codes.

First of all, WebSocket’s status codes differ from the usual HTTP ones.

Secondly, the specification of the status codes is rather vague — some codes can be attributed to different cases. For instance, an authorized request may refer to either 1008 or 1003.

Therefore, error handling highly relies on the development decisions (of course, as in any API) and testing is strongly tied to documentation. But if WebSocket API uses HTTP status codes — it can definitely be considered a bug.

Finally, it is a bit tricky to get a status code from a public WebSocket API which involves only sending data (like wss://polkadot.webapi.subscan.io/socket). WebSocket API with the ability to respond to received messages is more «testable» for this case:

Trying to send an empty message (wcat used here because Postman does not allow sending empty messages to WebSockets):

*Fig. 14. Endpoint terminated the connection with code 1007*

Trying to send an unsupported message type:

*Fig. 15. Endpoint terminated the connection with code 1003*

You can come up with more test cases, but these checks are already crossing the area of business logic.

Testing business logic

That is the most crucial part of API testing and the most diverse — all test cases depend on the documentation, test design, and behavior of the application.

I will give some examples based on the documentation of CoinAPI.io Market Data WebSocket API.

First positive check: server should start sending response messages after a valid request message.

After WebSocket connection is established, you must send a «hello message» which contains several required parameters:

{"type": "hello", "apikey": "73034021-THIS-IS-SAMPLE-KEY", "heartbeat": false, "subscribe_data_type": ["trade"], "subscribe_filter_asset_id": ["DOT"]}

Response body validation: the response messages should be parsable, and the values of the keys must be valid.

Let’s take a random response message — it is a JSON format, and a certain key-value pair contains certain data — each value can be validated for data types and boundary values.

{
  "time_exchange": "2022-07-11T13:37:30.2810000Z",
  "time_coinapi": "2022-07-11T13:37:30.4243323Z",
  "uuid": "4d4dfbb0-a69a-4ceb-a24a-810991907d2d",
  "price": 6.817,
  "size": 1.03845,
  "taker_side": "BUY",
  "symbol_id": "BINANCEFTS_PERP_DOT_USDT",
  "sequence": 71397,
  "type": "trade"
}

Extended check: symbol_id value should contain data according to the filter subscribe_filter_asset_id (it means that the string "BINANCEFTS_PERP_DOT_USDT" should include "DOT").

Negative checks: server should not respond to an invalid request message.

What if the request message does not satisfy the required format? What if the type will not be hello? apikey will be invalid? subscribe_data_type will be in an unavailable range for a free API key? message will not contain the required parameters? and etc.