Version 1.3.13 of nginx is out and with it comes support for Connection: upgrade and Upgrade header, meaning proxying of WebSockets!
Many people have been waiting for this and “are websockets in nginx supported?” is one of the most frequent questions in #nginx on freenode.
With that out of the way, time to have a look at the nginx WebSocket implementation.
The New Nginx Websockets Configuration Directives
The documentation provided states the configuration as follows:
location /chat/ {
proxy_pass http://backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
This is indeed fairly simple and configuring the HTTP version used isn’t even new at all. We can go ahead and improve the versatility a bit by creating a connection variable thus allowing us to move these proxy_set_headers into a generic included file.
map $http_connection $upgrade_requested {
default upgrade;
'' close;
}
This makes the variable $upgrade_requested available to proxy_set_header Connection and if connection upgrade wasn’t requested then it will default back to “” to not interfere with normal requests. The idea here being that if you always proxy HTTP/1.1 then you don’t need a location specifically to handle WebSockets.
Connection upgrading seems unlikely to be back ported into the stable branch so if you wish to use this you will have to use the development branch. Thankfully development in nginx does not mean that it’s not run-time stable, it just means that the API might change and as such it only affects module authors. Don’t be afraid to install the development version to play around with this new feature.
Limitations
So far a few limitations are known in the websocket implementation.
Client Must Specify Connection Upgrade
The client need to have requested the connection upgrade, otherwise nginx will fail. Currently that part is listed as a todo in the code so I can’t say how it will fail, but I’m sure it will be implemented eventually so don’t rely on it. This limitation should not be an issue to anyone and the only issue with it is a theoretical situation where a module author might want to use connection upgrade to a backend and then process the response itself.
WebSockets Time Out
WebSockets are still affected by proxy_read_timeout which defaults to 60 seconds. This means that if you have an application using WebSockets but not sending any data more than once per 60 seconds you either need to increase the timeout or implement a ping message to keep the connection alive. The ping solution has the added benefit of discovering if the connection was closed unexpectedly.
Keep-Alive & WebSockets
Keep-alive pings aren’t usable for working around the above mentioned timeout as they work on the TCP level are are just empty packets. They aren’t reported to the application and thus the application will never respond to them meaning that proxy_read_timeout will still trigger.
WebSockets Supports SSL
Since WebSockets tie into the normal proxy module SSL works the exact same way it normally would.
Proxy Buffers
WebSockets utilize two memory buffers the size of proxy_buffer_size, one for upstream data and another for downstream data. The difference between WebSockets and a normal proxy request is that WebSockets will never buffer to disk.
Case Sensitive Upgrade Header
Some backends do a case sensitive check on the upgrade header and will require the header to be either “upgrade” or “Upgrade”, failing on one and working on the other. If things seem to be right but doesn’t work then try changing the case of the header.
12 Comments. Leave new
“The client need to have requested the connection upgrade, otherwise nginx will fail. Currently that part is listed as a todo in the code so I can’t say how it will fail, but I’m sure it will be implemented eventually so don’t rely on it.”
This is an incorrect interpretation of commentary in the code: “TODO: prevent upgrade if not requested or not possible”. This comment is from a function that handles the response from backend. So, it actually means that currently it is possible for backend to upgrade protocol even if nobody has requested.
I actually based it on the quote “Connection upgrade is allowed as long as it was requested by a client via the Upgrade request header.” from http://trac.nginx.org/nginx/changeset/5073/nginx. I did just briefly glance at the todo and assumed it wasn’t yet implemented, though, it might already be. Do you interpret the commit message differently?
“Connection upgrade is allowed as long as it was requested by a client via the Upgrade request header.”
This is actually how WebSocket works according to RFC 6455.
Yeah but my point is that there might not be a way for nginx to act as a client itself. There are 3rd party modules that add extensive logic to nginx which make it more of an application server. This limitation might affect modules ability to interact with WebSockets for whatever reason they might want to. I specifically said that this isn’t relevant to most people but I still think it’s worth mentioning.
One thing I’ve noticed, nginx ram usage, which used to be stable, now continually increases. Restarting nginx doesn’t solve the issue. but restarting the node.js websocket server returns the ram to its usual level.
Could be something I have misconfigured elsewhere. Could be dodgy stats reporting.
In certain version of websocket implementation, the headers for Upgrade and Connection are case-sensitive and have to match “Upgrade: WebSocket\r\nConnection: Upgrade\r\n”.
So I’ve started using:
proxy_set_header Upgrade “WebSocket”;
proxy_set_header Connection “Upgrade”;
Sorry, I posted too soon. This included some of my own debug code. It appears the only line that’s needs to be case-sensitive is the Connection: Upgrade line. So these two pairs of proxy_set_header’s work:
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection “Upgrade”;
Would that be the correct way to proxy standard HTTP as well as websockets? I don’t want the Upgrade header or Connection being set to “upgrade” unless that’s what the browser sent, but these proxy_set_header lines are required for websockets to work. Why doesn’t nginx just forward the original Upgrade/Connection headers?
If you use the map to create the header then it will be fine. You have to set these lines because nginx does not forward anything, nginx is a terminating reverse proxy, not a transparent pass-through proxy.
I really thank you. This lines saves my life : )
map $http_connection $upgrade_requested {
default upgrade;
” close;
}
[…] WebSockets in Nginx walks through the Nginx WebSockets configuration directives. […]
Hi Martin,
I try to pass-through the https of the nginx conf as below:
stream {
upstream web_server {
server 10.90.72.38:443;
}
server {
listen 443;
proxy_pass web_server;
}
}
It’s pass through SSL traffic successfully.
I also try to pass-through the wss of the nginx conf as below:
stream{
upstream wssapp {
#hash $remote_addr consistent;
server 10.90.72.38:3000;
#server 10.90.72.39:5000;
}
#map $http_upgrade $connection_upgrade {
# default upgrade;
# ” close;
#}
server {
listen 8080;
proxy_pass wssapp;
#ssl on;
#ssl_certificate /etc/ssl/server.crt;
#ssl_certificate_key /etc/ssl/server.key;
#location / {
# proxy_http_version 1.1;
# proxy_set_header Upgrade $http_upgrade;
# proxy_set_header Connection $connection_upgrade;
# proxy_buffering off;
# proxy_pass http://wsapp;
#}
}
}
It’s not work and show websocket connection event code 1006. I think the root cause is disabled the “http_upgrade”.
But nginx config not allow enable “http_upgrade” in the stream-pass-through case.
How could I modified the nginx config file to pass through wss traffic?
many thanks.