Reinventing

for the

REAL-TIME WEB
Andrew Godwin
@andrewgodwin

Hi, I'm

Andrew Godwin
Django core developer
Senior Software Engineer at
Likes complaining about networking

Django, HTTP and WSGI

You take a request...

...and return a response.

Browser
HTTP

Webserver
WSGI

Django
Handler

View

Browser

Server
reque
st

se
respon
reque
st
e

s
respon

reque
st

s
respon

reque
st

s
respon

HTTP 1

Browser

Server
reque
st

se
respon
reque
st
e

s
respon

reque
st 1
reque
st 2

se 1
n
o
p
s
e
r
e2
s
n
o
p
s
re

HTTP 2

Browser

Server
send
send
receiv
e

send
receiv
e
receiv
e

send

WebSockets

???

Easy to use
Secure by default
Hard to break/deadlock
Python 2 & 3 compatible
Optional

Python Concurrency &

Django not being async

Message-passing

Process 1
HTTP

WSGI
Server

WSGI

WSGI
App
Process 2

HTTP

WSGI
Server

WSGI

WSGI
App

Process 1
WebSocket

Worker
Protocol
Server

Worker
Worker

Process 2
WebSocket

Worker
Protocol
Server

Worker

Process 1
WebSocket

Protocol
Server

Worker

Process 2
WebSocket

Protocol
Server

Synchronous
Django project
Process 4

Asynchronous
socket handling

ASGI

Worker
Server

Process 1

Interface
Server
Interface
Server
Process 2

ASGI

ASGI

Channel
Layer

ASGI

ASGI

Worker
Server
Worker
Server

Process 3

Worker 1

Worker 2

Worker 3

Interface
Server

Interface
Server

Interface
Server

Channel Layer

Channel Layer

Channel Layer

Worker
Server

Worker
Server

Worker
Server

Channel Layer
Redis

Channels
Named FIFO task queues

Groups
Named sets of channels you broadcast to

send("channel_name", {"ponies": True})

receive_many(["channel_name", "channel_two"])

group_add("group_name", "channel_name")
group_discard("group_name", "channel_name")
send_group("group_name", {"ponies": True})

You take a request...

...and return a response.

You receive a message...

...and send zero or more messages.

Request
View

Response
Message
Consumer

Messages

websocket.connect
Consumer

websocket.receive
Consumer

websocket.send

websocket.connect
Consumer

websocket.receive
Consumer
websocket.send!jq3x41

http.request
Consumer

http.response!a34cxw

Worker

websocket.receive

Worker
Worker

websocket.send!a1b2c3

Socket

Receive events from channels,

and send events to them/groups.
WebSocket/HTTP messages come
with a reply_channel

Using it

Installation

pip install channels

Add channels to INSTALLED_APPS

Liveblog

We want people to get new blog posts as they

are published, without refreshing

Liveblog
People open a WebSocket when they open the page
Their WebSocket is added to a group
When the BlogPost model is saved, we send the
post to that group

Their WebSocket is added to a group

def ws_connect(message):
Group("liveblog").add(message.reply_channel)

When the BlogPost model is saved, we send the

post to that group
class BlogPost(models.Model):
...
def save(self, *args, **kwargs):
...
Group("liveblog").send({
"text": json.dumps({"id": self.id}),
})

Chat

People can send messages, and they

get sent to everyone connected.

Chat

When people connect they join a chat group

When we receive a message we send it to the group

When people connect they join a chat group

def ws_connect(message):
Group("chat").add(message.reply_channel)

When we receive a message we send it to the group

def ws_receive(message):
Group("chat").send({
"text": message["text"],
})

...and we tell Django what consumers are

joined to which actions.
# in routing.py
routing = [
route("websocket.connect", consumers.ws_connect),
route("websocket.receive", consumers.ws_receive),
]

Important notes
Runserver just works with WebSockets now
Django sessions + auth work with WebSockets
Generic Consumers exist
Fully worked versions of these two are at
github.com/andrewgodwin/channels-examples

Channel

Group
Messages

FIFO queue with send and

receive_many operations,
named with a string.
Named set of channels with
add/remove/send operations
Representations for HTTP
and WebSocket sessions
Cross-process

ASGI
&

API specication for

channel layer backends

Message formats for

HTTP and WebSocket

Redis

Reference network layer

POSIX IPC

For single-machine installs

In-memory
For testing or single-process installs

Synchronous
Django project
Process 4

Asynchronous
socket handling

ASGI

Worker
Server

Process 1

Interface
Server
Interface
Server
Process 2

ASGI

ASGI

Channel
Layer

ASGI

ASGI

Worker
Server
Worker
Server

Process 3

Interface
Server

Channel
Layer

Worker
Server

Daphne

asgi_redis

Django

HTTP + WS

Cross-network, shards

Consumer system

WSGI Adapter

asgi_ipc

WSGI Adapter

HTTP

Single-machine

Most WSGI apps

WSGI and/or ASGI

An ASGI system can serve HTTP and WebSocket

...or your WSGI system can send onto channels

Worker 1

Worker 2

Interface
Server

Interface
Server

Channel Layer

Channel Layer

Worker
Server

Worker
Server

Channel Layer
Redis

Worker 3

WSGI
Server
WSGI
App

Scaling?

Interface servers scale horizontally

Worker servers scale horizontally
Channel layer has to as well

Consistent hash sharding designed in

Other approaches possible

Looking Ahead

Being Django
Ofcial external app
Merge in 1.11 or 2.0

Maturity
Load tests and tweaking
Learning from production installs

Community
Third-party tools and extensions
Tutorials, write-ups and case studies

Co-existence
Not everyone needs WebSockets!
Make sure it's easy to add or remove.

Expansion
Email, chat and other message formats
Add more events to code against!

Specication
ASGI is not Django-specic
WSGI needs supplementing for new protocols

channels.readthedocs.io
github.com/andrewgodwin/channels-examples

Thanks.
Andrew Godwin
@andrewgodwin