This is the second post in a series of posts on Network Programming.
For the first part of the series click here.
In the first post we introduced UDP and TCP sockets, showed how to create server sockets, bind them to an address, and showed what happens when a message is too long to be read in a single call to socket.recv.
Recall that the first time we had our client talk to our server we had it send “the rain in spain stays mainly in the plain” which is only 43 characters long, but the server was listening for 100 characters. This is because (per the documentation) the 100 we are passing into the recv method is the maximum number of bytes to read.
So how do we receive a message that is more than 100 bytes? It’s easy we make additional calls to socket.recv.
#!/usr/bin/env python
from __future__ import print_function
import socket
def main(*args, **kwargs):
print('creating server socket')
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print('done. now binding to address')
server_socket.bind(('0.0.0.0', 54321))
print('done. now listening for connections')
server_socket.listen(20)
try:
while True:
client_socket, client_address = server_socket.accept()
print('new client connected from: ', client_address)
needs_to_keep_reading = True
while needs_to_keep_reading:
content = client_socket.recv(100)
print('#### new data received ####')
print(content)
needs_to_keep_reading = len(content) == 100
print('#### finished receiving data ####')
client_socket.shutdown(socket.SHUT_RDWR)
client_socket.close()
finally:
server_socket.shutdown(socket.SHUT_RDWR)
server_socket.close()
if __name__ == '__main__':
main()
So here we keep reading while we receive exactly 100 bytes.
Great!!!
Unfortunately, this won’t work in the real world. What we didn’t mention earlier was that we can’t control how many bytes are recieved. Even if the client on the far side does indeed send exactly 150 bytes in a single socket.send command, that doesn’t mean that two calls to socket.recv(100) will return the first 100 bytes followed by the second 50 bytes. This is because those bytes may be split up along the way as they travel throughout the internet.
This sample client program is contrived, but shows that you can’t count on always reading the maximum 100 bytes, even if more than 100 bytes are headed your way.
#!/usr/bin/env python
from __future__ import print_function
import socket
def main(*args, **kwargs):
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(('0.0.0.0', 54321))
total_stuff = 'banana'*100
first_send = total_stuff[:20]
second_send = total_stuff[20:]
client_socket.send(first_send)
client_socket.send(second_send)
client_socket.shutdown(socket.SHUT_RDWR)
client_socket.close()
if __name__ == '__main__':
main()
So crud. How do we know when “the message” has ended.
This is what network protocols are for. Before we talk about what a network protocol is, we should dig in to understand what a protocol is. I think the simplest example of a protocol is Morse Code. Morse Code was a simple agreement between all parties using Morse Code on how to represent letters and numbers. It also said how to separate words (seven units of silence indicates the separation of a word). However, because it had no way of representing a period or other punctuation, there was no way of indicating the end of a sentence. That’s why, when ever you see people doing Morse code in historical TV shows they are always saying STOP, blah blah blah STOP.
The word STOP was used informally as a means of indicating the end of a sentence.
Network protocols behave the same way, in HTTP a “message” is either an HTTP request or an HTTP response but both end with two carriage return line feeds CF LF CR LF. So and HTTP server must continue to read on the socket until it comes across two carriage return line feeds. But what happens if that sequence of characters never occurs? Well, that probably is an indication of either a buggy or malicious http client (or server) and typically server implementations might limit how large a message can be (if the protocol standard doesn’t explicitly state how to handle this type of situation).
There are two main ways of indicating the end of a message. One would be to state that all messages are of a certain length. The other would be to implement some sort of delimited indication that the message has ended. Using STOP with morse code and two carriage return line feeds imply that these protocols are both delimited.
Naturally, a delimited protocol is more flexible than a fixed size one, but there is no doubt that a fixed length protocol is easier to implement (no need to scan the content of the message to seek for delimiters).
So we’ll finish off this post by creating a simple echo server and echo client. The echo server will accept client sockets and it will read in as much data from the client as possible until it gets to a colon character (:). Once it sees the colon it will send back whatever the client sent (including the colon) back to the client. Once the data has been sent it will start over, reading more data from the client until it sees another colon and sending that back to the client. The server will continue in this fashion until the client closes the connection.
In our client we will allow the user to enter as much data as the user wants then send the data to the server. After it sends the data to the server it will read everything back from the server and then ask the client for more info.
Now before we get coding on this we have to address the scenario on the server when the client disconnects. How do we know when the client disconnects? From this article in the python documentation, socket.recv returns zero bytes it means that the other end has closed the connection.
So here’s our server program now:
#!/usr/bin/env python
from __future__ import print_function
import socket
def main(*args, **kwargs):
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(('0.0.0.0', 54321))
server_socket.listen(20)
try:
while True:
client_socket, client_address = server_socket.accept()
handle_client(client_socket)
finally:
server_socket.shutdown(socket.SHUT_RDWR)
server_socket.close()
def handle_client(client_socket):
socket_is_alive = True
while socket_is_alive:
complete_content = ''
needs_to_keep_reading = True
while ":" not in complete_content:
current_chunk = client_socket.recv(100)
complete_content = complete_content + current_chunk
if len(current_chunk) == 0:
print('---the socket died---')
socket_is_alive = False
break
print('---got full message---')
print(complete_content)
print('---end of message---')
if socket_is_alive:
print('---sending the echo---')
client_socket.send(complete_content)
print('---done echoing---')
if __name__ == '__main__':
main()
and here’s our client program.
#!/usr/bin/env python
from __future__ import print_function
import socket
def main(*args, **kwargs):
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(('0.0.0.0', 54321))
data_to_send = ''
try:
while True:
data_to_send = data_to_send + raw_input('enter more data:')
if ':' in data_to_send:
client_socket.send(data_to_send)
data_to_send = ''
print_received_data_from(client_socket)
except KeyboardInterrupt:
client_socket.shutdown(socket.SHUT_RDWR)
client_socket.close()
def print_received_data_from(client_socket):
complete_data = ''
while ':' not in complete_data:
complete_data = complete_data + client_socket.recv(100)
print('got data back')
print(complete_data)
print('done getting data')
if __name__ == '__main__':
main()
Go ahead and run them both and notice how the server can detect when you close the client program by pressing control-c.
But try running two clients at the same time. You should notice that the second client won’t receive any data from the server until the first client stops. This makes sense because socket.accept on line 13 of the server doesn’t get called again until the last client has been handled.
So how can we support multiple clients at the same time? Well there are lots of ways to do that, but we’ll leave that for part 3 of the series.
Hacker Farm Blog 