BYTESTACKDIY
Build Your Own HTTP ServerBuild Your Own HTTP Server
Build / 01PythonStep 9 of 10

Handling Errors

Answer broken requests, unsupported methods and server failures with the right status codes, and keep the server alive.

So far we have tested only the happy path. Real clients don't stay on it. A request can be broken, a method can be unsupported, our own code can fail, and a client can connect and never say anything.

Right now, each of these ends badly:

  • A broken request gets no reply at all.
  • A POST request is treated like a GET.
  • If our code raises an error, the whole server stops for everyone.
  • A silent client blocks the server, because we wait for its request forever.

In this lesson, we fix all these four problems. Status codes help us here. Codes starting with 4 mean "the client did something wrong", and codes starting with 5 mean "the server failed". We already use 404. Here is what we will add:

  • 400 Bad Request: The request is malformed or broken.
  • 501 Not Implemented: The method is not GET or HEAD.
  • 500 Internal Server Error: Something fails in our server code.
  • Timeout: If the client sends nothing for 5 seconds, we close the connection.

Add an Error Helper

Every error response has the same shape: a status, a reason and a short text. Let's write that once.

Add error_response() below build_response():

server.py
def error_response(status: int, reason: str) -> bytes:
    return build_response(status, reason, f"{status} {reason}".encode("utf-8"))

It uses the status and reason as the body too, so error_response(400, "Bad Request") sends the text 400 Bad Request.

Handle One Request in a Function

The code inside main() is getting long, and now it needs to answer errors in several places. Let's move the work for a single request into its own function, handle().

Add it below serve():

server.py
def handle(conn: socket.socket) -> None:
    raw = read_request(conn)
    try:
        method, target, version, headers = parse_request(raw)
    except ValueError as exc:
        print(f"bad request: {exc}")
        conn.sendall(error_response(400, "Bad Request"))
        return

    print(f"{method} {target} {version}")
    for name, value in headers.items():
        print(f"  {name}: {value}")

    if method not in ("GET", "HEAD"):
        print("  -> 501 Not Implemented")
        conn.sendall(error_response(501, "Not Implemented"))
        return

    status, reason, body, content_type = serve(target)
    print(f"  -> {status} {reason}")
    response = build_response(
        status, reason, body, content_type, head_only=(method == "HEAD")
    )
    conn.sendall(response)

It is the same code as before, with two changes:

  1. Broken requests get a reply. : When parse_request() raises a ValueError, we now send 400 Bad Request instead of only printing a message.
  2. Unsupported methods are rejected. : Our server only knows how to GET and HEAD. For anything else, we answer 501 Not Implemented.

Each return ends the function early, so we never reach serve() for a request we have already answered.

Catch Everything Else in main

handle() deals with problems we expect. But some problems we don't expect, such as a bug in our code or a file we cannot read. We catch those in main(), around the call to handle().

Replace the contents of the with conn: block in main() with this:

server.py
                conn, addr = server.accept()
                with conn:
                    print(f"connection from {addr}")
                    conn.settimeout(5)  # give up on a silent client after 5 seconds
                    try:
                        handle(conn)
                    except TimeoutError:
                        print("  timed out waiting for the request")
                    except ConnectionError:
                        print("  client disconnected")
                    except Exception as exc:
                        print(f"  server error: {exc!r}")
                        conn.sendall(error_response(500, "Internal Server Error"))

Let's walk through it:

  1. Set a timeout. : settimeout(5) makes recv() raise an error if nothing arrives for 5 seconds, instead of waiting forever. Browsers sometimes open a connection and never send anything, and without a timeout, one of those would block every other client.
  2. Silent client. : The TimeoutError means the client never sent anything. We print a message, and with conn: closes the connection. There is nobody waiting for an answer.
  3. Client left. : A ConnectionError means the client closed the connection while we were talking to it. There is nobody to answer, so we move on.
  4. Anything else. : Any other error is a failure on our side, so we send 500 Internal Server Error and the server keeps running.

The order matters. Python checks the except clauses from top to bottom, and except Exception catches almost everything, so it must come last.

Run The Server

Here is the full file:

server.py
import socket
from pathlib import Path

HOST = "127.0.0.1"
PORT = 8000
PUBLIC_DIR = (Path(__file__).parent / "public").resolve()

CONTENT_TYPES = {
    ".html": "text/html; charset=utf-8",
    ".css": "text/css; charset=utf-8",
    ".js": "text/javascript; charset=utf-8",
    ".txt": "text/plain; charset=utf-8",
    ".png": "image/png",
    ".jpg": "image/jpeg",
}

def read_request(conn: socket.socket) -> bytes:
    data = b""
    while b"\r\n\r\n" not in data:
        chunk = conn.recv(1024)
        if not chunk:  # the client closed the connection
            break
        data += chunk
    return data

def parse_request(raw: bytes) -> tuple[str, str, str, dict[str, str]]:
    if b"\r\n\r\n" not in raw:
        raise ValueError("incomplete request")

    head = raw.split(b"\r\n\r\n", 1)[0].decode("iso-8859-1")
    lines = head.split("\r\n")

    parts = lines[0].split(" ")
    if len(parts) != 3:
        raise ValueError("malformed request line")
    method, target, version = parts

    headers = {}
    for line in lines[1:]:
        name, sep, value = line.partition(":")
        if not sep:
            raise ValueError("malformed header")
        headers[name.strip().lower()] = value.strip()

    return method, target, version, headers

def build_response(
    status: int,
    reason: str,
    body: bytes,
    content_type: str = "text/plain; charset=utf-8",
    head_only: bool = False,
) -> bytes:
    head = (
        f"HTTP/1.1 {status} {reason}\r\n"
        f"Content-Type: {content_type}\r\n"
        f"Content-Length: {len(body)}\r\n"
        "Connection: close\r\n"
        "\r\n"
    ).encode("iso-8859-1")
    return head if head_only else head + body

def error_response(status: int, reason: str) -> bytes:
    return build_response(status, reason, f"{status} {reason}".encode("utf-8"))

def serve(target: str) -> tuple[int, str, bytes, str]:
    path = target.split("?", 1)[0]
    if path == "/":
        path = "/index.html"

    file = (PUBLIC_DIR / path.lstrip("/")).resolve()
    if PUBLIC_DIR not in file.parents or not file.is_file():
        return 404, "Not Found", b"404 Not Found", "text/plain; charset=utf-8"

    content_type = CONTENT_TYPES.get(file.suffix, "application/octet-stream")
    return 200, "OK", file.read_bytes(), content_type

def handle(conn: socket.socket) -> None:
    raw = read_request(conn)
    try:
        method, target, version, headers = parse_request(raw)
    except ValueError as exc:
        print(f"bad request: {exc}")
        conn.sendall(error_response(400, "Bad Request"))
        return

    print(f"{method} {target} {version}")
    for name, value in headers.items():
        print(f"  {name}: {value}")

    if method not in ("GET", "HEAD"):
        print("  -> 501 Not Implemented")
        conn.sendall(error_response(501, "Not Implemented"))
        return

    status, reason, body, content_type = serve(target)
    print(f"  -> {status} {reason}")
    response = build_response(
        status, reason, body, content_type, head_only=(method == "HEAD")
    )
    conn.sendall(response)

def main() -> None:
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as server:
        server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        server.bind((HOST, PORT))
        server.listen()
        print(f"listening on http://{HOST}:{PORT} (Press Ctrl+c to stop)")

        try:
            while True:
                conn, addr = server.accept()
                with conn:
                    print(f"connection from {addr}")
                    conn.settimeout(5)  # give up on a silent client after 5 seconds
                    try:
                        handle(conn)
                    except TimeoutError:
                        print("  timed out waiting for the request")
                    except ConnectionError:
                        print("  client disconnected")
                    except Exception as exc:
                        print(f"  server error: {exc!r}")
                        conn.sendall(error_response(500, "Internal Server Error"))
        except KeyboardInterrupt:
            print("\nbye")

if __name__ == "__main__":
    main()

Start the server and test the four cases one by one.

A broken request. - This sends a request line with only one part:

Terminal
python3 -c "import socket; s=socket.create_connection(('127.0.0.1',8000)); s.sendall(b'hello\r\n\r\n'); print(s.recv(1024).decode())"

Expected output:

HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Content-Length: 15
Connection: close

400 Bad Request

An unsupported method. - The -X POST option makes curl use POST:

Terminal
curl -i -X POST http://127.0.0.1:8000/

Expected output:

HTTP/1.1 501 Not Implemented
Content-Type: text/plain; charset=utf-8
Content-Length: 19
Connection: close

501 Not Implemented

A silent client. - This connects and says nothing for 8 seconds:

Terminal
python3 -c "import socket, time; s = socket.create_connection(('127.0.0.1', 8000)); time.sleep(8)"

After 5 seconds, the server's terminal shows:

connection from ('127.0.0.1', 44962)
  timed out waiting for the request

A failure in our own code. - To cause one on purpose, add this line as the first line of serve(), then restart the server:

server.py
    raise RuntimeError("boom")

Now ask for any page:

Terminal
curl -i http://127.0.0.1:8000/

Expected output:

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
Content-Length: 25
Connection: close

500 Internal Server Error

The server's terminal shows the real reason, which the client never sees:

connection from ('127.0.0.1', 54313)
GET / HTTP/1.1
  ...
  server error: RuntimeError('boom')

Run the curl command again. The server answers again. It did not stop.

Remove the raise line and restart the server before you continue.

What You Built

You started with an empty file, and you now have a working HTTP server. It opens a socket, accepts connections, reads and parses requests, sends well-formed responses, serves files from a folder, and answers errors with the right status codes. All of it fits in one Python file, and every part is something you wrote and understand.

It is still a teaching server. A real one would also handle many clients at the same time, limit the size of requests, reuse connections, and more. Congratulations! You have built your own simple HTTP server. Next, let's look at where you can take it from here.

Serving Static Files

Replace the routes table with real files on disk and label each response with the right Content-Type.

Where to Go Next

What our simple HTTP server cannot do yet, and a few experiments to make it better.

On this page

Add an Error HelperHandle One Request in a FunctionCatch Everything Else in mainRun The ServerWhat You Built