Bidirectional HTTP/1.1 with Transfer-Encoding: chunked

Lately I haven’t update anything, so I thought about reviving this blog with an engineering technical.

Today, I got intrigued by a thread. It seems that this interviewer stated in their interview that “HTTP protocol is not bidirectional”. Let’s dive in and prove he’s wrong.

What is bidirectional even means

Bidirectional means data can move both ways — from client to server and from server to client — over the same connection.

TCP is bidirectional because, once a connection is established, each side can send and receive data independently.

Example of bidirectional of a TCP connection:

• Let A and B be 2 servers, A establishes TCP handshake with B.
• Now, we have 1 connection between:

A <----------------------------> B

• 1. A can send data to B through this connection to do something:

A -----"what is 1+1"-----------------> B

• 2. B can response back to A with the result:

A <----"2"---------------------------- B

• 3. However, because it is bidirectional, B can also tells A to do something:

A <---"Can I get your local time?"---- B

• 4. And A can answer back to B

A ----"9/27/2025, 4:02:50 PM"--------> B

• 5. Both sides can continue exchanging messages Even while A is thinking or waiting for B, it can still send new questions, and B can also send new messages back.

Myth about HTTP is unidirectional

Nowadays, people often overlook details and tend to trust what they read in books, courses, or anything that shapes their knowledge. I’ve done the same with ChatGPT’s code, no shame, everyone makes mistakes.

I tried looking through RFC 2616 and RFC 7231 which is the originates of the HTTP/1.1 protocol, there is no line that says HTTP is unidirectional.

However, I tried chatting with my ChatGPT, it says that http is unidirectional, and it reference from the RFC 2616 which says:

The HTTP protocol is a request/response protocol.

But actually what is request/response protocol ? Isn’t everything happen on networking is a request/response model ? Where one machine have to initiate the interaction with other machine, then after that the other machine can response ? Seems like ChatGPT has misunderstood the RFC.

What is Transfer-Encoding

Transfer-Encoding tells how HTTP message bodies are sent over the network. In HTTP/1.1 a request or response body must use one of two ways:

• Content-Length: <length>: you know the total size before sending.
• Transfer-Encoding: chunked: you don’t know the total size; you send it in pieces (chunks).

Example using Content-Length

This is the normal case when the body size is known.

Example request:

client A ----------------------------------
      |   PUT /api/v1/users/1 HTTP/1.1     |
      |   Host: clientb.com                |
      |   Content-Type: application/json   |
      |   Content-Length: 10               |-----> client B
      |                                    |
      |   {"age":11}                       |
      |____________________________________|

Example response:

client A
    ^
    |__________________________________________
      |                                       |
      |  HTTP/1.1 200 OK                      |
      |  Content-type: application/json       |
      |  Date: Fri, 26 Sep 2025 18:48:36 GMT  |
      |  Content-Length: 16                   |---- client B
      |                                       |
      |  {"message":"ok"}                     |
      |_______________________________________|

Notice that the number of bytes of:

• The request body: {"age":11} is 10
• The response body: {"message":"ok"} is 16 Because both sides know the exact size up front, the client can allocate a buffer and read the whole body.

When Content-Length isn’t practical

Sometimes the body is huge or generated on the fly (like a game download or live data stream). The server can’t easily calculate the full length first.

In those cases the server (or client) switches to Transfer-Encoding: chunked, the sender just need put it onto the HTTP packet’s header and the receiver will understand and parse it correctly.

This encoding scheme lets data be sent piece by piece. Each piece starts with its size, followed by the data, and ends with a 0-size chunk when finished. The client can start processing chunks as they arrive instead of waiting for the whole body.

Example request:

client A ----------------------------------
      |   POST /upload HTTP/1.1            |
      |   Host: clientb.com                |
      |   Content-Type: text/plain         |
      |   Transfer-Encoding: chunked       |
      |   Accept-Encoding: chunked         |
      |                                    |
      |   5\r\n                            |
      |   Hello\r\n                        |------> client B
      |   9\r\n                            |
      |   World\r\n                        |
      |   0\r\n                            |
      |   \r\n                             |
      |____________________________________|

Noted that the body is: 5\r\nHello\r\n9\r\nWorld\r\n0\r\n\r\n, on the above diagram I press enter down for easily view the packet.

The Accept-Encoding: chunked header is there to tell the server that the client can receive chunked encoding.

Example response:

client A
    ^
    |__________________________________________
      |                                       |
      |  HTTP/1.1 200 OK                      |
      |  Content-type: text/plain             |
      |  Date: Fri, 26 Sep 2025 18:48:36 GMT  |
      |  Transfer-Encoding: chunked           |
      |                                       |
      |  A\r\n                                |
      |  I have rec\r\n                       |
      |  A\r\n                                |
      |  eived your\r\n                       |---- client B
      |  A\r\n                                |
      |  hello wor\r\n                        |
      |  2\r\n                                |
      |  ld\r\n                               |
      |  0\r\n                                |
      |  \r\n                                 |
      |                                       |
      |_______________________________________|

This would be interpreted by the client as: I have received your hello world.

The MDN definition:

The HTTP Transfer-Encoding request and response header specifies the form of encoding used to transfer messages between nodes on the network.

This means we can use Transfer-Encoding: chunked in both request or response just as I just demonstrated. Thus, the HTTP/1.1 connection does not have to be a one-way, fire-and-forget transfer. A client can send a streaming body to the server in chunks at the same time the server starts sending its own chunked response back.

Achieving full-duplex bidirectional with Transfer-Encoding: chunked

Outline of all the steps

In this excercise, I’ll try to create a HTTP server and a client. The steps are as follow:

• 1. Client establish a HTTP connection to the server
• 2. Server response to the client, telling the client that it should prove a proof of work before using the service.
• 3. The client solve the proof of work challenge, then send the solution to the server
• 4. The server validates it, if it is correct then it would response as “Correct”
• 5. The client can then send numbers onto the server
• 6. For each number that the client has sent, the server will calculate the sum of all numbers that has been sent by the client
• Diagram of the whole flow:

HTTP establish connection:
            client <--------------------> server
POW phase:
            client <--{salt;difficulty}-- server
            client --------"abcd"-------> server (where "abcd" is the result of POW)
            client <------"Correct"------ server
Sum calculator phase:
            client ---------"2"---------> server
            client <--------"2"---------- server
            client --------"109"--------> server
            client <-------"111"--------- server
            client --------"-10"--------> server
            client <-------"101"--------- server

As you can see, if we could implement the above steps then we could prove that on 1 connection, HTTP can be bidirectional.

Let’s show the code

Every code is implemented with Bun in the mind, if you want to run please install it. I used Bun because it automatically has typescript supported. Full code is at: https://github.com/phvietan/bidirectional-http-chunked , and the server is hosted at https://bi-http.drstrange.org:10000/

First, let’s define some common functions, for sending data, receiving data with delimiter ‘\n’, and PoW class.

common.ts

import { ReadableStreamDefaultReader } from 'node:stream/web'
import crypto from 'node:crypto';

var bufferBefore = '';
export async function read(reader: ReadableStreamDefaultReader<any>) {
  const { value, done } = await reader.read();
  if (done) return null;
  const res = bufferBefore + new TextDecoder().decode(value);
  bufferBefore = '';
  return res;
}
export async function readUntil(delimiter: string, reader: ReadableStreamDefaultReader<any>) {
  let result = "";
  while (true) {
    const chunk = await read(reader);
    if (chunk === null) return null;
    result += chunk;
    if (result.includes(delimiter)) {
      const parts = result.split(delimiter);
      result = parts[0]!;
      bufferBefore = parts.slice(1).join(delimiter);
      break;
    }
  }
  return result;
}

function sha256(s: string) {
  const hash = crypto.createHash('sha256');
  hash.update(s);
  return hash.digest('hex');
}

export class POW {
  difficulty: number;

  constructor(difficulty: number) {
    this.difficulty = difficulty;
  }

  generateChallenge() {
    const salt = crypto.randomBytes(16).toString('hex');
    return {
      salt,
      challenge: `Find a string such that sha256(salt + string) starts with ${'0'.repeat(this.difficulty)} in hex. Salt: ${salt}\n`,
    }
  }

  parseChallengeMessage(message: string) {
    const match = message.match(/Salt: (\w{32})/);
    if (match) {
      return match[1];
    }
    throw new Error("Invalid challenge message");
  }

  verifySolution(salt: string, candidate: string) {
    const hashed = sha256(salt + candidate);
    return hashed.startsWith('0'.repeat(this.difficulty));
  }
}

Then, server.ts should implement the following code to response as Transfer-Encoding: chunked:

server.ts

Bun.serve({
  // Comment out tls if you try to run as http for development
  port,
  hostname: "0.0.0.0",
  routes: {
    "/": {
      "POST": async (request) => {
        const reader = request.body?.getReader();
        if (!reader) {
          return new Response("Request does not have body", { status: 400 });
        }

        const stream = new ReadableStream({
          async start(controller) {
            ...
          }
        });
        return new Response(stream, {
          headers: {
            "Content-Type": "text/plain",
            "Transfer-Encoding": "chunked",
          }
        });
      },
    },
  },
});

Inside ReadableStream.start function, the core implementation would be

server.ts[25:59]

async start(controller) {
  // POW phase
  const powChallenge = new POW(2);
  const { salt, challenge } = powChallenge.generateChallenge();
  controller.enqueue(challenge + '\n');

  const powSolution = await readUntil('\n', reader);
  const candidate = powSolution?.trim() || "";
  console.log('Client POW solution:', candidate);
  if (!powChallenge.verifySolution(salt, candidate)) {
    controller.enqueue("Proof of work failed. Try again.\n");
    controller.close();
    console.log('POW failed');
    return;
  }
  console.log('POW succeeded');
  controller.enqueue("Correct" + "\n");

  // Sum phase
  let sum = 0;
  while (1) {
    const message = await readUntil('\n', reader);
    if (message === "end" || message === null) break;
    const num = Number(message);
    if (isNaN(num)) {
      controller.enqueue("Not a number, sum is still " + sum + "\n");
      continue;
    } else {
      sum += num;
      controller.enqueue("Current sum: " + sum + "\n");
    }
  }
  controller.close();
  console.log('Ended sum calculation with', sum);
}

In above code:

• controller.enqueue(challenge + '\n'); is used to response to the client the challenge message as chunked without closing the connection.
• const powSolution = await readUntil('\n', reader); the readUntil function is defined in common.ts file, if you scroll up to common.ts you will see that I’m using while(true) { reader.read(); ..., you can try putting console.log(1) inside the while(true) loop, you might expect the server to spit out a bunch of 1’s, but instead it stops and waits for a message to be sent from the client, this API works just like the C api recv when waiting for incomming TCP message.

Then, the client.ts:

client.ts

import { type ReadableStreamController } from "bun";
import { ReadableStream } from "stream/web";
import { POW, readUntil } from "./common";

async function main(url: string) {
  let globalController!: ReadableStreamController<any>;

  const response = await fetch(url, {
    method: "POST",
    body: new ReadableStream({
      async start(controller) {
        globalController = controller;
      }
    }),
  });

  const reader = response.body?.getReader();
  if (!reader) {
    console.error("No body in response");
    return;
  }

  // ====== POW PHASE ======
  // Get POW challenge
  let val = await readUntil('\n', reader);
  console.log(`[Server] ${val}`);

  // Solve POW challenge
  const powChallenge = new POW(2);
  const salt = powChallenge.parseChallengeMessage(val);
  const solution = powChallenge.solveChallenge(salt);
  console.log(`[Client] POW solution: ${solution} sending it to server`);
  globalController.enqueue(solution + '\n');

  // Wait for "Correct" response
  val = await readUntil('\n', reader);
  console.log(`[Server] ${val}`);

  if (val !== "Correct") {
    console.error("POW failed, exiting");
    globalController.close();
    return;
  }

  // ====== SUM PHASE ======
  console.log("POW succeeded, you can now send numbers to sum. Type 'end' to finish.");
  const prompt = "[Client] Input number: ";
  process.stdout.write(prompt);
  for await (const line of console) {
    const input = line.trim();
    globalController.enqueue(input + '\n');

    val = await readUntil('\n', reader);
    console.log(`[Server] ${val}`);
    if (input === 'end') break;

    process.stdout.write(prompt);
  }

  globalController.close();
}

// Example usage
main("https://bi-http.drstrange.org:10000/");

I have hosted a server at https://bi-http.drstrange.org:10000/, you can test it with the client.ts file in the github code. Or if you’d like to experiment, you can run your own server.ts at localhost, and change the url in client.ts.

In the cli terminal, you can input numbers and the server will response the sum just as I designed the excercise.

With this, I have proved that HTTP/1.1 protocol can achieve bidirectional with chunked encoding, but what about running javascript on the browser ? Can we do that too ?

index.html

<!DOCTYPE html>
<html lang="en">

<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Document</title>
</head>

<body>
  Please see console for output
  <br />
  --------------------------------------------
  <br />
  The main fetch with body should throw error because browser does not allow both response and request to be chunked
  <br />
  The fetch without body should went through, but server response "Request does not have body" and not run the challenge
  <br />
  --------------------------------------------

  <script>

    async function connectWithoutBody() {
      const response = await fetch('/', {
        method: 'POST',
      });
      const reader = response.body.getReader();

      const {value} = await reader.read();
      const text = new TextDecoder().decode(value);
      console.log('[no-body] Received', text);

      console.log('[no-body] Response fully received');
    }

    async function main() {
      var globalController;
      const response = await fetch('/', {
        method: 'POST',
        duplex: 'half',
        body: new ReadableStream({
          start(controller) {
            globalController = controller;
          }
        })
      });
      const reader = response.body.getReader();

      const {value} = await reader.read();
      const text = new TextDecoder().decode(value);
      console.log('[main] Received', text);

      console.log('[main] Response fully received');
    }

    main();
    connectWithoutBody();
  </script>
</body>

</html>

There are 2 main functions here:

• main(): the main function will try to establish the Transfer-Encoding: chunked with the server like what I did in client.ts file. But it should throw out error net::ERR_ALPN_NEGOTIATION_FAILED. At least on my machine it shows that, please contact me if on your machine it shows differently.
• connectWithoutBody(): this function also try to establish Transfer-Encoding: chunked with the server, but this time it does not have the body. This time the browser happily accept it, but the server response that there is no body to start the whole POW phase.

Thus, we can see that the browser does not allow bidirectional HTTP with chunked encoding, we can only do that with 2 backends.

Recap

Anyone who managed to read all of my yap deserves a raise. Thank you for reading. With this, I hope readers will really question things before declaring them as true - actually research, dig in, and get your hands dirty, rather than just believing something because someone said so.

PEACE !!!

References:

• https://developer.chrome.com/docs/capabilities/web-apis/fetch-streaming-requests
• https://datatracker.ietf.org/doc/html/rfc2616
• https://datatracker.ietf.org/doc/html/rfc7231