Chapter 6: Building a Backend from Scratch: How a Node.js Server Actually Works

project/
├── package.json
├── app.js              ← The starting point of our entire application
├── db.js               ← Database connection setup
└── features/
    └── users/
        ├── user.model.js       ← Defines what a "user" looks like in the database
        ├── user.service.js     ← Business logic (the real work)
        ├── user.controller.js  ← Translates HTTP requests into service calls
        └── user.routes.js      ← Maps URLs to controller functions

{
  "name": "my-backend",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    "start": "node app.js"
  },
  "dependencies": {
    "cors": "^2.8.5",
    "express": "^4.21.2",
    "mongoose": "^8.9.5"
  }
}

// app.js
import http from "node:http";

const server = http.createServer((req, res) => {
  res.writeHead(200, { "Content-Type": "text/plain" });
  res.end("Hello World!\n");
});

server.listen(3000, () => {
  console.log("Server is running on port 3000");
});

node app.js

Hello World!

Your computer is running many processes:
├── Chrome Browser      (PID: 1234)
├── VS Code             (PID: 5678)
├── Spotify             (PID: 9012)
└── node app.js         (PID: 3456)  ← Our new process!

const server = http.createServer((req, res) => { ... });

Your Computer (Building at 192.168.1.5)
├── Port 80    → Web server (Apache)
├── Port 443   → HTTPS server
├── Port 3000  → Our Node.js app ← This is us!
├── Port 5432  → PostgreSQL database
└── Port 27017 → MongoDB

┌─────────────────────────────────────────────────────┐
│  1. OPERATING SYSTEM (OS)                           │
│     The boss. Owns the network card, manages ports, │
│     and actually sends/receives data over the wire. │
└──────────────────────▲──────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────┐
│  2. libuv                                           │
│     Node's tireless watchman. It sits by the        │
│     network door and shouts "Hey, someone's here!"  │
│     whenever data arrives on a socket.              │
└──────────────────────▲──────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────┐
│  3. V8 Engine                                       │
│     The brain. It reads and executes your           │
│     JavaScript code line by line.                   │
└──────────────────────▲──────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────┐
│  4. Your JavaScript Code                            │
│     The instructions you wrote in app.js.           │
└─────────────────────────────────────────────────────┘

Node.js Process
├── fd 0  → Standard Input (your keyboard)
├── fd 1  → Standard Output (your terminal screen)
├── fd 2  → Standard Error (error messages)
└── fd 18 → TCP Socket listening on port 3000

curl http://localhost:3000

1. YOUR BROWSER
   │  Types the URL and hits Enter
   │
   │  "I need to talk to localhost on port 3000"
   ▼

2. TCP CONNECTION (The Handshake)
   │  Your browser and server do a "3-way handshake":
   │    Browser: "Hey, can we talk?" (SYN)
   │    Server:  "Sure, I'm ready!"  (SYN-ACK)
   │    Browser: "Great, let's go!"  (ACK)
   │
   │  Now there's an open two-way channel between them.
   ▼

3. OPERATING SYSTEM
   │  The OS receives the raw network packets on port 3000.
   │  It creates a new dedicated socket just for THIS conversation.
   │  It notifies libuv: "Hey, data arrived!"
   ▼

4. libuv (Event Loop)
   │  Wakes up and reads the raw bytes from the socket.
   │  Passes those bytes to Node's HTTP parser.
   ▼

5. HTTP PARSER (llhttp)
   │  Reads the raw text:
   │    "GET / HTTP/1.1\r\nHost: localhost:3000\r\n\r\n"
   │
   │  Extracts:
   │    Method: GET
   │    Path: /
   │    Headers: Host=localhost:3000
   │
   │  Creates two JavaScript objects:
   │    req (the incoming request)
   │    res (the outgoing response)
   ▼

6. YOUR CALLBACK FUNCTION
   │  V8 runs your code:
   │    (req, res) => { res.end("Hello World!\n"); }
   │
   │  Your code writes "Hello World!" into the response.
   ▼

7. RESPONSE GOES BACK
   │  Node serializes the response into HTTP format:
   │    "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\n\r\nHello World!\n"
   │
   │  libuv sends these bytes back through the socket.
   │  The OS transmits them over the network.
   │  The browser reads them and displays "Hello World!"

TCP Connection (opened once)
├── HTTP Request 1: GET /index.html
├── HTTP Request 2: GET /styles.css
├── HTTP Request 3: GET /logo.png
└── (Connection stays open for a while, then closes)

// db.js
import mongoose from "mongoose";

async function connectDB() {
  const uri = process.env.MONGO_URI || "mongodb://127.0.0.1:27017/my_app_db";

  try {
    const conn = await mongoose.connect(uri, {
      maxPoolSize: 10,
      serverSelectionTimeoutMS: 5000,
    });

    console.log(`[DB] Connected to MongoDB at ${conn.connection.host}`);
  } catch (error) {
    console.error("[DB] Failed to connect:", error.message);
    process.exit(1);
  }
}

export default connectDB;

┌──────────────────────────┐         ┌──────────────────────────┐
│   Your Node.js Server    │  TCP    │     MongoDB Server       │
│                          │ ──────► │                          │
│   Listens on port 3000   │         │   Listens on port 27017  │
│   (for HTTP clients)     │ ◄────── │   (for database queries) │
└──────────────────────────┘         └──────────────────────────┘

} catch (error) {
  console.error("[DB] Failed to connect:", error.message);
  process.exit(1);
}

// ❌ WRONG: The Naive Approach
import http from "node:http";
import connectDB from "./db.js";

connectDB();       // Starts connecting (but doesn't wait!)
server.listen(3000); // Opens the door immediately!

Timeline:
  0ms:  Node starts
  1ms:  connectDB() begins TCP handshake to MongoDB... (still working)
  2ms:  server.listen(3000) opens port 3000 to the public
 10ms:  A user visits http://localhost:3000/users
 11ms:  Your code tries User.find() → 💥 CRASH! Database not connected yet!
200ms:  MongoDB connection finally completes (too late for that user!)

// ✅ CORRECT: Wait for dependencies before accepting traffic
import http from "node:http";
import connectDB from "./db.js";

const PORT = process.env.PORT || 3000;

async function startServer() {
  try {
    // Step 1: Wait for database connection FIRST
    await connectDB();

    // Step 2: Create the server
    const server = http.createServer((req, res) => {
      res.writeHead(200, { "Content-Type": "application/json" });
      res.end(JSON.stringify({ status: "ready" }));
    });

    // Step 3: ONLY open the port after database is confirmed ready
    server.listen(PORT, () => {
      console.log(`[Server] Ready on port ${PORT}`);
    });
  } catch (error) {
    console.error("[Server] Failed to start:", error);
    process.exit(1);
  }
}

startServer();

1. Process starts
      ↓
2. Connect to database (await — waits until done)
      ↓
3. Database confirmed connected ✅
      ↓
4. Open port 3000 to the public
      ↓
5. Ready to safely handle requests

curl http://localhost:3000/users
curl http://localhost:3000/products
curl -X POST http://localhost:3000/users

Route = HTTP Method + URL Path

GET    /users       →  "Show me all users"
POST   /users       →  "Create a new user"
GET    /users/123   →  "Show me user number 123"
DELETE /users/123   →  "Delete user number 123"

// Routing with raw Node.js (ugly but works)
const server = http.createServer((req, res) => {
  if (req.method === "GET" && req.url === "/users") {
    res.end("Here are the users");
  } else if (req.method === "POST" && req.url === "/users") {
    // To read the POST body, we need to:
    let body = "";
    req.on("data", (chunk) => { body += chunk; });
    req.on("end", () => {
      const parsed = JSON.parse(body);
      res.end("Created user: " + parsed.name);
    });
  } else if (req.method === "GET" && req.url.startsWith("/users/")) {
    const id = req.url.split("/")[2]; // Manual URL parsing 😬
    res.end("User ID: " + id);
  } else {
    res.writeHead(404);
    res.end("Not Found");
  }
});

npm install express cors

// app.js — now with Express
import express from "express";
import connectDB from "./db.js";

const app = express();
const PORT = process.env.PORT || 3000;

app.get("/users", (req, res) => {
  res.json({ message: "List of users" });
});

async function startServer() {
  try {
    await connectDB();
    app.listen(PORT, () => {
      console.log(`[Express] Running on port ${PORT}`);
    });
  } catch (error) {
    console.error("[Express] Startup failed:", error);
    process.exit(1);
  }
}

startServer();

app.listen(3000);

import http from "node:http";

const server = http.createServer(app);  // Express is just a callback function!
server.listen(3000);

Your Express Code (app.get, app.post, middleware)
       ↓ sits on top of
Node.js HTTP Server (parses raw HTTP bytes into req/res objects)
       ↓ sits on top of
libuv (watches network sockets asynchronously)
       ↓ sits on top of
Operating System (manages TCP connections and hardware)

Incoming Request
       │
       ▼
┌──────────────────┐
│ Checkpoint 1     │  "Do you have a JSON body? Let me parse it for you."
│ (express.json()) │
└──────┬───────────┘
       │ next()  ← "All clear, move to next checkpoint"
       ▼
┌──────────────────┐
│ Checkpoint 2     │  "Do you have form data? Let me parse that too."
│ (urlencoded)     │
└──────┬───────────┘
       │ next()
       ▼
┌──────────────────┐
│ Checkpoint 3     │  "Is this request from a different website? Let me
│ (cors)           │   add the right permission headers."
└──────┬───────────┘
       │ next()
       ▼
┌──────────────────┐
│ Your Route       │  "GET /users? Here's your data!"
│ (handler)        │
└──────┬───────────┘
       │ res.json()  ← "Here's the response, send it back!"
       ▼
Response Sent

import express from "express";
import cors from "cors";
import connectDB from "./db.js";

const app = express();

// Middleware 1: Parse JSON request bodies
app.use(express.json());

// Middleware 2: Parse URL-encoded form data
app.use(express.urlencoded({ extended: true }));

// Middleware 3: Handle cross-origin requests
app.use(cors());

// features/users/user.routes.js
import { Router } from "express";

const router = Router();

router.get("/", (req, res) => {
  res.json({ users: [] });
});

router.get("/:id", (req, res) => {
  res.json({ id: req.params.id, name: "Sample User" });
});

export default router;

import userRoutes from "./features/users/user.routes.js";

// Mount all user routes under the /api/users prefix
app.use("/api/users", userRoutes);

app.use("/api/users", userRoutes);
app.use("/api/products", productRoutes);
app.use("/api/orders", orderRoutes);

Check /api/users     → URL starts with /api/products, NOT /api/users → SKIP
Check /api/products  → MATCH! → Enter productRoutes → find /:id → handle it
Check /api/orders    → Never reached (already handled above)

// ❌ Everything jammed into one function
router.get("/:id", async (req, res) => {
  // Reading HTTP params
  // Validating input
  // Querying the database
  // Formatting the response
  // Handling errors
  // All in one place!
});

┌──────────────────────────────────────────────────────┐
│                    THE RESTAURANT                     │
│                                                       │
│  HOST (Routes)                                        │
│  "Welcome! Table for GET /api/users/123?"             │
│  → Directs you to the right waiter                    │
│                                                       │
│  WAITER (Controller)                                  │
│  "What would you like?" (reads req.params)            │
│  → Takes the order to the kitchen                     │
│  → Brings back the plate (sends res.json())           │
│                                                       │
│  CHEF (Service)                                       │
│  Cooks the food (business logic)                      │
│  → Doesn't know or care about the customer            │
│  → Just receives an order and produces a result       │
│                                                       │
│  PANTRY (Model)                                       │
│  The ingredients catalog (database schema)            │
│  → Defines what exists and how it's stored            │
│                                                       │
│  WAREHOUSE (MongoDB)                                  │
│  Where all ingredients are physically stored          │
└──────────────────────────────────────────────────────┘

// features/users/user.model.js
import mongoose from "mongoose";

const userSchema = new mongoose.Schema(
  {
    name: { type: String, required: true, trim: true },
    email: { type: String, required: true, unique: true, lowercase: true },
    role: { type: String, default: "member" },
  },
  { timestamps: true }
);

const User = mongoose.models.User || mongoose.model("User", userSchema);

export default User;

// features/users/user.service.js
import mongoose from "mongoose";
import User from "./user.model.js";

export async function getUserById(id) {
  if (!mongoose.Types.ObjectId.isValid(id)) {
    throw new Error("Invalid User ID format");
  }

  const user = await User.findById(id).lean();

  if (!user) {
    throw new Error("User not found");
  }

  return user;
}

export async function createUser(data) {
  const user = await User.create(data);
  return user.toObject();
}

// features/users/user.controller.js
import * as userService from "./user.service.js";

export async function getUser(req, res, next) {
  try {
    const { id } = req.params;
    const user = await userService.getUserById(id);

    return res.status(200).json({
      success: true,
      data: user,
    });
  } catch (error) {
    next(error);  // Pass the error to our global error handler
  }
}

export async function createUser(req, res, next) {
  try {
    const newUser = await userService.createUser(req.body);

    return res.status(201).json({
      success: true,
      data: newUser,
    });
  } catch (error) {
    next(error);
  }
}

// features/users/user.routes.js
import { Router } from "express";
import { getUser, createUser } from "./user.controller.js";

const router = Router();

router.get("/:id", getUser);
router.post("/", createUser);

export default router;

app.use((req, res, next) => {
  console.log("Request received at:", new Date());
  next();  // Move to the next middleware or route
});

app.get("/health", (req, res) => {
  res.json({ status: "ok" });  // Response sent. Request cycle ends.
});

app.use((req, res, next) => {
  if (!req.headers.authorization) {
    const error = new Error("No auth token provided");
    error.statusCode = 401;
    next(error);  // SKIP all remaining routes, jump to error handler
  } else {
    next();
  }
});

// This goes AFTER all routes in app.js
app.use((err, req, res, next) => {
  const statusCode = err.statusCode || 500;

  console.error(`[Error] ${req.method} ${req.url} → ${statusCode}: ${err.message}`);

  res.status(statusCode).json({
    success: false,
    error: {
      message: err.message || "Internal Server Error",
    },
  });
});

function normalMiddleware(req, res, next) {}
console.log(normalMiddleware.length);  // 3

function errorMiddleware(err, req, res, next) {}
console.log(errorMiddleware.length);  // 4

GET /api/users/65bb0a38e8202d6b1d1e44a2

 1. BROWSER
    │  User clicks a link or types the URL
    ▼

 2. TCP CONNECTION
    │  Browser connects to localhost:3000 (3-way handshake)
    ▼

 3. OPERATING SYSTEM
    │  Receives raw packets on port 3000
    │  Notifies libuv: "Data arrived!"
    ▼

 4. libuv
    │  Reads raw bytes from the network socket
    │  Passes them to Node's HTTP parser
    ▼

 5. HTTP PARSER (llhttp)
    │  Parses: "GET /api/users/65bb0a... HTTP/1.1"
    │  Creates req and res JavaScript objects
    │  Calls the Express app function
    ▼

 6. express.json()
    │  "Is there a JSON body?" → No, it's a GET request → next()
    ▼

 7. express.urlencoded()
    │  "Is there form data?" → No → next()
    ▼

 8. cors()
    │  Sets Access-Control-Allow-Origin header → next()
    ▼

 9. ROUTE MATCHING
    │  Checks: does /api/users/65bb0a... match /api/users? → YES!
    │  Strips prefix, passes /65bb0a... to userRoutes
    ▼

10. userRoutes
    │  Checks: does /65bb0a... match /:id? → YES!
    │  Sets req.params.id = "65bb0a38e8202d6b1d1e44a2"
    │  Calls getUser controller
    ▼

11. user.controller.js
    │  Extracts id from req.params
    │  Calls userService.getUserById(id)
    ▼

12. user.service.js
    │  Validates that ID is a valid format
    │  Calls User.findById(id)
    ▼

13. MONGOOSE → MongoDB
    │  Borrows an idle connection from the pool
    │  Sends database query to MongoDB over TCP (port 27017)
    │
    │  ════════════════════════════════════════════════
    │  THE NON-BLOCKING MAGIC:
    │  While MongoDB searches for the user on disk,
    │  Node.js does NOT sit frozen waiting!
    │  The event loop is FREE to handle other requests.
    │  Other users can simultaneously get responses.
    │  ════════════════════════════════════════════════
    │
    │  MongoDB finds the user and sends it back
    │  Mongoose converts the raw data into a JS object
    ▼

14. user.service.js (resumes)
    │  Returns the user object to the controller
    ▼

15. user.controller.js (resumes)
    │  Calls res.status(200).json({ success: true, data: user })
    ▼

16. EXPRESS → NODE HTTP SERVER
    │  Converts the JS object to JSON text
    │  Adds HTTP headers (Content-Type: application/json)
    ▼

17. libuv → OPERATING SYSTEM
    │  Sends the response bytes through the TCP socket
    ▼

18. BROWSER
    │  Receives JSON and displays the user data!

                             CLIENT
                     (Browser, Mobile App, curl)
                                │
                                ▼
                        OPERATING SYSTEM
                     (TCP Socket on Port 3000)
                                │
                                ▼
                              libuv
                     (Event Loop / I/O Watcher)
                                │
                                ▼
                        Node.js HTTP Server
                       (Parses HTTP → req/res)
                                │
                                ▼
                        Express Application
                                │
              ┌─────────────────┼─────────────────┐
              ▼                 ▼                 ▼
        express.json()        cors()       Other Middleware
        (Parse body)     (Browser perms)    (Auth, Logging)
              │                 │                 │
              └─────────────────┼─────────────────┘
                                │ next()
                                ▼
                           Root Router
                                │
                                ▼ Matches /api/users
                         User Feature Router
                   (features/users/user.routes.js)
                                │
                                ▼ Matches GET /:id
                         User Controller
                 (features/users/user.controller.js)
                                │
                                ▼ Calls service with plain data
                         User Service
                   (features/users/user.service.js)
                                │
                                ▼ Database query
                         User Model
                    (features/users/user.model.js)
                                │
                                ▼ TCP to port 27017
                             MongoDB
                                │
                                ▼ Data returned
                         User Service
                                │
                                ▼ Plain JavaScript object
                         User Controller
                                │
                                ▼ res.json(data)
                        Node.js HTTP Server
                                │
                                ▼ Response bytes
                              libuv
                                │
                                ▼
                        OPERATING SYSTEM
                                │
                                ▼ TCP packets
                             CLIENT