How I Scaled My First Production System as a University Student

I was a final-year university student when the system I built started falling apart under real traffic.

Not in one big dramatic crash. It was slower than that. Response times crept up. The billing API started timing out during peak morning hours when ISP operators collected payments from hundreds of customers at once. Support messages came in. I sat there refreshing server metrics on a dashboard, realizing I had built something real enough to break in real ways.

This is the story of how I fixed it. No managed services, no cloud budget, and no one to ask at 2 AM except Stack Overflow.

The system and what broke

At ShunnoIT, I built a multi-tenant billing SaaS for internet service providers. Each ISP got their own isolated tenant with their customers, plans, invoices, and payment history. The stack was straightforward: Express.js on Node.js, MongoDB as the primary database, all sitting on a single VPS.

It worked fine during development. It worked fine for the first few tenants. Then our sales team did their job too well.

More ISPs onboarded. More field operators started using the system at the same time. The load profile we designed for, a few dozen concurrent users, became a few hundred. And the system showed exactly what single-server architecture looks like under that kind of pressure:

• Response times crept from under 100ms to 600ms, then over a second
• The billing collection API, our most write-heavy endpoint, started timing out during peak morning load when operators collected cash in the field
• MongoDB queries that felt instant were now showing up in slow query logs
• The server CPU would spike above 90% and stay there

We had no budget for MongoDB Atlas or AWS RDS. What we had was limited hosting resources, and my job was to make our infrastructure carry a lot more weight.

Vertical scaling or horizontal scaling

The classic call. Throw bigger hardware at the problem, or distribute across more servers?

Vertical scaling is tempting because it is simple. No architectural changes, just pay for a bigger machine. But it has a ceiling. You can only buy so much CPU before costs explode, and it leaves you with one box. If that single server goes down, every ISP's billing system goes down with it.

Horizontal scaling requires more thought upfront. But for our situation it made total sense. Our API was stateless because we used JWT-based auth and stored no session state in process memory, so any backend could handle any request. Splitting the workload across multiple affordable servers gave us room to grow, and if one node failed, the rest kept serving traffic.

So we decided to go horizontal across both the application tier and the database tier. Here is how we put it together.

Step 1: Maximizing multi-core performance with PM2

The first bottleneck was right inside Node.js itself.

Node runs in a single process on a single thread for JavaScript execution. Even if your server has multiple CPU cores, running a plain node server.js command only uses one of them. The other cores sit idle while your single thread struggles under heavy request volume.

Instead of writing raw cluster boilerplate code from scratch, we used PM2.

PM2 is a production process manager for Node.js. It comes with built-in cluster mode that automatically spawns a worker process for every CPU core available on the machine. It shares the incoming port across all workers, restarts workers automatically if they crash, and allows zero-downtime reloads when pushing updates.

We created an ecosystem.config.js file:

// ecosystem.config.js
module.exports = {
  apps: [
    {
      name: "billing-api",
      script: "./server.js",
      instances: "max", // Automatically forks a worker per available CPU core
      exec_mode: "cluster",
      env: {
        NODE_ENV: "production",
        PORT: 3000,
      },
    },
  ],
};

Starting the app in cluster mode was as simple as:

pm2 start ecosystem.config.js

Instantly, the application was utilizing every core on the box. CPU utilization smoothed out, and response throughput jumped immediately.

Step 2: Distributing traffic across 3 backend servers with Nginx

PM2 solved our local core utilization, but running on one server still meant a single point of failure. If that machine had network trouble or rebooted, all billing operations halted.

To solve this, we set up 3 separate backend servers.

Each of the three servers ran an identical copy of the Node.js application managed by PM2. Then we placed an Nginx server in front of them to act as a reverse proxy and load balancer.

Incoming client requests hit Nginx first. Nginx then distributed the requests across the three backend servers using round-robin.

# /etc/nginx/sites-available/billing-api
upstream billing_backend {
    server 10.0.0.11:3000; # Backend Server 1
    server 10.0.0.12:3000; # Backend Server 2
    server 10.0.0.13:3000; # Backend Server 3

    keepalive 64;
}

server {
    listen 80;
    server_name api.yourdomain.com;

    location / {
        proxy_pass http://billing_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_cache_bypass $http_upgrade;
    }
}

Because our API was completely stateless, round-robin worked smoothly. Server 1, server 2, server 3, back to server 1. If we needed to pull one backend server down for maintenance or update the code, the other two kept handling requests without any dropped traffic.

Step 3: MongoDB replica set across 3 dedicated database servers

Once the application tier had plenty of breathing room, the database quickly surfaced as the next bottleneck.

Our app was read-heavy. ISP operators were constantly checking customer account status, viewing invoices, and browsing plans on the dashboard. Writes like payment records were critical, but reads outnumbered them by far. Yet every read and write was hammering a single MongoDB database server, pushing disk I/O and memory to the limit.

We needed read scaling and automatic failover, so we set up a 3-node MongoDB replica set across 3 different physical servers.

In this architecture, each replica set member lived on its own dedicated server:

Primary (Server 1)     <- All write operations go here
    | replication
Secondary (Server 2)   <- Serves read traffic
Secondary (Server 3)   <- Serves read traffic + election voting

On each database server, we configured /etc/mongod.conf with the replica set name and network bindings:

# /etc/mongod.conf on each database server
net:
  port: 27017
  bindIp: 0.0.0.0

replication:
  replSet: billing-rs

Then we initiated the replica set from the primary node:

// In mongo shell on Primary (10.0.0.21)
rs.initiate({
  _id: "billing-rs",
  members: [
    { _id: 0, host: "10.0.0.21:27017", priority: 2 },
    { _id: 1, host: "10.0.0.22:27017", priority: 1 },
    { _id: 2, host: "10.0.0.23:27017", priority: 1 },
  ],
});

In the Node.js connection string, we passed the full list of replica set servers so the MongoDB driver knew about the whole cluster:

const mongoose = require("mongoose");

mongoose.connect(
  "mongodb://10.0.0.21:27017,10.0.0.22:27017,10.0.0.23:27017/billing?replicaSet=billing-rs",
  {
    readPreference: "secondaryPreferred",
    w: "majority",
  }
);

The setting readPreference: "secondaryPreferred" was the game changer. Read queries, which made up the vast majority of our traffic, were automatically routed across the two secondary servers. That left the primary server completely unblocked to handle bill collections and writes.

Even better, if Server 1 went down, the remaining two nodes immediately held an election and promoted one of the secondaries to primary. No data loss, and no manual panic at midnight.

Step 4: Redis to stop hammering the database

Even with read replicas in place, some data was being fetched on nearly every API call.

Tenant configuration, service plan details, customer account status. Stuff that rarely changed but got queried repeatedly. We were still asking MongoDB for the same records over and over.

Adding Redis as an in-memory cache gave the database another layer of relief:

const redis = require("redis");
const client = redis.createClient({ host: "127.0.0.1", port: 6379 });

async function getTenantConfig(tenantId) {
  const cacheKey = `tenant:${tenantId}:config`;

  const cached = await client.get(cacheKey);
  if (cached) {
    return JSON.parse(cached);
  }

  const config = await TenantModel.findOne({ _id: tenantId }).lean();
  await client.setEx(cacheKey, 300, JSON.stringify(config));

  return config;
}

For data that almost never changes, like service plans, we used longer TTLs. For anything that changes on user action, like account status after a payment, we invalidated the cache key immediately after the write.

This simple cache layer cut a massive percentage of repetitive read queries before they ever reached the database.

What actually changed

After rolling out these changes, the results showed up in our metrics right away:

• Response times dropped from over a second under peak load back down to under 80ms
• The billing collection API stopped timing out. The morning rush was no longer enough to overwhelm it
• CPU usage remained steady across all backend instances instead of redlining on a single core
• The system gained high availability across both the backend instances and the database cluster
• Cost remained low. We used modest, cost-effective servers instead of expensive managed cloud services

The sales team kept onboarding ISPs, and the system handled the influx without breaking a sweat.

What I would do differently now

Looking back, here are a few things I learned that I carry into every project:

Automate deployments from day one. Managing 3 backend servers and 3 database servers manually with SSH, Git pulls, and system commands got messy fast. Deploying an update meant logging into each machine one by one. If I were setting this up today, I would use tools like Ansible or Docker Compose to make provisioning and deployments repeatable with one command.

Centralize logging and metrics early. Jumping between six different servers to check PM2 logs or MongoDB status during an incident was painful. Having a lightweight central logging tool or basic Prometheus dashboards would have saved hours of hunting down edge cases.

Start the replica set early. Setting up a multi-node replica set during initial setup is straightforward. Migrating a busy, standalone production database into a replica set while live users are making payments is nerve-racking.

Profile before you optimize. I added MongoDB indexes somewhat blindly at first. The right move is to run explain("executionStats") on slow queries first, confirm the exact index coverage, and only then create the index.

The thing I actually took away from this

When you cannot afford a managed cloud service, you are forced to learn how systems actually work under the hood.

No MongoDB Atlas meant setting up the replica set manually, learning how elections and heartbeats work, and understanding replica set lag. No managed cloud load balancer meant writing Nginx configurations by hand, learning reverse proxying, and understanding why stateless application design matters.

I spent late nights reading documentation, tuning configs, and testing failovers. No LLM, just me and the terminal. That kind of hands-on problem solving builds an intuition for systems that clicking a button in a cloud console simply cannot give you.

If you are working with limited hardware or budget constraints right now, lean into it. Those constraints often end up teaching you the most valuable lessons of your engineering career.