DevOps Fundamentals

DevOps is not a job title, a tool, or a department. It is a cultural and professional movement that tears down the wall between software developers (who build features) and operations engineers (who keep systems running). Before DevOps, these two groups worked in silos — developers threw code "over the fence" to operations, who then struggled to deploy software they had not built. The result was slow releases, frequent failures, and endless blame cycles. DevOps replaces that wall with shared ownership, shared tools, and shared responsibility for the entire software lifecycle.

The term was popularised at the Velocity 2009 conference by John Allspaw and Paul Hammond of Flickr, who described deploying over 10 times a day — an unimaginable frequency at the time. The core insight: if developers and operations share the same goals and feedback loops, software can be shipped safely at high speed. This was the beginning of what we now call DevOps. Learn with LSGP introduces this philosophy early because you cannot understand any specific DevOps tool without understanding the problem it is solving.

The CALMS Framework

CALMS is the most widely cited framework for understanding DevOps culture:

The Four DORA Metrics

DORA (DevOps Research and Assessment) metrics are the industry standard for measuring DevOps team performance. Every DevOps interview at a top company will expect you to know these:

Git is the foundation of every DevOps workflow. Without version control, there is no reliable way to track changes, collaborate across teams, roll back broken code, or trigger automated pipelines. Git is not just a file backup system — it is the single source of truth for your entire codebase's history, and in a GitOps workflow, it is also the source of truth for your infrastructure's desired state.

Every DevOps tool — CI/CD systems, deployment platforms, code review workflows — plugs into Git. A push to a branch triggers a build. A merged pull request triggers a deployment. A tagged commit triggers a release. Git is the heartbeat of the DevOps pipeline.

Core Git Concepts

• Repository (repo): The directory containing your project and its full version history, stored in a hidden .git folder.
• Commit: A snapshot of all tracked files at a point in time. Each commit has a unique SHA hash, an author, a timestamp, and a message. Commits are immutable — you never rewrite a committed hash in shared history.
• Branch: A lightweight pointer to a commit. Creating a branch is free and instant. Branches let you work on features in isolation without affecting the main codebase.
• Merge: Combines two branches. A merge commit records the integration point. Fast-forward merge is used when the base branch has not diverged.
• Rebase: Replays commits from one branch on top of another, creating a linear history. Preferred for cleaner history on feature branches before merging.
• Remote: A copy of the repository hosted on a server (GitHub, GitLab, Bitbucket). git push and git pull synchronise local and remote.

Branching Strategies

A branching strategy defines how your team uses branches to organise work and releases. The two most common in DevOps:

# Start a new feature
git checkout -b feature/user-auth

# Stage and commit changes
git add .
git commit -m "feat: add JWT login endpoint"

# Keep branch up to date with main
git fetch origin
git rebase origin/main

# Push and open a pull request
git push origin feature/user-auth

# After PR review and approval — squash merge
git checkout main
git merge --squash feature/user-auth
git commit -m "feat: user authentication with JWT (#42)"

Conventional Commits

Conventional Commits is a standard for writing commit messages that can be parsed by tools. Format: type(scope): description. Common types: feat (new feature), fix (bug fix), docs, chore, refactor, test. Following this standard lets you auto-generate changelogs and trigger semantic versioning automatically in your CI pipeline.

Continuous Integration (CI) is the practice of automatically building and testing code every time a developer pushes a commit. Continuous Delivery (CD) extends CI by automatically deploying code that passes all tests to a staging or production environment. Together, CI/CD is the engine of modern software delivery — replacing manual, error-prone deploy procedures with a reliable, repeatable automated pipeline.

The CI/CD Pipeline Visualised

GitHub Actions — A Real Pipeline

GitHub Actions is the most accessible CI/CD platform for beginners. Workflows are YAML files in .github/workflows/ that define what runs on which events.

name: CI Pipeline

on:
  push:
    branches: [main, 'feature/**']
  pull_request:
    branches: [main]

jobs:
  test-and-build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run linter
        run: npm run lint

      - name: Run tests
        run: npm test -- --coverage

      - name: Build Docker image
        run: docker build -t lsgp-app:${{ github.sha }} .

      - name: Push to registry (main only)
        if: github.ref == 'refs/heads/main'
        run: |
          docker tag lsgp-app:${{ github.sha }} ghcr.io/lsgp/app:latest
          docker push ghcr.io/lsgp/app:latest

CI/CD Best Practices

• Fast feedback: A CI pipeline that takes 30 minutes kills velocity. Target under 10 minutes for the critical path. Run the fastest checks (lint, unit tests) first; defer slow integration tests.
• Fail fast: If any step fails, abort the pipeline immediately. Do not deploy broken code; do not waste compute running further steps on a broken build.
• Immutable artefacts: Build the Docker image once and promote the same image through staging → production. Never build separately for different environments — that defeats reproducibility.
• Environment parity: Staging should mirror production as closely as possible. Surprises in production are often caused by differences from staging (different OS, different library versions, different config).
• Secrets management: Never put passwords, API keys, or tokens in source code or CI config YAML. Use the CI platform's secrets store (GitHub Secrets, GitLab CI Variables) injected as environment variables at runtime.

Docker solved one of the most persistent problems in software deployment: "it works on my machine." A container packages your application and all its dependencies — the runtime, libraries, config, and code — into a single portable unit that runs identically on any machine with Docker installed. Containers are not virtual machines (VMs). VMs virtualise hardware and run a full OS; containers share the host OS kernel and are isolated using Linux namespaces and cgroups. This makes containers dramatically lighter, faster to start, and more efficient.

Key Docker Concepts

• Image: A read-only template for creating containers. Built from a Dockerfile. Layers are cached — only changed layers re-download. Images are immutable.
• Container: A running instance of an image. Isolated process with its own filesystem, network, and process space. Ephemeral by default — data is lost when the container stops unless you use volumes.
• Dockerfile: A text script of instructions for building an image. Each instruction creates a layer. Layers are cached, so ordering matters — put rarely changed instructions (installing dependencies) before frequently changed ones (copying source code).
• Registry: A storage and distribution system for Docker images. Docker Hub is the public registry. GitHub Container Registry (GHCR), Amazon ECR, and Google Artifact Registry are private options.
• Volume: Persistent storage that survives container restarts. Mounted from the host or managed by Docker. Used for databases, file uploads, and any data that must persist.

# Stage 1: build
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production   # install deps first (cached)
COPY . .
RUN npm run build

# Stage 2: minimal production image
FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
USER node              # never run as root
EXPOSE 3000
CMD ["node", "dist/index.js"]

Docker Compose — Multi-Container Apps

Most real applications need multiple containers working together — a web server, a database, a cache, a background worker. Docker Compose defines and runs multi-container applications with a single YAML file.

services:
  web:
    build: .
    ports: ["3000:3000"]
    environment:
      DATABASE_URL: postgresql://user:pass@db:5432/lsgp
    depends_on: [db, redis]

  db:
    image: postgres:16-alpine
    volumes: [pgdata:/var/lib/postgresql/data]
    environment:
      POSTGRES_PASSWORD: pass

  redis:
    image: redis:7-alpine

volumes:
  pgdata:

Kubernetes — Container Orchestration at Scale

Docker runs containers on one machine. Kubernetes (K8s) orchestrates containers across a cluster of machines. It handles scheduling (which node runs which container), scaling (run 10 replicas of a service), self-healing (restart failed containers, replace unhealthy nodes), and service discovery (route traffic to the right pods). Kubernetes is the standard for production-scale container workloads. Key concepts: Pods (smallest deployable unit), Deployments (declare desired state), Services (stable networking), Ingress (external HTTP routing), ConfigMaps and Secrets (configuration injection). Learn with LSGP covers Kubernetes in depth in its Cloud and DevOps advanced track.

Before Infrastructure as Code (IaC), servers were configured by hand — someone SSHed in, ran some commands, installed packages, and hoped they remembered everything for next time. This is called a "snowflake server" — unique, fragile, and impossible to reproduce. IaC replaces this with code: you declare what you want (a web server with 8 cores, a managed database, a load balancer), and the tool provisions it automatically. The code lives in Git — versioned, reviewed, auditable, and reproducible.

Terraform — Declarative Cloud Infrastructure

Terraform (by HashiCorp) is the dominant IaC tool for cloud infrastructure. You write HCL (HashiCorp Configuration Language) files declaring the desired state, and Terraform figures out what to create, change, or destroy to reach that state. It works across AWS, GCP, Azure, and dozens of other providers.

resource "aws_instance" "web" {
  ami           = "ami-0c02fb55956c7d316"
  instance_type = "t3.micro"
  key_name      = var.key_name

  tags = {
    Name    = "lsgp-web-server"
    Project = "learnwithlsgp"
  }
}

output "public_ip" {
  value = aws_instance.web.public_ip
}

# terraform init → terraform plan → terraform apply

Ansible — Configuration Management

Terraform provisions infrastructure; Ansible configures it. Ansible connects to servers over SSH and runs "playbooks" — YAML files describing the desired state of the server's configuration: which packages are installed, which files exist, which services are running. Ansible is agentless (no software needed on the managed machine), idempotent (running a playbook twice produces the same result), and human-readable.

The IaC Workflow in Practice

1. Write infrastructure code in HCL/YAML and commit to a Git repository.
2. Open a pull request. A CI pipeline runs terraform plan and posts the diff as a PR comment — reviewers see exactly what will change before approving.
3. After approval and merge, a CD pipeline runs terraform apply automatically to enforce the declared state.
4. If something goes wrong, roll back by reverting the Git commit — the previous state is re-applied automatically.

Deploying code is not the end of the story. Once software is in production, it must be observed — are requests succeeding? Is latency acceptable? Is memory growing unbounded? Are errors spiking? Without monitoring, you are flying blind, and your users discover problems before you do. Learn with LSGP treats observability as a first-class engineering discipline, not an afterthought.

The Three Pillars of Observability

SLIs, SLOs, and SLAs

Site Reliability Engineering (SRE) — Google's approach to running production systems — introduced a clear language for reliability targets:

• SLI (Service Level Indicator): The metric you measure. Example: "the ratio of successful HTTP requests to total requests."
• SLO (Service Level Objective): The target for your SLI. Example: "99.9% of requests succeed over a rolling 30-day window."
• SLA (Service Level Agreement): The contractual commitment to your customers. If you breach it, there are consequences (credits, refunds). The SLO should be tighter than the SLA to give you headroom.
• Error Budget: 100% − SLO. If your SLO is 99.9%, your error budget is 0.1% — about 43 minutes of downtime per month. When the budget is healthy, ship fast. When it's burning, slow down and focus on reliability.

Alerting Done Right

Alert on symptoms, not causes. Alert when users are being hurt — high error rate, high latency, unavailability — not on every CPU spike. Each alert should be actionable: if an engineer gets paged and cannot do anything about it, the alert is noise. Keep your on-call rotation humane: too many alerts cause alert fatigue, which causes engineers to start ignoring pages. A good alert has a clear owner, a runbook (step-by-step guide to investigate and resolve), and a severity level that dictates whether to wake someone at 3am.

DevSecOps extends DevOps by integrating security at every stage of the pipeline — "shift left" security means catching vulnerabilities in development rather than in production. Traditionally, a security team reviewed code at the end of a release cycle. In DevSecOps, automated security checks run on every commit, just like tests.

Security at Each Pipeline Stage

The Principle of Least Privilege

Every service, user, and process should have exactly the permissions it needs — nothing more. A web server should not have write access to the database schema. A CI pipeline should not have production deploy permissions just to run tests. A developer workstation should not have production database credentials. Least privilege limits the blast radius when any component is compromised.

DevOps and cloud engineering are among the highest-demand and highest-paid specialisations in the software industry globally and in India. The transition from software developer to DevOps engineer is natural — strong programming skills are a prerequisite, and most great DevOps engineers are former developers who developed a passion for infrastructure, automation, and reliability.

Core Skills Roadmap

Certifications Worth Pursuing