DevOps interview questions

DevOps: a culture and set of practices that unify software development (Dev) and IT operations (Ops) to shorten the delivery cycle and improve reliability. Traditional problems DevOps solves: - "Works on my machine": dev environment differs from production → containerization fixes this - Long release cycles: months between deployments → CI/CD enables daily/hourly deployments - Blame culture: dev throws code "over the wall" to ops → shared ownership across the lifecycle - Manual, error-prone deployments → automated pipelines - Slow feedback: bugs found weeks after code was written → shift-left testing catches issues early Core principles: - Continuous Integration: merge code frequently, run tests automatically on every commit - Continuous Delivery: code is always in a deployable state - Infrastructure as Code: servers and config defined in version-controlled files - Monitoring and feedback: observe production, feed insights back to development - Collaboration: shared tools, shared responsibility, shared metrics Metrics (DORA — DevOps Research and Assessment): - Deployment frequency: how often you deploy to production - Lead time for changes: commit to production time - Mean time to restore (MTTR): how fast you recover from failures - Change failure rate: % of deployments causing incidents Elite teams: deploy on-demand (multiple times/day), lead time < 1 hour, MTTR < 1 hour, change failure rate < 5%.

Docker: platform for building and running applications in containers — isolated, portable environments that package code + dependencies + runtime. Image: read-only template. Built from a Dockerfile. Each instruction creates a layer. Container: running instance of an image. Isolated process with its own filesystem, network, PID namespace. Layer: each Dockerfile instruction adds a read-only layer. Layers are cached and shared between images — if a layer hasn't changed, Docker reuses the cache. Dockerfile: FROM node:20-alpine # base image layer WORKDIR /app # set working directory COPY package*.json ./ # copy package files first (cache-friendly) RUN npm ci --only=production # install deps — cached if package.json unchanged COPY . . # copy source code EXPOSE 3000 CMD ["node", "server.js"] # default command Layer caching strategy: put things that change least at the top (base image, package install), things that change most at the bottom (source code). This way, a source code change doesn't invalidate the npm install layer. Key commands: docker build -t myapp:1.0 . # build image docker run -p 3000:3000 myapp:1.0 # run container docker ps # list running containers docker exec -it <id> sh # shell into container docker logs <id> # view logs docker stop <id> # stop container Image vs container analogy: image is a class, container is an object/instance. One image → many containers.

Multi-stage builds: use multiple FROM instructions in one Dockerfile. Each stage can copy artifacts from previous stages. Only the final stage ends up in the shipped image. Problem without multi-stage: A Go or Java app needs a compiler to build but not to run. A single-stage Dockerfile that includes the compiler results in a 1GB+ image containing build tools never needed at runtime. Multi-stage example (Go app): # Stage 1: build FROM golang:1.22-alpine AS builder WORKDIR /app COPY go.mod go.sum ./ RUN go mod download COPY . . RUN CGO_ENABLED=0 go build -o server . # Stage 2: minimal runtime image FROM scratch COPY --from=builder /app/server /server EXPOSE 8080 CMD ["/server"] Result: final image is just the compiled binary on a scratch (empty) base — ~10MB instead of ~300MB. Node.js example: FROM node:20-alpine AS deps WORKDIR /app COPY package*.json ./ RUN npm ci --only=production FROM node:20-alpine AS builder WORKDIR /app COPY --from=deps /app/node_modules ./node_modules COPY . . RUN npm run build FROM node:20-alpine AS runtime WORKDIR /app COPY --from=deps /app/node_modules ./node_modules COPY --from=builder /app/dist ./dist CMD ["node", "dist/server.js"] Benefits: - Smaller images: faster pulls, less storage, smaller attack surface - Build tools not in production: compilers, test frameworks, dev dependencies excluded - Better security: fewer packages = fewer CVEs

Docker Compose: tool for defining and running multi-container applications. A single docker-compose.yml file describes all services, networks, and volumes. docker-compose.yml example: version: '3.9' services: api: build: . ports: - "3000:3000" environment: - DATABASE_URL=postgres://user:pass@db:5432/myapp - REDIS_URL=redis://cache:6379 depends_on: db: condition: service_healthy cache: condition: service_started db: image: postgres:16-alpine environment: POSTGRES_USER: user POSTGRES_PASSWORD: pass POSTGRES_DB: myapp volumes: - pgdata:/var/lib/postgresql/data healthcheck: test: ["CMD-SHELL", "pg_isready -U user"] interval: 5s retries: 5 cache: image: redis:7-alpine volumes: pgdata: Commands: docker compose up -d # start all services in background docker compose down # stop and remove containers docker compose logs -f api # tail logs for api service docker compose exec api sh # shell into running api container docker compose ps # show service status When to use: - Local development: spin up entire stack (app + DB + Redis + queues) in one command - Integration testing in CI: bring up dependencies for test suite - Simple staging deployments When NOT to use: production at scale — use Kubernetes or Docker Swarm for production orchestration.

Kubernetes (K8s): open-source container orchestration platform. Automates deployment, scaling, and management of containerized workloads across a cluster of machines. Control Plane (runs on master nodes): - kube-apiserver: the front door — all kubectl commands and API calls go here. Validates and processes requests. - etcd: distributed key-value store — the cluster's source of truth. All cluster state (what's running, config) stored here. - kube-scheduler: watches for new Pods with no node assigned. Selects the best node based on resources, affinity, taints. - kube-controller-manager: runs controllers that watch state and reconcile toward desired state. ReplicaSet controller ensures correct Pod count. Node controller monitors node health. - cloud-controller-manager: integrates with cloud provider APIs (provision load balancers, storage volumes). Worker Nodes: - kubelet: agent on each node. Receives PodSpecs from API server, ensures containers are running and healthy. - kube-proxy: maintains network rules. Implements Service abstraction — routes traffic to correct Pod IPs using iptables or IPVS. - Container runtime: actually runs containers. containerd (default), CRI-O. Flow for a new deployment: kubectl apply → kube-apiserver validates + stores in etcd → controller creates ReplicaSet → ReplicaSet controller creates Pods → scheduler assigns Pods to nodes → kubelet on each node pulls image + starts container → kube-proxy updates routing rules. Key design principle: declarative + reconciliation loop. You declare desired state, controllers continuously reconcile actual state toward desired state.

Pod: the smallest deployable unit in Kubernetes. One or more containers sharing a network namespace (same IP) and storage volumes. Containers in a pod communicate via localhost. Most pods run one container. Multi-container pods use sidecar pattern: main app + logging agent, main app + service mesh proxy (Envoy). ReplicaSet: ensures a specified number of identical Pod replicas are running at all times. If a Pod dies, ReplicaSet creates a replacement. selector: matchLabels: app: api replicas: 3 Deployment: manages ReplicaSets. Adds rolling update and rollback capabilities on top of ReplicaSet. apiVersion: apps/v1 kind: Deployment metadata: name: api spec: replicas: 3 selector: matchLabels: app: api template: metadata: labels: app: api spec: containers: - name: api image: myacr.azurecr.io/api:v2 ports: - containerPort: 3000 resources: requests: cpu: "100m" memory: "128Mi" limits: cpu: "500m" memory: "256Mi" strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 maxUnavailable: 0 Hierarchy: Deployment → ReplicaSet → Pods You almost never create ReplicaSets directly — always use Deployments. kubectl rollout history deployment/api # see revision history kubectl rollout undo deployment/api # rollback to previous version kubectl rollout status deployment/api # watch rollout progress

Service: stable network endpoint that routes traffic to a set of Pods, selected by label. Pods are ephemeral (IPs change), Services provide a stable DNS name and IP. ClusterIP (default): virtual IP accessible only within the cluster. Use for: internal service-to-service communication. DNS: api-service.default.svc.cluster.local resolves to the ClusterIP. kube-proxy load-balances across all matching Pod IPs. NodePort: exposes the service on a static port (30000–32767) on every node's IP. Traffic: NodeIP:NodePort → ClusterIP → Pod Use for: dev/test, exposing services without a cloud load balancer. Not for production — exposes ports on all nodes. LoadBalancer: provisions a cloud load balancer (AWS ALB/NLB, Azure LB, GCP LB) with an external IP. Traffic: External IP → LB → NodePort → ClusterIP → Pod Use for: production internet-facing services. Costs money — each LoadBalancer service creates a cloud LB. ExternalName: maps a service to an external DNS name. Used to abstract external dependencies. Ingress (not a Service type, but commonly paired): HTTP layer-7 routing — one LoadBalancer for many services. rules: - host: api.myapp.com http: paths: - path: /v1 pathType: Prefix backend: service: name: api-v1-service port: number: 80 Ingress Controller (NGINX, Traefik, AWS ALB Ingress) processes Ingress resources. One LB for all HTTP/HTTPS traffic → route by host/path → different Services.

ConfigMap: stores non-sensitive key-value configuration data. Decouples config from container images. kubectl create configmap app-config --from-literal=LOG_LEVEL=debug --from-literal=MAX_CONNECTIONS=100 --from-file=config.yaml Secret: stores sensitive data. Encoded in base64 (NOT encrypted by default — use encryption at rest + sealed secrets for real security). kubectl create secret generic db-creds --from-literal=username=admin --from-literal=password=s3cr3t Two injection methods: 1. Environment variables: envFrom: - configMapRef: name: app-config - secretRef: name: db-creds # Or individual keys: env: - name: DB_PASSWORD valueFrom: secretKeyRef: name: db-creds key: password 2. Volume mount (better for files, certificates, large configs): volumes: - name: config-volume configMap: name: app-config - name: secret-volume secret: secretName: db-creds containers: - volumeMounts: - name: config-volume mountPath: /app/config - name: secret-volume mountPath: /app/secrets readOnly: true Secret security in production: - Enable etcd encryption at rest (Secrets are stored in etcd) - Use external secret managers: External Secrets Operator (syncs from AWS Secrets Manager, Azure Key Vault, GCP Secret Manager into Kubernetes Secrets) - Sealed Secrets: encrypt secrets before committing to Git (Bitnami Sealed Secrets)

HPA (Horizontal Pod Autoscaler): automatically scales the number of Pod replicas based on observed metrics. kubectl autoscale deployment api --cpu-percent=60 --min=2 --max=20 Or declarative YAML: apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: api-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: api minReplicas: 2 maxReplicas: 20 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 60 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 70 How it works: - Metrics Server collects CPU/memory from kubelets - HPA controller polls metrics every 15 seconds - Desired replicas = ceil(current * (currentMetric / targetMetric)) - Scale up immediately; scale down after cooldown period (default 5 min) to prevent thrashing Custom metrics (KEDA — Kubernetes Event Driven Autoscaling): Scale based on any external metric: SQS queue depth, Kafka consumer lag, Pub/Sub backlog, Redis list length, HTTP request rate. KEDA enables scale-to-zero for event-driven workloads. VPA (Vertical Pod Autoscaler): adjusts CPU/memory requests per pod — right-sizes pods. Use VPA for right-sizing, HPA for scaling out. Don't use both for the same resource metric simultaneously. Cluster Autoscaler: scales the node pool itself (adds/removes VMs) when pods can't be scheduled due to resource constraints.

Helm: package manager for Kubernetes. Bundle related Kubernetes resources into a "chart" — a collection of templated YAML files with default values. Chart structure: mychart/ Chart.yaml # chart metadata (name, version, description) values.yaml # default configuration values templates/ deployment.yaml # templated Kubernetes manifests service.yaml ingress.yaml _helpers.tpl # reusable template snippets values.yaml: replicaCount: 2 image: repository: myacr.azurecr.io/api tag: "1.0.0" pullPolicy: IfNotPresent service: type: ClusterIP port: 80 ingress: enabled: true host: api.myapp.com templates/deployment.yaml (using Go templating): spec: replicas: {{ .Values.replicaCount }} containers: - image: {{ .Values.image.repository }}:{{ .Values.image.tag }} Commands: helm install my-api ./mychart # install with defaults helm install my-api ./mychart -f prod-values.yaml # override values helm upgrade my-api ./mychart --set image.tag=2.0.0 # upgrade helm rollback my-api 1 # rollback to revision 1 helm list # list releases helm uninstall my-api # remove Why Helm: - Environment promotion: same chart, different values for dev/staging/prod - Rollback: helm keeps release history, one command rollback - Community charts: Artifact Hub has charts for Postgres, Redis, Kafka, Prometheus — install complex apps in minutes - Versioning: chart versions + app versions tracked together Helm 3 vs 2: Helm 3 removed Tiller (server-side component) — now client-only. Simpler, more secure.

Continuous Integration (CI): developers merge code to the main branch frequently (multiple times per day). Every merge triggers an automated pipeline: build + lint + unit tests + integration tests. Goal: catch integration bugs early. If the build breaks, the team fixes it immediately. Key practices: trunk-based development, feature flags, fast test suites (< 10 minutes). Continuous Delivery (CD): every code change that passes CI is automatically prepared for release to production. The software is always in a deployable state. Deployment is a manual decision — a human presses "deploy to production". Use when: regulated industries, need human approval before release. Continuous Deployment: every code change that passes CI/CD is automatically deployed to production without human intervention. Use when: high test confidence, fast rollback capability, e-commerce/SaaS where speed matters. Typical pipeline stages: 1. Source: git push triggers pipeline 2. Build: compile code, build Docker image 3. Test: unit tests, integration tests, static analysis, security scan (SAST) 4. Artifact: push image to registry, tag with commit SHA 5. Deploy to staging: update staging environment 6. Acceptance tests: E2E tests, smoke tests against staging 7. Manual approval (CD) OR auto-deploy (Continuous Deployment) to production 8. Monitor: observe error rates, latency — auto-rollback if thresholds breached Pipeline tools: - GitHub Actions: native to GitHub, YAML-based, large marketplace - GitLab CI: built into GitLab, powerful, self-hostable - Jenkins: most flexible/customizable, heavy ops overhead, on-prem friendly - CircleCI, TeamCity, Drone CI: alternatives

name: CI/CD Pipeline on: push: branches: [main] pull_request: branches: [main] env: REGISTRY: ghcr.io IMAGE_NAME: myorg/myapp jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Setup Node.js uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' - name: Install dependencies run: npm ci - name: Lint run: npm run lint - name: Run tests run: npm test -- --coverage - name: Upload coverage uses: codecov/codecov-action@v4 build-push: needs: test runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' permissions: contents: read packages: write steps: - uses: actions/checkout@v4 - name: Log in to registry uses: docker/login-action@v3 with: registry: ghcr.io username: myorg password: ${{ secrets.GITHUB_TOKEN }} - name: Extract metadata id: meta uses: docker/metadata-action@v5 with: images: ghcr.io/myorg/myapp tags: | type=sha,prefix=sha- type=raw,value=latest - name: Build and push uses: docker/build-push-action@v5 with: context: . push: true tags: ${{ steps.meta.outputs.tags }} cache-from: type=gha cache-to: type=gha,mode=max deploy: needs: build-push runs-on: ubuntu-latest environment: production steps: - name: Deploy to production run: | kubectl set image deployment/api api=ghcr.io/myorg/myapp:sha-${GITHUB_SHA::7}