GitOps with ArgoCD in Kubernetes - Deep Dive

Overview - GitOps with ArgoCD

What is it?

GitOps with ArgoCD is a way to manage and deploy applications on Kubernetes using Git as the single source of truth. ArgoCD watches your Git repository and automatically applies changes to your Kubernetes cluster to match what is defined in Git. This means your cluster state is always in sync with your code, making deployments predictable and easy to track.

Why it matters

Without GitOps and tools like ArgoCD, managing Kubernetes deployments can be manual, error-prone, and hard to track. Changes might be made directly on the cluster, causing confusion and drift from the desired state. GitOps ensures every change is recorded in Git, making deployments auditable, repeatable, and safer, which is crucial for reliable software delivery.

Where it fits

Before learning GitOps with ArgoCD, you should understand basic Kubernetes concepts like pods, deployments, and manifests, as well as Git basics like repositories and commits. After mastering GitOps, you can explore advanced topics like multi-cluster management, progressive delivery, and integrating GitOps with CI/CD pipelines.

Mental Model

Core Idea

GitOps with ArgoCD treats Git as the single source of truth and automatically syncs Kubernetes clusters to match the Git state.

Think of it like...

Imagine Git as a recipe book and ArgoCD as a chef who constantly checks the recipe book and makes sure the kitchen (Kubernetes cluster) is cooking exactly what the recipe says, no more, no less.

┌─────────────┐      ┌───────────────┐      ┌─────────────────────┐
│   Developer  │─────▶│     Git Repo   │─────▶│     ArgoCD Agent     │
└─────────────┘      └───────────────┘      └─────────┬───────────┘
                                                      │
                                                      ▼
                                             ┌─────────────────┐
                                             │ Kubernetes Cluster│
                                             └─────────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding Git as Source of Truth

Concept: Git stores the desired state of your Kubernetes applications as code.

Git is a system that tracks changes to files over time. In GitOps, you store Kubernetes manifests (configuration files) in Git. These files describe how your applications should run in the cluster.

Result

You have a clear, versioned record of your application's desired state.

Understanding Git as the single source of truth is key because it makes your infrastructure changes trackable and reversible.

2

Foundation

Basics of Kubernetes Manifests

Concept: Kubernetes manifests define the desired state of resources like pods and services.

Manifests are YAML files that describe what your application needs, such as how many replicas, what image to use, and network settings. Kubernetes reads these files to create and maintain resources.

Result

You can describe your application setup in files that Kubernetes understands.

Knowing how manifests work lets you control your cluster declaratively, which is essential for GitOps.

3

Intermediate

How ArgoCD Watches Git Repositories

🤔Before reading on: do you think ArgoCD pushes changes to Git or pulls changes from Git? Commit to your answer.

Concept: ArgoCD continuously monitors Git repositories and pulls changes to apply them to Kubernetes.

ArgoCD runs inside your cluster and regularly checks the Git repo for updates. When it detects changes, it compares the desired state in Git with the actual cluster state and applies updates to match Git.

Result

Your cluster automatically updates to reflect changes committed in Git.

Knowing ArgoCD pulls changes ensures you understand it reacts to Git commits, not manual cluster changes.

4

Intermediate

Syncing and Drift Detection

🤔Before reading on: do you think ArgoCD automatically fixes all differences between Git and cluster, or does it require manual approval? Commit to your answer.

Concept: ArgoCD detects when the cluster state drifts from Git and can sync them automatically or manually.

If someone changes the cluster directly, ArgoCD notices the difference (drift). Depending on settings, it can auto-correct by applying Git state or alert you to approve the sync.

Result

Your cluster stays consistent with Git, preventing unexpected changes.

Understanding drift detection helps prevent configuration drift, a common cause of bugs and outages.

5

Intermediate

Declarative vs Imperative Deployment

Concept: GitOps uses declarative deployment, describing desired state, unlike imperative commands that tell what to do step-by-step.

In imperative deployment, you run commands like 'kubectl apply' manually. In declarative GitOps, you declare the desired state in Git, and ArgoCD ensures the cluster matches it automatically.

Result

Deployments become more predictable and less error-prone.

Knowing the difference clarifies why GitOps reduces manual errors and improves automation.

6

Advanced

Managing Multiple Environments with ArgoCD

🤔Before reading on: do you think one Git repo can manage multiple environments like dev and prod, or do you need separate repos? Commit to your answer.

Concept: ArgoCD supports managing multiple environments from one or multiple Git repositories using application definitions and project scopes.

You can organize manifests in folders or branches for different environments. ArgoCD applications point to these locations and deploy accordingly, enabling environment-specific configurations.

Result

You can safely manage dev, staging, and production environments with clear separation and control.

Understanding environment management helps scale GitOps practices across teams and stages.

7

Expert

Handling Secrets and Sensitive Data Securely

🤔Before reading on: do you think storing secrets directly in Git is safe with GitOps? Commit to your answer.

Concept: GitOps requires special handling of secrets to avoid exposing sensitive data in Git repositories.

Tools like Sealed Secrets or External Secrets integrate with ArgoCD to encrypt or fetch secrets securely. This prevents secrets from being stored as plain text in Git while keeping deployments automated.

Result

Your deployments remain secure without sacrificing GitOps automation benefits.

Knowing secure secret management prevents critical security risks in GitOps workflows.

Under the Hood

ArgoCD runs as a Kubernetes controller that continuously polls configured Git repositories. It fetches manifests, compares them with the live cluster state using Kubernetes APIs, and applies Kubernetes manifests via the API server to reconcile differences. It maintains an internal cache of application states and reports sync status and health to users.

Why designed this way?

ArgoCD was designed to leverage Git's version control and Kubernetes' declarative API to automate deployments reliably. Pull-based syncing reduces security risks compared to push models, and continuous reconciliation ensures cluster state consistency. Alternatives like manual kubectl apply or CI/CD pipelines lacked this continuous, automated state enforcement.

┌───────────────┐       ┌───────────────┐       ┌─────────────────────┐
│   Git Server  │──────▶│   ArgoCD Agent │──────▶│ Kubernetes API Server│
└───────────────┘       └───────────────┘       └─────────────┬───────┘
                                                           │
                                                           ▼
                                                  ┌─────────────────┐
                                                  │ Kubernetes Cluster│
                                                  └─────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does ArgoCD push changes to the cluster or pull them from Git? Commit to your answer.

Common Belief: ArgoCD pushes changes to the cluster whenever you tell it to.

Reality: ArgoCD pulls changes from Git and applies them to the cluster automatically or on demand.

Why it matters: Thinking ArgoCD pushes changes can lead to confusion about how updates happen and security concerns about cluster access.

Quick: Is it safe to store Kubernetes secrets as plain text in Git with GitOps? Commit to your answer.

Common Belief: Storing secrets directly in Git is fine because GitOps automates deployments.

Reality: Storing secrets in plain text in Git is insecure; special tools or encryption must be used.

Why it matters: Exposing secrets can lead to security breaches and data leaks.

Quick: Does ArgoCD automatically fix all drift without user intervention? Commit to your answer.

Common Belief: ArgoCD always auto-syncs and fixes any drift immediately without approval.

Reality: ArgoCD can be configured for manual approval before syncing to prevent unintended changes.

Why it matters: Assuming auto-sync can cause unexpected disruptions in production environments.

Quick: Can one Git repository manage multiple Kubernetes environments easily? Commit to your answer.

Common Belief: You must use separate Git repositories for each environment like dev and prod.

Reality: One Git repository can manage multiple environments using folders, branches, or overlays.

Why it matters: Believing otherwise can lead to unnecessary repo sprawl and complex management.

Expert Zone

1

ArgoCD's reconciliation loop frequency affects how quickly drift is detected and fixed, balancing responsiveness and resource use.

2

Using ArgoCD projects allows fine-grained access control and resource limits, which is critical in multi-team environments.

3

ArgoCD supports custom health checks and hooks, enabling complex deployment strategies beyond simple sync.

When NOT to use

GitOps with ArgoCD is not ideal for applications requiring frequent manual interventions or dynamic runtime changes. Alternatives like imperative kubectl commands or CI/CD pipelines with manual steps may be better for such cases.

Production Patterns

In production, teams use ArgoCD with branching strategies for environment promotion, integrate it with CI pipelines for automated testing, and combine it with secret management tools for secure deployments. Multi-cluster setups use ArgoCD ApplicationSets to manage many clusters efficiently.

Connections

Infrastructure as Code (IaC)

GitOps builds on IaC principles by using Git to declare infrastructure state and automate deployments.

Understanding IaC helps grasp how GitOps treats infrastructure changes as code, enabling automation and version control.

Continuous Integration/Continuous Deployment (CI/CD)

GitOps complements CI/CD by automating deployment steps after code passes tests, ensuring cluster state matches Git.

Knowing CI/CD pipelines clarifies how GitOps fits into the software delivery lifecycle for faster, safer releases.

Version Control Systems (VCS)

GitOps relies on VCS like Git to track and manage changes, making VCS knowledge foundational.

Understanding version control concepts like commits, branches, and merges is essential to use GitOps effectively.

Common Pitfalls

#1 Applying changes directly to the Kubernetes cluster instead of through Git.

Wrong approach:

kubectl apply -f updated-manifest.yaml

Correct approach:

git commit -am 'Update manifest' && git push origin main

Root cause:

Misunderstanding that Git is the source of truth leads to manual cluster changes that cause drift.

#2 Storing Kubernetes secrets as plain text in Git repositories.

Wrong approach:

apiVersion: v1
kind: Secret
metadata:
name: my-secret
data:
password: cGFzc3dvcmQ= # base64 encoded password

Correct approach:

Use Sealed Secrets or External Secrets to encrypt or fetch secrets securely instead of storing them directly.

Root cause:

Lack of awareness about security risks and proper secret management in GitOps workflows.

#3 Assuming ArgoCD will always auto-sync changes without configuration.

Wrong approach:

Relying on ArgoCD default settings without setting sync policies or approvals.

Correct approach:

Configure ArgoCD sync policies explicitly to require manual approval in sensitive environments.

Root cause:

Overlooking ArgoCD's configurable sync behavior can cause unexpected production changes.

Key Takeaways

GitOps with ArgoCD uses Git as the single source of truth to manage Kubernetes deployments declaratively and automatically.

ArgoCD continuously monitors Git repositories and syncs the cluster state to match, preventing configuration drift.

Proper secret management is critical in GitOps to avoid exposing sensitive data in Git repositories.

Understanding the difference between declarative and imperative deployment helps appreciate GitOps benefits.

Advanced GitOps practices include managing multiple environments, access controls, and integrating with CI/CD pipelines for production readiness.