GitOps Bridge Pattern On A Local Kind Cluster

Last week, I had the opportunity to attend the “Bridging Worlds: The Magic of GitOps in Kubernetes” virtual event organized by Cloud Native Nairobi. The session was eye-opening, particularly the deep dive into the GitOps Bridge Pattern – a powerful approach that combines Infrastructure as Code (IaC) and GitOps methodologies for managing Kubernetes resources.

The Problem the Pattern Solves

During the event, the speaker highlighted a common challenge in Kubernetes environments: while we use IaC tools like Terraform to provision cloud resources, and GitOps tools like ArgoCD to manage Kubernetes workloads, there’s often a disconnect between these two worlds.

The crux of the issue lies in Kubernetes addons that depend on external cloud resources. These addons need configuration metadata from those external resources, but traditionally, there hasn’t been a clean way to bridge the gap between the IaC-provisioned resources and the GitOps-managed deployments.

My Local Implementation

Inspired by the event, I decided to implement the GitOps Bridge Pattern myself using a local Kind cluster. This would allow me to test the pattern without incurring cloud costs while gaining hands-on experience with this architectural approach.

I set up a project that utilizes Terraform to configure the overall environment and ArgoCD to manage the actual Kubernetes resources. My approach involved:

1. Creating a local Kind cluster as the foundation
2. Using Terraform with the gitops-bridge Helm module to establish the bridge
3. Configuring ArgoCD to monitor a Git repository for changes
4. Deploying both a simple application (Nginx) and addons(helm charts)- Prometheus monitoring stack

The Implementation Details

The full project and setup instructions can be found in : https://github.com/markbosire/gitops-repo

The heart of my implementation is the Terraform configuration that establishes the GitOps bridge. In my main.tf file, I defined the necessary providers for Kubernetes and Helm, then configured the locals block with my cluster details and addon preferences:

locals {
  cluster_name = "gitops-demo"
  environment  = "dev"
  gitops_repo  = "https://github.com/markbosire/gitops-repo"

  # OSS addons configuration
  oss_addons = {
    enable_kube_prometheus_stack = true  
    enable_prometheus_adapter    = true  
  }

  # Merge all addon categories
  addons = merge(
    local.oss_addons,
    {
      kubernetes_version = "1.26"
    }
  )
}

The magic happens in the gitops_bridge module, which takes these configurations installs ArgoCD and creates the necessary ArgoCD applications:

module "gitops_bridge" {
  source = "gitops-bridge-dev/gitops-bridge/helm"

  cluster = {
    cluster_name = local.cluster_name
    environment  = local.environment
    metadata     = merge(
      {
        repo_url  = local.gitops_repo
        repo_path = "apps"
      },
      local.addons_metadata
    )
    addons = local.addons
  }

  apps = local.argocd_apps
}

For the applications side, I created ArgoCD application manifests like nginx.yaml:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: nginx
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/markbosire/gitops-repo
    path: nginx-manifests
    targetRevision: HEAD
  destination:
    server: https://kubernetes.default.svc
    namespace: default
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

And for the addons, I utilized an ApplicationSet (addons.yaml) that dynamically creates helm charts found in the addons folder in the repo:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: cluster-addons
  namespace: argocd
spec:
  generators:
    - clusters:
        selector:
          matchLabels:
            environment: dev
  template:
    metadata:
      name: 'bootstrap-addons'
    spec:
      project: default
      source:
        repoURL: 'https://github.com/markbosire/gitops-repo'
        path: 'addons'
        targetRevision: 'main'
        directory:
          recurse: true
      destination:
        namespace: 'argocd'
        name: '{{name}}'
      syncPolicy:
        automated: {}

The “Aha” Moment

Running terraform apply brought everything together. I watched as Terraform handled the initial setup and then ArgoCD took over, continuously ensuring my cluster state matched what was defined in my Git repository. When I accessed the ArgoCD UI, I could see my Nginx application and the Prometheus stack being deployed and maintained automatically.

The most satisfying part was seeing how cleanly the pattern handled the Prometheus stack deployment. The ApplicationSet for kube-prometheus-stack was particularly elegant:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: kube-prometheus-stack
  namespace: argocd
spec:
  generators:
    - clusters:
        selector:
          matchLabels:
            environment: dev
  template:
    metadata:
      name: 'kube-prometheus-stack'
    spec:
      project: default
      source:
        chart: kube-prometheus-stack
        repoURL: https://prometheus-community.github.io/helm-charts
        targetRevision: 55.4.0
        helm:
          valueFiles:
            - values.yaml
      destination:
        server: https://kubernetes.default.svc
        namespace: monitoring
      syncPolicy:
        automated: {}
        syncOptions:
          - CreateNamespace=true
          - ServerSideApply=true

What I Learned

Implementing the GitOps Bridge Pattern in my local environment taught me several valuable lessons:

1. The bridge is real: The pattern effectively connects the IaC world with the GitOps world, allowing for seamless management of resources across both domains.

2. Local testing is powerful: By using Kind, I could experiment freely without worrying about cloud costs or affecting cloud environments.

3. Automation is transformative: Once set up, the automatic synchronization between my Git repository and cluster state eliminated manual deployment steps and configuration drift.

4. Monitoring is integrated: Including the Prometheus stack demonstrated how the pattern can handle complex addons with multiple components.

Next Steps

Having successfully implemented the pattern locally, my next steps are to:

1. Deploy this pattern in a cloud environment where I can work with real external resources
2. Expand the addons to include other common tools like Ingress controllers and certificate managers
3. Implement a full CI/CD pipeline that updates the Git repository, triggering the GitOps process

The Cloud Native Nairobi event opened my eyes to the GitOps Bridge Pattern, but actually implementing it myself solidified my understanding. If you’re working with Kubernetes and struggling with the gap between IaC and GitOps tools, I highly recommend giving this pattern a try – even if just in a local Kind cluster to start.