Overview
Kimbernetes uses Flux v2.7.5 to manage all cluster resources through GitOps. All changes are made via Git commits, which Flux automatically reconciles to the cluster.
Resource Structure
The repository is organized into:
cluster/kimawesome/ - Flux system configuration and cluster-level Kustomizationsoverlays/base/ - Base application definitions (HelmReleases, Deployments, Services)overlays/kimawesome/ - Environment-specific customizations
GitOps Workflow
Make changes in Git
Edit YAML files in your local clone of the repository:git clone ssh://[email protected]/kim-ae/kimbernetes-k8s-flux
cd kimbernetes-k8s-flux
Commit and push changes
git add .
git commit -m "Add new application"
git push origin main
Flux reconciles automatically
Flux polls the Git repository every minute and applies changes automatically. You can force immediate reconciliation:flux reconcile source git flux-system
flux reconcile kustomization flux-system
Flux CLI Commands
View Resource Status
# View all Flux resources
flux get all
# View Kustomizations
flux get kustomizations
# View HelmReleases
flux get helmreleases -A
# View GitRepositories
flux get sources git
View Logs
# View Flux controller logs
flux logs --level=error --all-namespaces
# View specific controller logs
flux logs --kind=Kustomization --name=flux-system
flux logs --kind=HelmRelease --name=cert-manager -n flux-system
Force Reconciliation
# Reconcile a Kustomization
flux reconcile kustomization overlays --with-source
# Reconcile a HelmRelease
flux reconcile helmrelease cert-manager -n flux-system
# Reconcile Git source
flux reconcile source git flux-system
Adding a New HelmRelease
Create the HelmRepository
Create overlays/base/myapp/helm-repository.yaml:apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
name: myapp
namespace: myapp
spec:
interval: 24h
url: https://charts.example.com
Create the HelmRelease
Create overlays/base/myapp/helm-release.yaml:apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: myapp
namespace: flux-system
spec:
chart:
spec:
chart: myapp
sourceRef:
kind: HelmRepository
name: myapp
namespace: myapp
version: "=1.2.3"
interval: 24h
releaseName: myapp
targetNamespace: myapp
install:
crds: Create
upgrade:
crds: CreateReplace
values:
replicas: 2
resources:
limits:
memory: 512Mi
Create the Kustomization
Create overlays/base/myapp/kustomization.yaml:resources:
- namespace.yaml
- helm-repository.yaml
- helm-release.yaml
Reference in parent Kustomization
Add to overlays/kimawesome/kustomization.yaml:resources:
- ../base/myapp
Commit and push
git add overlays/
git commit -m "Add myapp HelmRelease"
git push origin main
Pin versions using version: "=1.2.3" syntax to prevent automatic upgrades. See examples in overlays/base/sealed-secrets/helm-release.yaml and overlays/base/metallb/helm-release.yaml.
Adding Custom Kubernetes Resources
Create resource YAML
Create overlays/base/myapp/deployment.yaml:apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
namespace: myapp
spec:
replicas: 2
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: myapp:latest
ports:
- containerPort: 8080
Add to Kustomization
Create or update overlays/base/myapp/kustomization.yaml:resources:
- deployment.yaml
- service.yaml
Commit and push
git add overlays/base/myapp/
git commit -m "Add myapp deployment"
git push origin main
Creating a Flux Kustomization
Flux Kustomizations define what paths to reconcile from Git:
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: myapp
namespace: flux-system
spec:
interval: 10m
path: "./overlays/myapp"
prune: true
sourceRef:
kind: GitRepository
name: flux-system
healthChecks:
- apiVersion: apps/v1
kind: Deployment
name: myapp
namespace: myapp
Setting prune: true means Flux will delete resources removed from Git. Be careful when removing resources.
Modifying Existing Resources
Locate the resource
Find the YAML file in overlays/base/ or overlays/kimawesome/:find overlays/ -name "helm-release.yaml" -path "*/cert-manager/*"
Edit the file
Modify the resource definition:# overlays/base/cert-manager/helm-release.yaml
spec:
values:
resources:
limits:
memory: 256Mi # Changed from 128Mi
Commit and push
git add overlays/base/cert-manager/
git commit -m "Increase cert-manager memory limit"
git push origin main
Monitor reconciliation
flux reconcile helmrelease cert-manager -n flux-system
kubectl -n cert-manager get pods -w
Removing Resources
Remove from Kustomization
Delete or comment out the resource reference in kustomization.yaml:resources:
# - ../base/old-app # Removed
- ../base/cert-manager
Delete resource files (optional)
rm -rf overlays/base/old-app/
Commit and push
git add overlays/
git commit -m "Remove old-app"
git push origin main
Verify deletion
Flux will automatically delete the resources (if prune: true):kubectl get all -n old-app
Suspending and Resuming Resources
# Suspend a HelmRelease (prevents reconciliation)
flux suspend helmrelease myapp -n flux-system
# Resume a HelmRelease
flux resume helmrelease myapp -n flux-system
# Suspend a Kustomization
flux suspend kustomization overlays
# Resume a Kustomization
flux resume kustomization overlays
Suspending resources is useful during maintenance or troubleshooting to prevent Flux from reverting manual changes.
Checking Resource Health
# Check HelmRelease status
kubectl -n flux-system get helmrelease cert-manager -o yaml | grep -A 10 status
# Check Kustomization status
kubectl -n flux-system get kustomization overlays -o yaml | grep -A 10 status
# View events
kubectl -n flux-system get events --sort-by='.lastTimestamp'
Best Practices
- Always commit first: Never apply resources directly with
kubectl apply. Always commit to Git.
- Use version pinning: Pin Helm chart versions with
version: "=1.2.3" to prevent unexpected upgrades.
- Test in overlays: Use environment-specific overlays (
overlays/minikube/) for testing before production.
- Small commits: Make small, focused commits for easier rollback.
- Monitor reconciliation: Watch Flux logs during changes to catch issues early.
Next Steps