A year ago I couldn’t tell you what a PVC or ingress was. Today I’m recovering from broken clusters, migrating core services, and — most importantly — actually feeling confident. 🚀

#DevOps #learnk8s #100DaysOfCode #100DaysOfDevOps #homelab #kubernetes

- Stumbled upon mailgun, and am now using their free tier for my SMTP needs. Which aren't much.
- Deployed Komodo to monitor my docker containers.
#DevOps #learnk8s #100DaysOfCode #100DaysOfDevOps #homelab #kubernetes

I was experimenting with Longhorn and accidentally corrupted a few PVCs, including the one for Pi-hole. Since Pi-hole handles DNS inside my cluster, its downtime meant the entire cluster lost DNS. As a result, Infisical couldn’t resolve DNS, and my apps couldn’t access their secrets.

- Moved PiHole and Infisical (Secrets Manager) to external VMs outside of my K8s cluster. Not sure why I didn't do this when I initially built my cluster 🤷🏾‍♂️ . So what happened? #DevOps #learnk8s #100DaysOfCode #100DaysOfDevOps #homelab #kubernetes

- Fully migrated my K8s stack to RKE2. Having my apps "GItOps"ed made the process less painful then what it could've been.

#DevOps #learnk8s #100DaysOfCode #100DaysOfDevOps #homelab #kubernetes

In the end, I resolved it by just manually recreating the secret pihole was looking for.
✅

My secrets are managed by External Secrets, which was also failing because it couldn't resolve the dns name for Infisical because.... Pi-hole was down

After bringing my cluster back online, services were recovering except for a few. Pi-hole was complaining about a missing secret.

Oh wow! Thanks for sharing!!

Yeah this is the worst!

All is well now, though. This is why home labs are crucial. In my case, there's no actual negative consequences of breaking my homelab. This is literally my playground to break and learn #DevOps #k8s #learnk8s #cncf #100DaysOfCode #100DaysOfDevOps #kubernetes