Dude, Where's my Logs?
Fun! So, I started looking into our log stack, and realized we haz no logs…. lets figure this out.
I’d set up my grafana + loki + promtail + prometheus + syslog-ng rig with the help of a few blog posts.
- Dave Schmid made this blog post1 alongside an all-in-one repo2 This kinda started me down the path. I got it set up… and didn’t get to the point where everything is ACTUALLY making it into the cluster. …and then I got distracted….
However, I started digging again recently and managed to find a few spots I’d messed up and fixed them, an now we can start actually refining how we log and monitor. Whee!