Discussion
Loading...

Post

Log in
  • About
  • Code of conduct
  • Privacy
  • About Bonfire
Rachel
Rachel
@rachel@transitory.social  ·  activity timestamp 2 days ago

lmao.

talos linux just added a log folder in 1.12. Perfect, lemme enable node logs finally on the grafana/loki/alloy stack!

Oh. Wait, alloy/k8s-monitoring expects /var/log/journal to exist, so systemd, which talos is not using. (far too minimal for that, not a philosophical aversion to it)

Which means if I want to export node logs using alloy I'll have to do a bunch of file log configs, whoops

neocat_flop

Or I could export to syslog.... but I have mtls setup on that and I don't see a way to get a client cert for the talos nodes, or configure the log exporter to use it?

Hmm. I have another idea, but I'll have to think a bit on which is the best option, some of the ideas seem cursed....
#Homelab #Talos #Grafana

  • Copy link
  • Flag this post
  • Block
Rachel
Rachel
@rachel@transitory.social replied  ·  activity timestamp 2 days ago

Option 1: Configure file watches directly in k8s-monitoring helm chart, I should be able to do it similar to how I configured the syslog receiver. That would basically be embedding a manual alloy config iline with the chart.

Option 2. Forward talos logs to an unsecured syslog receiver. Bleh.

Option 3. use a daemonset with the host log directory mounted, and forward that out over secured rsyslog

Option 4. Poke the k8s-monitoring team to support non-systemd log directories for node logs

I actually think I like option 1 the most? It would run on the alloy daemonset that already has hostdir mounts (for pod logs)

  • Copy link
  • Flag this comment
  • Block
Mauricio Teixeira 🇺🇸🇧🇷
Mauricio Teixeira 🇺🇸🇧🇷
@badnetmask@hachyderm.io replied  ·  activity timestamp 2 days ago

@rachel
You remind me that I need to check that out.

But, reading everything you said, I would probably have chosen the daemonset path. Then you don't have to fumble with Alloy.

  • Copy link
  • Flag this comment
  • Block

BT Free Social

BT Free is a non-profit organization founded by @ozoned@btfree.social . It's goal is for digital privacy rights, advocacy and consulting. This goal will be attained by hosting open platforms to allow others to seamlessly join the Fediverse on moderated instances or by helping others join the Fediverse.

BT Free Social: About · Code of conduct · Privacy ·
Bonfire social · 1.0.1-beta.22 no JS en
Automatic federation enabled
Log in
  • Explore
  • About
  • Code of Conduct