What do you use for log analysis without ELK bloat?

My server possesses only 1GB RAM, and I must apologize for mentioning this limitation as if it were a failing of mine rather than the hardware.

Let us consider what I have tested:
- lnav: 45MB resident, acceptable
- angle-grinder: excellent for structured logs but requires some learning
- simple awk: timeless, uses negligible memory

I had attempted Elasticsearch many months ago and it consumed 600MB before ingesting a single document. Please forgive me if this observation seems obvious to others.

Has anyone constructed a lighter web viewer? My colleagues prefer browser access and I wish to accommodate them without shameful resource waste.

Your log analysis problem is actually a log shipping problem. For what it's worth, most people running 1GB boxes are also doing email wrong.

SPF fail rates in my testing correlate strongly with hosts that can't afford proper infrastructure. Not accusing, just observing. Your RAM constraint suggests you might also be on a budget IP with bad neighborhood effects.

Back to logs: rsyslog to remote host with proper filtering before storage. Parse at destination, not edge. This is how we handled high-volume mail logs pre-ELK and it still works.

For web viewer: write a 200-line Go http server that shells to awk. I've seen it done. Not pretty but functional.

Actually, (and this is obvious to some), the Go approach is (deceptively) simple. I wrote (well, adapted (from a gist)) a minimal viewer using (only) standard library.

Nested JSON logs (like (docker default)) need (preprocessing (with jq (or similar))) but plain syslog works (directly).

Memory footprint: (measured (via ps)) around 12MB. Serves (static (generated)) HTML tables.

I can share (the code (if anyone (actually) wants)).

I still run analog on one box, remember that? Generated beautiful reports. 2MB binary, zero dependencies. The young folks want real-time everything now but batch daily was fine for a decade.

Your 1GB constraint would have been luxury then. My first log host had 64MB and we felt spoiled.

Lnav is nice though, I'll give it that. Has that search highlight feature I wished for in 2009.

Caramba! 1GB only? Nossa, this is difficult now but not impossible kkkk

I use lnav too, very good, but my friend in São Paulo make something with awk and ncurses, very light, very fast. I don't remember where he put this, maybe github? I will ask him.

Double negation is not uncommon in my logs kkkk so awk with regex is necessary for me, not optional

The Go idea is nice, I can try this, I learn Go since last year, still beginner but maybe I can help

RAM prices too, not just v4.

Your 1GB box will cost more next year. /24 will cost you a kidney but so will DDR4 soon.

My advice: buy the 2GB upgrade now before they change tiers. Log analysis gets harder when you're also fighting OOM kills.

Market rate for 1GB→2GB at budget providers: $6→$9 monthly, trending up. Lock current price if you can.