The hidden flaws in other logging solutions
All of the log management solutions developed over the last five years, have the same major flaw. They all convert event logs into a normalized format without filtering out the redundant and convoluted data. This is a recipe for disaster which can lead to serious side effects from data pollution.
The old world concept of normalization is acceptable for firewall logs and other devices. In fact, UNIX server event logs can also still be managed in this format. None of these other log management solutions understand Window events at all. The collect all method is not acceptable for Windows event logs at all.
The reason this is true is because, the business landscape has changed, and become an Active Directory world. This has dramatically changed the playfield considerably for customers that need to meet compliance requirements and monitor event logs. The collect-all system that was once a smart format for all event logs has become an outdated process. Today, the majority of activity that is monitored takes place on Windows servers across the enterprise.
Unfortunately, the Windows auditing system generates an enormous amount of convoluted data. Windows events are extremely redundant, duplicated and complex. The Windows logs are inconsistent to work with and need special handling because event logs from Windows domain controllers, databases, and servers have become the majority of event logs that need to be collected and stored. Using collect-all solutions are a major oversight. Those methods don’t have the in depth knowledge required to overcome the Windows system issues. Those solutions also don’t address the event log structural flaws which exist either. Any administrator or security expert that has opened up Event Viewer, and attempted to search through the relevant event log data from Windows servers knows that Windows logs are extremely inconsistent in comparison to other log types.