Recently, I was conducting a security audit for an organization. They had deployed a WAF (Web Application Firewall) for their critical web apps. However, when I asked them about the web server access logs, they said they were not aware of whether they had them. In fact, they told me that since a WAF was deployed with all sorts of rules, what is the need for web server logs from a security viewpoint? The WAF will block all malicious attempts, they said.

I was a bit taken aback at the lack of understanding of the security folks at the organization. Let me spend some time explaining the reason behind my conclusion above.

Here is why web server access logs are important. In any security scenario – even though we try to ensure that we do as best as we can to protect the systems — we need to consider the possibility that we could do better. We need to learn from day-to-day traffic, from ways by which hackers attack our system, and use that to improve our WAF rules. Secondly, even the best of security could be breached. And this could be due to various reasons including discovery of zero day vulnerabilities in the platforms used. And in case of a breach or a successful WAF evasion, the only way we would get information about the hack or the hacker would be through web server access logs.

What then can we learn from web server logs? To elaborate further, usually, before a hacker is successfully able to breach the web site, he/she would probably have made a few unsuccessful attempts. These attempts if not blocked by WAF would be available as unusual entries in the web server logs. Also, in the normal operation of the web apps, regular users would be using certain urls, making certain type of requests etc.  This normal behaviour would result in certain log entries in the web server access logs. Security admins operating the website should be intimately familiar with normal web server logs corresponding to the normal use of their web apps. Thus, when unusual entries arise in the web server access logs, they represent anomalies. Some of them could be attempts to hack. Thus, security admins should write scripts or use automated tools to analyse web server logs. These scripts would filter out the normal entries and only throw out unusual entries which can then be looked at by a human. The source IP addresses corresponding to these unusual entries can be watched or subsequently blocked, as also more signatures can be added into the WAF corresponding to these attack attempts by understanding what these hackers are trying to do.

Here are a few scenarios that could happen.

  1. A url that contains the word admin could be an attempt to gain admin access or access using admin privileges.
  2. An attempt that has a name of a CMS (content management system) platform that is not supported by the website – say joomla where the website is not running on joomla – reflects an attempt by a bot to figure out the type of platform used by the site. Normal users who use a browser wouldn’t be able to come up with such a url in the normal course of their use.
  3. A zero day vulnerability found in the wild could result in an unusual url. A WAF wouldn’t have a signature for such a vulnerability till the vulnerability becomes well-known.

To conclude, it is important to keep a watch on web server logs in addition to having the best of signatures in the WAF for a defense in-depth strategy. This continuous process of monitoring and watching over logs all the time is best done by a Managed Services offering. Managed Services involves humans watching over logs all the time, filtering them using scripts or automated tools, and learning from the traffic to continuously improve the WAF rules.

In most cases, Managed Services provide 24x7x365 management and real-time monitoring for web application firewalls. This ensures that there is a dedicated support system in place for the entire WAF cycle providing maximum protection and minimizing risk exposures for all types of protected web applications.

Founder & Chief Marketing Officer, Indusface

Venky has played multiple roles within Indusface for the past 6 years. Prior to this, as the CTO @indusface, Venky built the product/service offering and technology team from scratch, and grew it from ideation to getting initial customers with a proven/validated business model poised for scale. Before joining Indusface, Venky had 10+ years of experience in security industry and had held various mgmt/leadership roles in Product Development, Professional Services and Sales @Entrust.