Detect Web Application Attacks Using Web Server Access Logs

Recently, I was conducting a security audit for an organization. They had deployed a WAF (Web Application Firewall) for their critical web apps. However, when I asked them about the web server access logs, they said they were not aware of whether they had them. In fact, they told me that since a WAF was deployed with all sorts of rules, what is the need for web server logs from a security viewpoint? The WAF will block all malicious attempts, they said.

I was a bit taken aback at the lack of understanding of the security folks at the organization. Let me spend some time explaining the reason behind my conclusion above.

Why Web Server Access Logs are Important?

In any security scenario – even though we try to ensure that we do as best as we can to protect the systems — we need to consider the possibility that we could do better. We need to learn from day-to-day traffic, from ways by which hackers attack our system, and use that to improve our WAF rules.

Secondly, even the best security could be breached. And this could be due to various reasons including the discovery of zero-day vulnerabilities in the platforms used. And in case of a breach or a successful WAF evasion, the only way we would get information about the hack or the hacker would be through web server access logs.

What can we learn from web server logs?

To elaborate further, usually, before a hacker is successfully able to breach the website, he/she would probably have made a few unsuccessful attempts. These attempts if not blocked by WAF would be available as unusual entries in the web server logs. Also, in the normal operation of the web apps, regular users would be using certain URLs, making a certain type of requests, etc.  This normal behavior would result in certain log entries in the web server access logs. Security admins operating the website should be intimately familiar with normal web server logs corresponding to the normal use of their web apps. Thus, when unusual entries arise in the web server access logs, they represent anomalies.

Some of them could be attempts to hack. Thus, security admins should write scripts or use automated tools to analyze web server logs. These scripts would filter out the normal entries and only throw out unusual entries which can then be looked at by a human. The source IP addresses corresponding to these unusual entries can be watched or subsequently blocked, as also more signatures can be added into the WAF corresponding to these attack attempts by understanding what these hackers are trying to do.

Here are a few scenarios that could happen

1. A URL that contains the word admin could be an attempt to gain admin access or access using admin privileges.
2. An attempt that has a name of a CMS (content management system) platform that is not supported by the website – say Joomla where the website is not running on Joomla – reflects an attempt by a bot to figure out the type of platform used by the site. Normal users who use a browser wouldn’t be able to come up with such a URL in the normal course of their use.
3. A zero-day vulnerability found in the wild could result in an unusual URL. A WAF wouldn’t have a signature for such a vulnerability until the vulnerability becomes well-known.

To conclude, it is important to keep a watch on web server logs in addition to having the best of signatures in the WAF for a defense-in-depth strategy. This continuous process of monitoring and watching over logs all the time is best done by a Managed Services offering. Managed Services involves humans watching over logs all the time, filtering them using scripts or automated tools, and learning from the traffic to continuously improve the WAF rules.

In most cases, Managed Service provider 24x7x365 management and real-time monitoring for web application firewalls. This ensures that there is a dedicated support system in place for the entire WAF cycle providing maximum protection and minimizing risk exposures for all types of protected web applications.