Tufin Expert Tip #2: Analyzing Network Connectivity Problems

Network connectivity problems are some of the most common – and aggravating – for business users. With distributed systems, as soon as an application does not behave as expected, the firewall is suspect. There are many other possible points of failure – the client application, the user’s PC, intermediate switches, routers, filters, load balancers and the application itself. But, because of its nature (secretive and designed to keep people out) the firewall is a prime suspect. As a firewall administrator, you are guilty until proven innocent.

So how can you quickly determine if the problem is due to the firewall or not?

One approach is to analyze the firewall traffic logs. Contact the user, obtain his IP address and ask him to access the application again. Ideally, this should trigger the connection in question. Then you can review the firewall traffic logs and locate the dropped or accepted packets. How easy this is depends on the tools – unless you have a smart log browser, you may have to work with syslogs.  Normally there will be a lot of logs so a filter on the source IP and, if possible, on the destination IP or port will make things easier.

Unfortunately, in many cases, you will find nothing. One possible reason is that the rule that allows or blocks the relevant traffic is not configured to generate logs. Another possibility is that you are not looking at the right firewall or are simply missing the relevant logs.

Another method is to analyze the firewall rule base. In many cases this is not feasible due to the size and complexity of the network and firewall policies. But if the network is relatively small and you know the rule base very well, you may be able to narrow the problem down to a specific rule or to a recent change that might have affected the application flow.

If you have Tufin SecureTrack, you can use the Policy Analysis tool to query the rule base. Get the user’s IP address, the IP address of the application and the service or port, if possible. Log into SecureTrack and create a policy analysis query with these inputs. You can run the query on all firewalls or, if you are sure which ones are relevant, on a subset. For the report, you can choose to show all traffic or only dropped or accepted traffic.  SecureTrack does not send any packets over the network. It analyzes its own copy of the rule base, which is always up to date from continuous monitoring. Since SecureTrack does not depend on traffic logs, it doesn’t matter whether log data is missing or unavailable.

Policy Analysis will quickly determine whether the firewalls are allowing the user’s traffic or not. If it turns out that the firewall is, in fact, blocking traffic, Policy Analysis will point you to the rule that’s causing the problem as well as when it was last changed, and by whom.

Tufin Firewall Expert Tip #1: Relocating a Server

An IT group needs to move a server and you need to update the firewall policy. The question is this: Which rules need to be changed? What is the fastest and safest way to get the job done?

You have two choices on where to start: the firewall rule base or the traffic logs. The rule base has one obvious advantage – it’s smaller. But it may be difficult to figure out which rules are relevant. Often the server will not be referenced explicitly, so you need to manually check every rule. This could be a tricky task, particularly if your rule base has been growing for a few years and has had multiple administrators. Rule shadowing complicates things even more.

The other alternative is reviewing firewall traffic logs. By identifying the traffic to and from the server over a reasonable period of time, you can create a new rule base. You do not need to worry about shadowing and potentially, this method will enable you to build a very accurate policy that does not include a lot of overly permissive rules. The downside is the large number of logs that must be analyzed. How far do you need to go back to make sure you’ve covered all legitimate use? One month? Three months? A year? What about your disaster recovery hot site traffic? Remember, you won’t have any logs for that unless you have tested your failover.

Whichever way you go, here are some things to look out for:

• Creating over-permissive rules. Though they provide a quick fix, hackers may use them as attack vectors later on.
• Access that is currently allowed and not really required.
• Implicit connectivity that seems superfluous but could actually be essential for business continuity.

If you have Tufin SecureTrack™, try using the Automatic Policy Generator (APG) to prepare for server relocation. Run APG on firewall logs from or to the server going back a year. It will generate a set of rules that allows the actual server access that took place during that period.

You should review it to make sure there are no rules that stem from malicious traffic, such as a port scan, (even if it runs slowly over several days), a conficker virus or a generic botnet.Change the server’s address to the new one and implement the new rules on relevant firewalls above any potentially blocking rules – you can consult with SecureTrack’s Policy Analysis tool or the Policy Change Advisor in SecureChange™ Workflow. After the relocation is done, you can use the rule and object usage report to clean up the obsolete access rules.

Automatic Policy Generation with Permissive Rule Analysis

One of my favourite activities, as CTO and founder, is meeting our users and talking to them about their needs and wishes in the areas of firewall policy management and beyond. It’s always nice to hear how SecureTrack is helping out and what our users like about it, it’s also useful to hear about things that don’t work and need improvement, but what I’m really after is requirements that lead to innovation.

A couple of years ago I was visiting one of our users, a mobile operator, and we were discussing a requirement they had to remove unused objects from the policy. At the time SecureTrack already provided rule usage analysis and this new requirement gave me the idea of mapping traffic logs onto objects within rules. This solution seemed right; unused objects would appear with zero hits and could then be safely deleted from the rule. The user also mentioned something about analyzing rules that are too wide which I kept in mind but didn’t have the bandwidth to deal with.

Anyway, we started brainstorming the object usage analysis requirement and came upon an interesting scenario:

Source	Destination	Service	Action
A B	X Y	HTTP	Accept

Assuming all objects are used, can this rule be improved?

If you like puzzles stop here and think.

Notes from the Blogosphere: PreachSecurity review of SecureTrack 4.5 review

Last week, Rafal Los posted an in-depth review of SecureTrack 4.5 on his PreachSecurity blog (aka Digital Soapbox).  As a former firewall admin, Raf has felt the pain our products were designed to get rid of.

What’s interesting about his review is that unlike the professional reviewers who wrote the semi-recent 5-star review of SecureTrack in SC Magazine, Raf’s familiarity with the issues that can really ruin a firewall admin’s day,  he is not an easy sell.

Not to say that SC Mag review wasn’t a huge achievement for us – getting a five star review from a leading security magazine in a MAJOR  high.   What’s so  appealing about Raf’s review (outside of his  high marks for usability, utility, and ‘worth your time’) is the street cred he brings to the table.

Raf  states: “I recently worked for a company that had 250+ firewalls in a single business unit, mostly Check Point and Cisco Pix… managed either out of one of a handful of Provider-1 consoles, or remotely via telnet (then SSH, as available). That was one of those nightmares I never thought I’d wake up from… so you can probably see why I’m so interested, at least personally, in tools that aim to simplify the life of the firewall admins out there.”

That sounds like a chunk of our customer base, for sure.

So, when he says things like  “Tufin were kind enough to give me a full demo and a demo image to play with and I have to say I’ve been genuinely impressed…”  and  “…where was this [SecureTrack] when I was jockeying firewalls back in ‘98/’99.?” and  especially ” This is a really cool product set with one of the most visible ROI models I have ever seen…”

Well… it’s hard not to feel good a about that.

But the ultimate compliment he gave us  (articulated in true firewall admin style) “… I think this is one of those products you shouldn’t go too long without… like socks or deodorant…”

Amen!

Click here to see this the full review, and while you’re at it, check out other posts on Raf’s blog, they are definitely worth a read.

John Pescatore on firewall complexity

Gartner’s John Pescatore wrote an interesting blog post today on firewall rule base complexity - “A Storm in Any Port” (firewall buffs will appreciate the pun – “Any Port”…)

We’ve been preaching about the complexity and inherent risks of large firewall rule bases for years, and it’s always great to receive validation from one of the most influential people in IT security.

Here are a couple of key quotes, in my opinion:

“… it is pretty rare to find an enterprise firewall policy that anyone is really sure about exactly what policy the rule set actually implements. Most firewall rule sets have mutated through incremental adds/drops/changes over the years and have turned into gargantuan linear lists that now have a life of their own.”

“Often you’ll find that easily 30% of the exceptions are no longer needed.”

The only thing missing from this great post is the solution – well, guess what:  we have it… Using Tufin SecureTrack, you can manage firewall policies proactively, and clean up unused firewall rules and objects. In some cases, we’ve seen customers with close to 50% of their firewall rule completely unused over a long period of time, out of hundreds of rules.

It’s time for some spring cleaning, folks…

Outsourcing Firewalls to MSSP

To anyone that’s been around the security industry for a while, the outsourcing of firewall management to Managed Security Service Providers (MSSP) is a fact life. The main players in this industry, estimated by Forrester at $3 billion in 2008, are well known names: AT&T, Verizon Business, EDS, Verisign (now SecureWorks), and several others.

Like many other IT outsourcing trends, the underlying reasons are cost savings and operational efficiency, with clear economic advantages for the MSSP vs. internal IT resources. The top MSSP’s focus on security operations management, retain highly-skilled personnel, maintain best practices across their customer base, and manage a 24×7 NOC / SOC. In addition, due to the sheer size of their infrastructure (hundreds to thousands of firewalls), MSSP’s receive major discounts from manufacturers for the underlying firewalls / routers / switches / servers / etc. All of these factors enable MSSP’s to dramatically reduce operational cost, and pass on the cost savings to the customer.

There are various challenges in working with MSSP’s, mostly related to giving up control over a complex and sensitive IT process, with a fine line drawn between the internal IT team and its interface with the MSSP. Organizations that have a corporate security policy and are governed by regulations mandating operational control, need to be able to retain some control over security configuration changes, and to be able to effectively manage the potential risk inherent in the outsourcing of security operations.

Here’s the first article in a series on the issues around outsourcing Firewalls to MSSP’s.

Welcome

Hi everyone,

Welcome to Tufin’s blog!

This is the corporate blog for Tufin Technologies, which leads the Firewall Operations Management market.

Several people are going to write, mostly about security management, Tufin, firewalls and other topics.

I hope you find it interesting, and keep coming back!

Take care,

Ruvi