Twists and Turns on the Road to the Best Software Firewall

I have several bits of info for the hoppers of those following along in my quest to find the best software firewall for Windows.

For those of you new to the saga, you’ll need to catch up with the rest of us by reading (or at least scanning) these previous articles:

Or, to get an up-to-date story that covers the bases of the three links above, including updated information, see this Computerworld story: Review Roundup: Slim Is in for Windows Desktop Firewalls (June 2007).

With that bit of housekeeping out of the way, on to the twists and turns.

Eset Smart Security Not So Stellar
Admittedly, I’m testing Beta 1b of Eset Smart Security, and rumor has it that Beta 2 is due out shortly. But I recently conducted a FirewallLeakTester.com-style leak test of Eset Smart Security, and the results weren’t good. For more information on the set of leak tests I used, please see my review of the free version of ZoneAlarm 7.0.337 in the last issue of the newsletter.

Eset Smart Security Beta 1b passed only two of the 17 off-the-shelf leak tests I ran on it — a very poor score. ZoneAlarm free, for example, scored five of 16 tests. Comodo 2.4, the best firewall according to the Matousec leak tests, passed 24 of 26 tests with its default settings; it passed them all after reconfiguration of the firewall.

I suspect that Eset is relying on its suite’s Nod32 anti-malware module to protect its customers from personal/financial information harvesting and Trojan malware. Indeed, in order to test the Eset firewall, I was forced to disable Nod32. There was no way to even copy the small leak test programs on the Windows desktop (or anywhere) on my test PC without Nod32 interrupting and automatically deleting those files. Eset is attacking the problem in another way. And it may, in fact, be the right way.

A couple of weeks back, I had a long talk with Symantec’s Tom Powledge, the product marketing manager in charge of Norton Internet Security, Norton 360, Norton AntiBot, Norton SystemWorks, and Norton AntiVirus. While he wasn’t directly referring to Eset’s product, he described functionality in the latest version of Norton Personal Firewall, Norton 360, and Norton Internet Security that also goes about protecting your computer from data-harvesting malware that requires very little input from users and is not dependent on the firewall. Both companies are heavily employing heuristics-based techniques for identifying and rapidly stopping the execution of malware products on your computer.

Powledge believes, in fact, that outbound leak testing is fairly useless. He believes that many firewall software makers game the system by adding code for the specific tests. The thinking goes — and I don’t disagree with it — that the firewall is not the right tool for blocking this type of threat. This is why Norton is now offering Norton AntiBot, and its suite products have several ID theft measures. I have pledged to myself to test both Norton Personal Firewall 2008 (when it comes out this fall) and Norton 360 (again). Norton 360 doesn’t meet the requirements I’ve set for either Best Antivirus or Best Firewall products. But it’s Symantec’s attempt to reduce the system footprint of its security suite. I looked at it in beta only, so now I’ll look at the shipping product.

There can be no doubt that antivirus and anti-malware technologies have merged. There’s becoming less and less need to run separate signature-based file-scanning engines for viruses and spyware. That’s especially the case if the security products are actively employing behavioral-based techniques for finding and eradicating all types of malware.

Bottom line: I’m testing firewalls at a time when it appears that the need for outbound protection has never been stronger, but also, when the thinking about how to add that layer of protection is changing, perhaps profoundly.

On the other hand, if you’re going to have a software firewall running on your system, wouldn’t you rather have one that stopped as many illicit outbound connections as possible? Matousec’s test methodology is hyperaware of firewalls that may be attempting to game the system. The security agency runs a test called FPR (Fake Protection Revealer) that attempts to ferret out custom coding to specific leak tests. They have publicly named names of companies whose products appear to be doing that based on their test data.

Check out this info Matousec provides about issues with specific firewall products.

In the end, security is about layers of protection. I’ll admit that my money has long been on heuristic (behavioral) based techniques employed in combination with signature-based identification of malware as the guts of the best security products of the future. But heuristics technology still has a way to go before it can cover all bases. And the threat keeps morphing. In the meantime, and possibly for the long run, I want the best firewall I can get.

The Plot Thickens Around Comodo
Apparently, not everyone is having as great an experience with Comodo 2.4 as I am. I’m running it on three test machines, including on the Parallels-based Windows XP that runs on my everyday Mac. I’m having no problems at all. It’s working like I want it to, and I see pop-ups very infrequently. And when they do appear, they make sense.

Since the 2.4 release, though, I have received a handful of messages from Scot’s Newsletter readers describing problems with Comodo that caused them to remove it from their computers. Bruce Marien was one of the readers who wrote in. Here’s how Bruce described the problem on his PC:

“My problems arose after only a few days of use. I noticed that Comodo didn’t seem to remember responses I clicked in the pop-up windows (something you mentioned having been problematic in an earlier version of Comodo). Then I started losing Internet connectivity. My cable modem company’s diagnostic tool flagged my system as having changed from dynamic to static access. Running the diagnostic tool’s repair function did correct the issue temporarily, but it always came back. Other times the diagnosis was corruption in my TCP stack and it was unable to effect a repair. At that point, the only fix was to reboot the cable modem and my computer. This got old fast and I uninstalled Comodo.”

Bruce is not alone in having difficulty with Comodo. Lockergnome’s Ron Schenone blogged about similar problems with Comodo last December.

Other people are reporting issues with pop-ups. I had the same problems with an earlier version of Comodo, but since the release of Comodo version 2.4, those woes have been purely a thing of the past for me. SFNL reader Ernie Marshburn is having this problem, and this is how he describes it:

“Comodo’s protection level is fine but I am constantly pinged with pop-up messages about authorizing applications, mostly Outlook. More annoyingly, many of the messages have multiple screens, which I guess must be checked individually. If this were only the first instance when I was being asked, it wouldn’t bother me. But the exact same messages reappear frequently and, it seems, at random — without any apparent relationship to what’s actually happening on the computer.”

In a later message, Ernie specifically mentions multiple repeat Comodo pop-ups related to Outlook, IE, Acrobat, ccApp (a Norton AntiVirus subroutine), and Microsoft Word.

I have to agree with Ernie that the way Comodo gangs up multiple pop-ups in a single window that you step through like a wizard is less convenient than it might be and also might be missed by some people. While it does cut down on the apparent number of pop-ups, you still have to step through each separate message and click the checkbox so that the program will “remember” your answer. Is it possible some people don’t realize that they have to do that? I suppose so; on the other hand, Ernie got that.

More likely, however, is another explanation. There’s a setting in Comodo’s Security > Advanced > Miscellaneous > Configure area that controls the level of pop-ups Comodo displays. By default, that setting is “low” in Comodo 2.4. It’s at least possible that some people are seeing a blizzard of pop-ups because they either changed this setting to “high” or upgraded a pre-existing installation of Comodo that had a higher setting.

Just as this issue was getting ready to mail, Ernie found that the pop-ups level setting in his Comodo installation was set to “high.” Setting it to low helped considerably, although he’s still seeing more pop-ups than I am.

While writing this article, I installed Comodo on a fourth machine. And, again, by default the pop-ups are minimal. Some people are having issues, but many others are not. I’m interested in your firsthand experiences with the 2.4 version of Comodo. Please send me your thoughts in an email message.

It would help greatly if you could list for me the applications that the pop-ups are related to, as Ernie did.

Comodo 3.0 Is Close
I got an email from a Comodo marketing VP letting me know that Comodo 3 is about six weeks away from release. I don’t have much detail on the product, but some of the product features are listed on this Comodo Forums post.

The most notable changes are Windows Vista support (both 32 bit and 64 bit) and a host-intrusion-prevention system (HIPS) module — both of which should be welcome additions.

Reminder: This evaluation focuses on software firewalls for Windows XP SP2. More and more software firewalls are being updated to support Vista, but at the time that I started this work, not enough of them supported Vista to make that a useful endeavor.

Status of this Test
I continue to favor Comodo 2.4 as the likely winner of this evaluation. I have ruled out all of the other contenders. No other product has a similar compromise of excellent protection and decent ease of use. But I’d like to give it another few weeks to hear from people who may be having problems. Given that a new version of the product is coming out, I may also wait to at least test a beta or the final version.

During this interim period, if you’re making a firewall selection, my recommendation would be to select Comodo 2.4. It’s free, so if you don’t like it, you can back out of it.

Leave a Reply

You must be logged in to post a comment.