Weblogs for simonw

Posted by simonw on Mon 12 May 2014 at 21:32
Tags: none.
Been trying out various web application vulnerabilities scanners, both Open Source and Proprietary.

These are tools that will analyse your website, or in some case an instrumented copy of your site, and identify some types of common security flaws, or in other cases simple omissions to use best practice.

My main goal is to find tools which are easy to integrate into a Continuous Integration process, so ideally looking for scanners with minimal user interaction, with a command line driven batch mode, to beef up the current CI process.

I've not reached any conclusions yet, but it is an interesting landscape. Almost everyone agrees more tools are better, because they are all slightly different, there are inevitably bugs that one finds, that another misses. CPU time is a LOT cheaper than developer, or Pen Tester time, so a lot of automation makes sense, however the significant false positive rate means that more tools do make some more work. This scales better than I expected, because the same issues trip up different tools in the same way.

For example the WordPress action "wp-post-comments.php" returns a "500 Server Error" HTTP response code under some circumstances, and lots of tools leap on this proudly noting they have "crashed" your server, when the response header shows otherwise.

But alas WordPress defaults to 500 status codes when people omit required fields on forms, and other common cases. This WordPress annoyance has been lurking for at least 4 years, and the Lead Developers seem in no hurry to stop it emitting error codes suggesting it has crashed just because someone forgot to enter their email address.

https://core.trac.wordpress.org/ticket/10551

Almost all the tools pick up on this, and arguably rightly so, but it isn't a useful find.


Intercepting Proxies

Intercepting Proxies sit between the browser, and website to test, and thus can easily identify (and modify if needed) any traffic from browser to server. For AJAX, web sockets, and a number of other web technologies this is an essential place to be for spotting certain common vulnerabilities (like failure to validate AJAX server side).

One of our tools of choice is BurpSuite, which is a fantastic little application. You do need to buy the Pro version to do proper automated scanning. Whilst we'll use it for manual, and pre-release testing, its focus is on manual testing, and whilst some folk have tried to drive it from the command line, I suspect down this route lies pain and eternal maintenance.

The OWASP Zed Attack proxy tool is targeted at almost exactly the same space as BurpSuite (Clone?). In true Open Source style it is harder to use, a little rougher around the edges, but allegedly does more, and seems to have a little more momentum and adaptability. Oh and a very responsive lead funded by Mozilla. So far it takes longer to run, and has more false positives, but in my initial tests identify a whole host of minor but legitimate issues, although I rushed it, it still took nearly 4 hours for a small website (CPU time may be cheap but we do want answers before the release date). I suspect many folk who've never used BurpSuite Pro will think ZAP fantastic, and it does look like a little attention to settings will pay dividends in both time to run, and false positive rate.

I suspect either tools can be used in Continuous Integration as a proxy, but I see more effort to support this from the OWASP ZAP lead.


Vulnerability Scanners

In a previous roles where I was working on PHP and Perl websites, predominantly doing simple forms, the tool of choice was Wapiti. My reading around suggests it is still an excellent choice, scoring well in comparisons and being quick and easy to use. The default version of Wapiti in Debian is 1 until Jessie, so you probably want either a dedicated network security distro like Kali in a machine somewhere, or just Debian testing.

I note also that OSSIM vulnerability scanner (OpenVAS) will run wapiti if it is installed, but again because it is based on Squeeze it is wapiti version 1 you get if you use "apt-get". Still if you need a network security scanner installing OSSIM is a lot easier than installing OpenVAS in Debian, sticking in wapiti is a no-brainer as a one line enhancement. OSSIM can run Nikto with a really minor tweak I've discussed on their forum as well.

Nikto still finds a place in my arsenal. Somehow I can't love w3af. Few of the other tools make me love them (except NMAP of course).

Next on my list are some of the more established proprietary tools, but there are a huge number of Open Source tools I simply won't have time to evaluate, so any tips on where to check first appreciated. I don't see an alternative to the Intercepting proxy, or something logically equivalent, there is also a requirement to handle newer features of browsers like Web Sockets and local storage (which I see as a greatly enriched version of cookie poisoning).

As always these tools won't keep you safe, but they may let you know when you are heading in the wrong direction, which is why using them as early in the development process as possible makes sense.

 

Posted by simonw on Thu 10 Oct 2013 at 11:21
Tags: none.
The latest Vodafone Sure Signal 3 box or Alcatel-Lucent 9361 Home Cell p3.0 as it says on the box did something a little odd, which I found a load of non-answers for.

When you complete registration, plug it in and the power light stays on, but the Internet light is flashing slowly, most likely explanation is it is downloading an update from the Internet (is the orange light on the ethernet port flashing showing traffic). No idea how big the download it has to download is, but it took a significant time on a 16Mbps download line (more than 20 minutes).

So basically leave it an hour before trying to figure it out, because it is less frustrating than trying to find good documentation for the right version from Vodafone. No really just leave it, if you must fiddle check it has picked up an IP address from your DHCP server or something non-disruptive of it doing a download.

 

Posted by simonw on Fri 14 Jun 2013 at 01:35
Tags: none.
Who knows if Matt Cutts and the folks at Google have alerts set for bugs and such like.

I am changing a lot of stuff currently because of a change of job.

One thing I have (a bad idea it seems) is some websites registered in Webmaster tools to my personal Google Account. I want to transfer ownership to a new account (done), and remove myself - at which point it says I should remove the verification codes (which it claims are used to allow me to delegate to the other admins). I can see why complications might arise with delegating a new owner and deleting myself, but ultimately if they let the people who control the website currently reset stuff if it goes wrong (and they do), it seems a harmless enough approach. Maybe I'm missing something, like the code is static, and so presumably would be the same if I tried to re-verify myself later, and they use the same inteface for everyone, or some such.... messy.

Google Business Listing: seems having verified a business as my own, the way to delete it is to ask Google nicely, who then send me an automatic email saying they'll get to my feedback eventually. Not that anyone can use the listing as website and email have gone, but it still seems odds having a manual process.

Now to address the 130 odd Webmaster tools messages - well spotted Googlebots.

 

Posted by simonw on Wed 1 May 2013 at 17:51
Tags: none.
Needed more disk space, snapshot, added a virtual disk copied /home across, mount, test, remove old /home.

Realized too far into the process it would have been much quicker to clone the disk, and then delete stuff that wasn't /home from the clone.

Deep in my heart that feels wrong to have a disk and its clone living side by side forever more.

Of course in a saner world the virtualization software would just let me extend the disk I was using, and the file system is happy to be extended on the fly, so I wouldn't have needed a reboot. Maybe one day soon, I've already had virtualization add memory of the fly (KVM).

 

Posted by simonw on Tue 30 Apr 2013 at 11:34
Tags: none.
Email to mailing list noted change of status

"The njabl.org DNSBL is in the process of shutting down. On March 1, 2013, the various njabl.org DNSBL zones were all emptied. Any systems configured to use any of the NJABL DNSBL zones should be reconfigured immediately to no longer use the NJABL DNSBL zones.

Today, April 29, 2013, NS for the NJABL DNSBL zones is being pointed into 192.0.2.0/24 (TEST-NET-1) which is unrouted IP space. This will likely cause any systems using the NJABL DNSBL zones to experience long delays in DNS resolution of NJABL DNSBL lookups. This is being done both to sink the DNS query traffic and to hopefully be noticed by the owners/managers of those systems."

And indeed name servers are indeed:

192.0.2.11
192.0.2.12
192.0.2.13
192.0.2.14
192.0.2.15

The Squeeze policyd-weight package uses it by default (fixed in Wheezy), but its absence doesn't seem to cause any particular issues with the daemon. Still you probably want to remove it.

Should already be long gone from Spam Assassin rules (life is easier if you keep Spam Assassin up to date!), but I don't use Spam Assassin.

 

Posted by simonw on Wed 24 Apr 2013 at 17:21
Tags: ,
Merely to stop the question being asked again and again.

NYTPROF claims that we have a bottleneck on 's/^\s*|\s*$//g' (I'll believe it when I see it run faster), google knows what to do, on our hardware the two line version below is ~50 times faster.

$x =~ s/^\s+//;
$x =~ s/\s+$//;

String::Util::trim() is only ~35 times faster than the single regular expression but has the advantage of readability, those who prefer code not to look like line noise will find comfort here.

http://stackoverflow.com/questions/184590/is-there-a-perl-compatible-regular-expression-to-trim-whitespace-from-both-sides

TIMTOWTDI

 

Posted by simonw on Wed 24 Apr 2013 at 15:43
Tags: ,
Exasperated at performance of some automated browser rendering we do, 6 seconds elapse, 0.7 seconds of CPU usages, I finally used strace and found it is calling sleep in the middle of my script.

This led me to the manual page for xvfb-run, which indeed notes it deliberately inserts a three second wait before it runs the command. Once upon a time maybe....

"xvfb-run --wait=0" reduces my run time for screenshot from 6 seconds elapse to 3 seconds elapse. So I still have 2.3 seconds of "idle" time. Some, possibly all, of this is due to slow response from the webserver. Still the first half of my elapse time gave itself up easily.

 

Posted by simonw on Sun 21 Apr 2013 at 01:05
Tags: none.
Previously bought from Vodafone and unlocked a Huawei Ascend G300.

Knew ICS was available, discovered that Huawei have a general purpose ICS version independent of Vodafone, so possibly a chance to lose all the Vodafone breakage (sorry features).

Phone also has had a couple of issues with stability resulting in numerous files in LOST.DIR on both SD cards, so a reinstall was overdue.

[U8815][SoftWare]HUAWEI Ascend G300 firmware(U8815,Android 4.0,V100R001C00B952,General Version) 2012-12-17

Backed up settings via Huawei Allbackup.
Backed up Apps via Huawei Allbackup.
Copied all photos to local disk (and Flickr).
Download and unzip firmware on Debian desktop, copy the dload folder to the root of the additional SD card.
Did the install from SD card under Storage in Gingerbread.

All went well, till I tried to restore "Huawei Allbackup" from its APK file, when it failed to install "Application not installed" (helpful not).

The "AppInstaller" found all the APK files from my old apps that Allbackup had saved, and made short work of reinstalling those I want back, but all the settings are lost till I figure out how to get "Allbackup" working with ICS.

Given I very purposely don't keep anything crucial on the phone, everything is synced (including the photos so I now have 4 copies of them), it is a minor inconvenience to lose the settings (I'll have to cut and paste some account passwords, and lose state in a few games). I'd decided before hand that if this happened I wouldn't back out the change.

The other Huawei app that failed to install from its APK is Huawei FM Radio, which is a minor inconvenience, as I rarely use it, but a bit annoying and I will kick Huawei support as seems odd they left it out of the install in the first place.

Now I just need to master ICS, I just hope it handles unclean shutdown of filesystems better than Gingerbread. Otherwise they'll be another reinstall all too soon.

 

Posted by simonw on Tue 2 Apr 2013 at 15:42
Tags: none.
An hour of my life deciphering why one Windows 2003 server didn't do daylight saving gracefully.

Didn't work as documented - didn't work as per our notes - didn't work as per the GUI, the command line commands didn't work as expected, but the third knowledge base article which said use the registry editor you point and click monkey, and set the various registry settings by hand (KB 816042 but not all of them as you may not want to become as time server) seemed to do the trick. I'll tell you if it really worked in the autumn.

Makes me crave the insanity of "cp ntp.conf /etc/ ; service ntp restart", although a lot of the time these days I go with the default ntp.conf depending where the server is hosted. Although even then I had to tweak the kernel boot parameter for XEN hardware based virtualization servers to enable and use jiffies before time was stable in multiprocessor Squeeze servers.

Oh symptom was that W2K3 box lost one hour when it did its time sync after the daylight saving change, would be correct on reboot, and lose an hour on resyncing later. I presume some insanity caused by the mysteries of XEN. Although only noted on one server, so probably something I didn't do when it was set up.

Some sort of race to the bottom amongst visualization providers, how insanely complicated can we make getting the correct time before they go back to a real server.

 

Posted by simonw on Thu 7 Mar 2013 at 00:29
Tags: none.
Reported issue of not being able to access a file in Drupal6 was due to the .htaccess file preventing access to files of that name (in this case a file name prefix sometimes associated with Subversion).

Easily fixed, but got me wondering how it could have been avoided.

There are several issues. Protecting me from subversion when it isn't in use is rather keen, but I don't mind a little mollycoddling.

However the underlying issue is, I think, treating uploaded files like files which are part of Drupal. Of course subversion could be being used to revision the uploaded files whereever they exist in the file system and it might be a bad idea to serve the revision controlled files associated with them.

Various mechanisms could be used to treat the files as a distinct type of thing from the Drupal application files, but probably for most people storing the files in the database would be perfectly fine, then they would receive similar protection, back-up, (replication?) and handling as other user content in Drupal. Of course someone somewhere will be distributing DVD images using Drupal and think this suggestion nuts.

Being Drupal there is already a module for doing this (dbfm), you just needed to know you wanted it that way first. I'm less clear how Drupal 7 handles this (Storage API?).

Microsoft also have some relevant comments on storing files in databases which make similar points, that treating them like other data may result in greater simplicity which may be more important than other concerns.

http://research.microsoft.com/apps/pubs/default.aspx?id=64525