Weblogs for simonw
#487
Posted by simonw on Wed 1 May 2013 at 17:51
Needed more disk space, snapshot, added a virtual disk copied /home across, mount, test, remove old /home.
Realized too far into the process it would have been much quicker to clone the disk, and then delete stuff that wasn't /home from the clone.
Deep in my heart that feels wrong to have a disk and its clone living side by side forever more.
Of course in a saner world the virtualization software would just let me extend the disk I was using, and the file system is happy to be extended on the fly, so I wouldn't have needed a reboot. Maybe one day soon, I've already had virtualization add memory of the fly (KVM).
Realized too far into the process it would have been much quicker to clone the disk, and then delete stuff that wasn't /home from the clone.
Deep in my heart that feels wrong to have a disk and its clone living side by side forever more.
Of course in a saner world the virtualization software would just let me extend the disk I was using, and the file system is happy to be extended on the fly, so I wouldn't have needed a reboot. Maybe one day soon, I've already had virtualization add memory of the fly (KVM).
[0 Comments
| Add Comment
|
]
]
#486
Posted by simonw on Tue 30 Apr 2013 at 11:34
Email to mailing list noted change of status
"The njabl.org DNSBL is in the process of shutting down. On March 1, 2013, the various njabl.org DNSBL zones were all emptied. Any systems configured to use any of the NJABL DNSBL zones should be reconfigured immediately to no longer use the NJABL DNSBL zones.
Today, April 29, 2013, NS for the NJABL DNSBL zones is being pointed into 192.0.2.0/24 (TEST-NET-1) which is unrouted IP space. This will likely cause any systems using the NJABL DNSBL zones to experience long delays in DNS resolution of NJABL DNSBL lookups. This is being done both to sink the DNS query traffic and to hopefully be noticed by the owners/managers of those systems."
And indeed name servers are indeed:
192.0.2.11
192.0.2.12
192.0.2.13
192.0.2.14
192.0.2.15
The Squeeze policyd-weight package uses it by default (fixed in Wheezy), but its absence doesn't seem to cause any particular issues with the daemon. Still you probably want to remove it.
Should already be long gone from Spam Assassin rules (life is easier if you keep Spam Assassin up to date!), but I don't use Spam Assassin.
"The njabl.org DNSBL is in the process of shutting down. On March 1, 2013, the various njabl.org DNSBL zones were all emptied. Any systems configured to use any of the NJABL DNSBL zones should be reconfigured immediately to no longer use the NJABL DNSBL zones.
Today, April 29, 2013, NS for the NJABL DNSBL zones is being pointed into 192.0.2.0/24 (TEST-NET-1) which is unrouted IP space. This will likely cause any systems using the NJABL DNSBL zones to experience long delays in DNS resolution of NJABL DNSBL lookups. This is being done both to sink the DNS query traffic and to hopefully be noticed by the owners/managers of those systems."
And indeed name servers are indeed:
192.0.2.11
192.0.2.12
192.0.2.13
192.0.2.14
192.0.2.15
The Squeeze policyd-weight package uses it by default (fixed in Wheezy), but its absence doesn't seem to cause any particular issues with the daemon. Still you probably want to remove it.
Should already be long gone from Spam Assassin rules (life is easier if you keep Spam Assassin up to date!), but I don't use Spam Assassin.
[0 Comments
| Add Comment
|
]
]
#485
Posted by simonw on Wed 24 Apr 2013 at 17:21
Merely to stop the question being asked again and again.
NYTPROF claims that we have a bottleneck on 's/^\s*|\s*$//g' (I'll believe it when I see it run faster), google knows what to do, on our hardware the two line version below is ~50 times faster.
$x =~ s/^\s+//;
$x =~ s/\s+$//;
String::Util::trim() is only ~35 times faster than the single regular expression but has the advantage of readability, those who prefer code not to look like line noise will find comfort here.
http://stackoverflow.com/questions/184590/is-there-a-perl-compatible-regular-expression-to-trim-whitespace-from-both-sides
TIMTOWTDI
NYTPROF claims that we have a bottleneck on 's/^\s*|\s*$//g' (I'll believe it when I see it run faster), google knows what to do, on our hardware the two line version below is ~50 times faster.
$x =~ s/^\s+//;
$x =~ s/\s+$//;
String::Util::trim() is only ~35 times faster than the single regular expression but has the advantage of readability, those who prefer code not to look like line noise will find comfort here.
http://stackoverflow.com/questions/184590/is-there-a-perl-compatible-regular-expression-to-trim-whitespace-from-both-sides
TIMTOWTDI
#484
Posted by simonw on Wed 24 Apr 2013 at 15:43
Exasperated at performance of some automated browser rendering we do, 6 seconds elapse, 0.7 seconds of CPU usages, I finally used strace and found it is calling sleep in the middle of my script.
This led me to the manual page for xvfb-run, which indeed notes it deliberately inserts a three second wait before it runs the command. Once upon a time maybe....
"xvfb-run --wait=0" reduces my run time for screenshot from 6 seconds elapse to 3 seconds elapse. So I still have 2.3 seconds of "idle" time. Some, possibly all, of this is due to slow response from the webserver. Still the first half of my elapse time gave itself up easily.
This led me to the manual page for xvfb-run, which indeed notes it deliberately inserts a three second wait before it runs the command. Once upon a time maybe....
"xvfb-run --wait=0" reduces my run time for screenshot from 6 seconds elapse to 3 seconds elapse. So I still have 2.3 seconds of "idle" time. Some, possibly all, of this is due to slow response from the webserver. Still the first half of my elapse time gave itself up easily.
[0 Comments
| Add Comment
|
]
]
#483
Posted by simonw on Sun 21 Apr 2013 at 01:05
Previously bought from Vodafone and unlocked a Huawei Ascend G300.
Knew ICS was available, discovered that Huawei have a general purpose ICS version independent of Vodafone, so possibly a chance to lose all the Vodafone breakage (sorry features).
Phone also has had a couple of issues with stability resulting in numerous files in LOST.DIR on both SD cards, so a reinstall was overdue.
[U8815][SoftWare]HUAWEI Ascend G300 firmware(U8815,Android 4.0,V100R001C00B952,General Version) 2012-12-17
Backed up settings via Huawei Allbackup.
Backed up Apps via Huawei Allbackup.
Copied all photos to local disk (and Flickr).
Download and unzip firmware on Debian desktop, copy the dload folder to the root of the additional SD card.
Did the install from SD card under Storage in Gingerbread.
All went well, till I tried to restore "Huawei Allbackup" from its APK file, when it failed to install "Application not installed" (helpful not).
The "AppInstaller" found all the APK files from my old apps that Allbackup had saved, and made short work of reinstalling those I want back, but all the settings are lost till I figure out how to get "Allbackup" working with ICS.
Given I very purposely don't keep anything crucial on the phone, everything is synced (including the photos so I now have 4 copies of them), it is a minor inconvenience to lose the settings (I'll have to cut and paste some account passwords, and lose state in a few games). I'd decided before hand that if this happened I wouldn't back out the change.
The other Huawei app that failed to install from its APK is Huawei FM Radio, which is a minor inconvenience, as I rarely use it, but a bit annoying and I will kick Huawei support as seems odd they left it out of the install in the first place.
Now I just need to master ICS, I just hope it handles unclean shutdown of filesystems better than Gingerbread. Otherwise they'll be another reinstall all too soon.
Knew ICS was available, discovered that Huawei have a general purpose ICS version independent of Vodafone, so possibly a chance to lose all the Vodafone breakage (sorry features).
Phone also has had a couple of issues with stability resulting in numerous files in LOST.DIR on both SD cards, so a reinstall was overdue.
[U8815][SoftWare]HUAWEI Ascend G300 firmware(U8815,Android 4.0,V100R001C00B952,General Version) 2012-12-17
Backed up settings via Huawei Allbackup.
Backed up Apps via Huawei Allbackup.
Copied all photos to local disk (and Flickr).
Download and unzip firmware on Debian desktop, copy the dload folder to the root of the additional SD card.
Did the install from SD card under Storage in Gingerbread.
All went well, till I tried to restore "Huawei Allbackup" from its APK file, when it failed to install "Application not installed" (helpful not).
The "AppInstaller" found all the APK files from my old apps that Allbackup had saved, and made short work of reinstalling those I want back, but all the settings are lost till I figure out how to get "Allbackup" working with ICS.
Given I very purposely don't keep anything crucial on the phone, everything is synced (including the photos so I now have 4 copies of them), it is a minor inconvenience to lose the settings (I'll have to cut and paste some account passwords, and lose state in a few games). I'd decided before hand that if this happened I wouldn't back out the change.
The other Huawei app that failed to install from its APK is Huawei FM Radio, which is a minor inconvenience, as I rarely use it, but a bit annoying and I will kick Huawei support as seems odd they left it out of the install in the first place.
Now I just need to master ICS, I just hope it handles unclean shutdown of filesystems better than Gingerbread. Otherwise they'll be another reinstall all too soon.
[0 Comments
| Add Comment
|
]
]
#482
Posted by simonw on Tue 2 Apr 2013 at 15:42
An hour of my life deciphering why one Windows 2003 server didn't do daylight saving gracefully.
Didn't work as documented - didn't work as per our notes - didn't work as per the GUI, the command line commands didn't work as expected, but the third knowledge base article which said use the registry editor you point and click monkey, and set the various registry settings by hand (KB 816042 but not all of them as you may not want to become as time server) seemed to do the trick. I'll tell you if it really worked in the autumn.
Makes me crave the insanity of "cp ntp.conf /etc/ ; service ntp restart", although a lot of the time these days I go with the default ntp.conf depending where the server is hosted. Although even then I had to tweak the kernel boot parameter for XEN hardware based virtualization servers to enable and use jiffies before time was stable in multiprocessor Squeeze servers.
Oh symptom was that W2K3 box lost one hour when it did its time sync after the daylight saving change, would be correct on reboot, and lose an hour on resyncing later. I presume some insanity caused by the mysteries of XEN. Although only noted on one server, so probably something I didn't do when it was set up.
Some sort of race to the bottom amongst visualization providers, how insanely complicated can we make getting the correct time before they go back to a real server.
Didn't work as documented - didn't work as per our notes - didn't work as per the GUI, the command line commands didn't work as expected, but the third knowledge base article which said use the registry editor you point and click monkey, and set the various registry settings by hand (KB 816042 but not all of them as you may not want to become as time server) seemed to do the trick. I'll tell you if it really worked in the autumn.
Makes me crave the insanity of "cp ntp.conf /etc/ ; service ntp restart", although a lot of the time these days I go with the default ntp.conf depending where the server is hosted. Although even then I had to tweak the kernel boot parameter for XEN hardware based virtualization servers to enable and use jiffies before time was stable in multiprocessor Squeeze servers.
Oh symptom was that W2K3 box lost one hour when it did its time sync after the daylight saving change, would be correct on reboot, and lose an hour on resyncing later. I presume some insanity caused by the mysteries of XEN. Although only noted on one server, so probably something I didn't do when it was set up.
Some sort of race to the bottom amongst visualization providers, how insanely complicated can we make getting the correct time before they go back to a real server.
[0 Comments
| Add Comment
|
]
]
#481
Posted by simonw on Thu 7 Mar 2013 at 00:29
Reported issue of not being able to access a file in Drupal6 was due to the .htaccess file preventing access to files of that name (in this case a file name prefix sometimes associated with Subversion).
Easily fixed, but got me wondering how it could have been avoided.
There are several issues. Protecting me from subversion when it isn't in use is rather keen, but I don't mind a little mollycoddling.
However the underlying issue is, I think, treating uploaded files like files which are part of Drupal. Of course subversion could be being used to revision the uploaded files whereever they exist in the file system and it might be a bad idea to serve the revision controlled files associated with them.
Various mechanisms could be used to treat the files as a distinct type of thing from the Drupal application files, but probably for most people storing the files in the database would be perfectly fine, then they would receive similar protection, back-up, (replication?) and handling as other user content in Drupal. Of course someone somewhere will be distributing DVD images using Drupal and think this suggestion nuts.
Being Drupal there is already a module for doing this (dbfm), you just needed to know you wanted it that way first. I'm less clear how Drupal 7 handles this (Storage API?).
Microsoft also have some relevant comments on storing files in databases which make similar points, that treating them like other data may result in greater simplicity which may be more important than other concerns.
http://research.microsoft.com/apps/pubs/default.aspx?id=64525
Easily fixed, but got me wondering how it could have been avoided.
There are several issues. Protecting me from subversion when it isn't in use is rather keen, but I don't mind a little mollycoddling.
However the underlying issue is, I think, treating uploaded files like files which are part of Drupal. Of course subversion could be being used to revision the uploaded files whereever they exist in the file system and it might be a bad idea to serve the revision controlled files associated with them.
Various mechanisms could be used to treat the files as a distinct type of thing from the Drupal application files, but probably for most people storing the files in the database would be perfectly fine, then they would receive similar protection, back-up, (replication?) and handling as other user content in Drupal. Of course someone somewhere will be distributing DVD images using Drupal and think this suggestion nuts.
Being Drupal there is already a module for doing this (dbfm), you just needed to know you wanted it that way first. I'm less clear how Drupal 7 handles this (Storage API?).
Microsoft also have some relevant comments on storing files in databases which make similar points, that treating them like other data may result in greater simplicity which may be more important than other concerns.
http://research.microsoft.com/apps/pubs/default.aspx?id=64525
[0 Comments
| Add Comment
|
]
]
#480
Posted by simonw on Wed 6 Mar 2013 at 15:27
Wrestled OpenCart in anger today for the first time.
Linkchecked a site and our server died under the load, added "--pause=1" to linkchecker - same result.
On inspection of mysql-slow-log it was taking a long time to count the products in a given category (0.43s).
Various people offer the fix of switching off the category counts when I searched, but I figured something was wrong at the database level since a simple count like that should be pretty much instantaneous in a modern database with only ~40,000 records (and three products in the category I was testing). Some sniffing around got me to:
https://github.com/opencart/opencart/issues/177.
So:
create index iproduct_description_language_id on product_description ( language_id );
create index iproduct_to_category_category_id on product_to_category ( category_id);
Now the problem query has gone from 0.43s to 0.00s - possibly the default precision of the MySQL timer will need addressing as Moore's law continues.
This was day one with OpenCart, this issue doesn't full me with confidence, or did the guy who installed it miss something?
Linkchecked a site and our server died under the load, added "--pause=1" to linkchecker - same result.
On inspection of mysql-slow-log it was taking a long time to count the products in a given category (0.43s).
Various people offer the fix of switching off the category counts when I searched, but I figured something was wrong at the database level since a simple count like that should be pretty much instantaneous in a modern database with only ~40,000 records (and three products in the category I was testing). Some sniffing around got me to:
https://github.com/opencart/opencart/issues/177.
So:
create index iproduct_description_language_id on product_description ( language_id );
create index iproduct_to_category_category_id on product_to_category ( category_id);
Now the problem query has gone from 0.43s to 0.00s - possibly the default precision of the MySQL timer will need addressing as Moore's law continues.
This was day one with OpenCart, this issue doesn't full me with confidence, or did the guy who installed it miss something?
[0 Comments
| Add Comment
|
]
]
#479
Posted by simonw on Wed 20 Feb 2013 at 16:04
The primary clipboard is cleared when you click to position mouse pointer in a text component.
GNOME developers seem not to get it at all.
https://bugzilla.gnome.org/show_bug.cgi?id=333514
The result is you can use PRIMARY to paste into a new Gedit document, you can paste content from gedit to itself as long as you don't click to set the pointer after making the selection. All horribly inconsistent with what I think is one of the best features of X, the automatic selection of text on highlight.
Has annoyed me once too often......
# apt-get install vim-gnome
Wonder also how this got declined as a papercut bug in Ubuntu....
#478
Posted by simonw on Tue 19 Feb 2013 at 10:05
Noted in logcheck this morning:
suhosin[7624]: ALERT - configured COOKIE variable limit exceeded - dropped variable 'ASPSESSIONIDCCCCSSST' (attacker 'HETZNER IP ADDRESS', file 'somephpfile')
Repeated for many PHP files on the site. Each page was fetched once with what looked like a browser, and then 4 times with Java. Total time of attack 16s.
User Agent of "browser": Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; InfoPath.2; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 1.1.4322)2011-10-16 20:22:33
User Agent of: "Java/1.7.0_04"
Hazard a guess that someone with ASP is going to discover there is a problem with their cookie handling.
Lack of other traffic suggests this was someone in a hurry to discover if this weakness exists on a lot of sites, rather than something more specific.
suhosin[7624]: ALERT - configured COOKIE variable limit exceeded - dropped variable 'ASPSESSIONIDCCCCSSST' (attacker 'HETZNER IP ADDRESS', file 'somephpfile')
Repeated for many PHP files on the site. Each page was fetched once with what looked like a browser, and then 4 times with Java. Total time of attack 16s.
User Agent of "browser": Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; InfoPath.2; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 1.1.4322)2011-10-16 20:22:33
User Agent of: "Java/1.7.0_04"
Hazard a guess that someone with ASP is going to discover there is a problem with their cookie handling.
Lack of other traffic suggests this was someone in a hurry to discover if this weakness exists on a lot of sites, rather than something more specific.
[0 Comments
| Add Comment
|
]
]