Weblog entry #240 for Steve

Code overhaul .. volunteers welcome
Posted by Steve on Fri 8 Aug 2014 at 10:14
Tags:

The code behind this site is badly in need of another overhaul.

At the moment it runs on a set of ropy CGI scripts that are neither efficient nor pleasant.

My current plan is to record the basics from scratch, then port over functionality as time goes on. As part of that I think I'll be coming up with some toy-servers:

  • A HTTP interface that just gets/updates/adds articles.
  • A HTTP server that is solely responsible for get/set comments.

With that structure in mind the core of the site can just mediate between requests and the actual backend - without worrying about actual implementation-details.

In terms of features the only thing I think I'm going to remove is the comment-feed RSSs.

There is a feed for the comments on every article - and that feed gets spidered like mad, for articles that are many years old.

I'm open to the idea of collaboration if there are users who wish to help - and the code will be on github in due course.

 

Comments on this Entry

Posted by Anonymous (2.126.xx.xx) on Tue 12 Aug 2014 at 12:48

I spent four hours today reworking the RSS feed handling; this is now significantly faster.

I also removed the site-message feature, on the basis that few people used it and the biggest recipient of messages was myself - and I usually get email alerts when the site goes down anyway ..

[ Parent | Reply to this comment ]

Posted by Steve (2.126.xx.xx) on Wed 13 Aug 2014 at 10:05
[ View Steve's Scratchpad | View Weblogs ]

Now our Ajax code is overhauled too, which means the tagging and similar operations are much faster.

Steve

[ Parent | Reply to this comment ]

Posted by Steve (2.126.xx.xx) on Sat 16 Aug 2014 at 15:45
[ View Steve's Scratchpad | View Weblogs ]

The general speed of the site should now be better - The most commonly-requested pages are now persistent:

  • login / logout
  • Static pages, such as the FAQ.
  • The tag-cloud and tag-search results page.

The next big job will be overhauling the front-page, and the article views.

Steve

[ Parent | Reply to this comment ]

Posted by Anonymous (212.110.xx.xx) on Tue 19 Aug 2014 at 14:24

I guess I should say, for completeness, the code overhaul is done for the moment.

The back-end has three distinct handlers, each of which is now a distinct CGI::Application process, running under fastcgi:

  • A handler for Ajax requests.
  • A handler for RSS feed requests.
  • A handler for the site-proper.

Bug reports welcome; I'd not be surprised by new ones ..

[ Parent | Reply to this comment ]

Posted by Steve (2.126.xx.xx) on Sun 24 Aug 2014 at 10:53
[ View Steve's Scratchpad | View Weblogs ]

My final update is now ready!

Previously we used varnish as a load-balancer, and pound for SSL termination.

That has now been dropped, and HAProxy is used for both purposes.

[ Parent | Reply to this comment ]

Posted by Anonymous (201.209.xx.xx) on Tue 26 Aug 2014 at 00:25
An article about HAProxy would be good.

[ Parent | Reply to this comment ]

Posted by Steve (2.126.xx.xx) on Tue 26 Aug 2014 at 23:10
[ View Steve's Scratchpad | View Weblogs ]

That isn't a bad idea, but I guess I'm not doing anything crazy:

  • Listen on :80.
  • Listen on :443.
  • Use least-loaded to pass to the four back-ends.

Steve

[ Parent | Reply to this comment ]