New User? Register here - Existing Users: Username: Password: [Advanced Login]

 

 

Current Poll

Your preferred Interactive shell?









( 1347 votes ~ 14 comments )

 

Setting up multiple Subversion repositories

Posted by hruske on Tue 9 Aug 2005 at 11:20

With subversion (and svn-buildpackage) becoming popular hosting multiple repositories with different access controls is something that is commonly required. Here we'll show how you can set this up in a very flexible manner using PostgreSQL

This article was inspired after reading the guide on LinuxJournal written by Cristiano Paris. However this introduction describes a more flexible setup.

Since this guide is for Debian Sarge, it may use different versions of the software.


We will be using PostgreSQL with Apache 2 WebDAV and mod-auth-pgsql. Packages you will need to install are:

  • postgresql
  • postgresql-contrib
  • subversion
  • apache2
  • libapache2-mod-auth-pgsql
  • libapache2-svn
PostgreSQL and database setup

Installing PostgreSQL is pretty easy. Be sure to select the proper encoding settings, ideally this would be the same as your system locale.

For storing password in PostgreSQL table we'd like some encryption, so we need to install pgcrypto too. Do this by switching to user postgres and running pgcrypto.sql:

root @ box # su postgres
postgres @ box $ psql template1 < /usr/share/postgresql/contrib/pgcrypto.sql  
SET
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION

This creates some new functions, including crpyt, which we will use to store encrypted passwords in database. First we need to create the database and new user:

postgres @ box $ createdb subversion
CREATE DATABASE
postgres @ box $ createuser subversion
Shall the new user be allowed to create databases? (y/n) n
Shall the new user be allowed to create more new users? (y/n) n
CREATE USER

Now we can already map the database structure, which is mostly defined by mod_auth_pgsql. For the SQL queries to work, you need to connect to the database:

postgres @ box $ psql subversion
subversion=#

Now we need a table for users ...

CREATE TABLE users (username char(32),pass text,email varchar(80));
GRANT SELECT ON users to subversion;

... and a table for groups ...

CREATE TABLE groups (username char(32),memberof char(64));
GRANT SELECT ON groups to subversion;

... and a table for logging authorizations. One can never know.

CREATE TABLE log (uname char(20), time timestamp(8), uri varchar(512),ip inet);
GRANT INSERT ON log TO subversion;

We also need to change the subversion's password.

ALTER USER subversion WITH ENCRYPTED PASSWORD 'secretpass';
Apache 2 and WebDAV

After getting Apache2 running make sure you enable mod_dav, mod_dav_svn and mod_auth_pgsql. On using the Debian tools for Apache2:

root @ box # a2enmod dav
root @ box # a2enmod dav_svn
root @ box # a2enmod auth_pgsql

When this is set up, we need to make the layout for repositories. Default Debian location is /var/lib/svn, and I'm going to use this default. In /var/lib/svn create this layout:

/var/lib/svn/
/var/lib/svn/conf
/var/lib/svn/conf/policies
/var/lib/svn/repository

We need some config files for Apache, so create /var/lib/svn/conf/mod_dav_svn.conf with following content:

Include /var/lib/svn/conf/default_policy.conf
Include /var/lib/svn/conf/policies/*
DavLockDB /var/lib/svn/repository/DavLock

Link location /etc/apache2/mods-available/dav_svn.conf to this file. Also create new file /var/lib/svn/conf/default_policy.conf:

<Location /svn/>
        Dav svn 
        SVNParentPath /var/lib/svn/repository
        <LimitExcept GET PROPFIND OPTIONS REPORT>
            Order deny,allow
            Deny from all
            # for access from 'safe' ip, uncomment following line
            # allow from 192.168.0.0/24
        </LimitExcept>
</Location>


The policies for each project are then stored in folder /var/lib/svn/conf/policies/ and are included by Apache. This means Apache needs to be restarted when new project is added. The repository needs to be Apache writable, therefore we need to adjust permissions.

find /var/lib/svn/repository -type f -exec chmod 660 {} \;
find /var/lib/svn/repository -type d -exec chmod 2770 {} \;
chown -R root.www-data /var/lib/svn/repository

Apache needs one more setting set up properly. Add this stanza to /var/lib/svn/conf/mod_dav_svn.conf:

<Location /svn>
        Order deny,allow
        Allow from all
</Location>

Viewing files with browsers should now work.

Setting up the authentication with mod_auth_pgsql

First we need to allow connections for user subversion to PostgreSQL database subversion. Add this line to /etc/postgresql/pg_hba.conf:

host    subversion  subversion  127.0.0.1         255.255.255.255   md5

Be sure, to put it before line

host    all         all         127.0.0.1         255.255.255.255   ident sameuser

You may need to add your real IP too, not only 127.0.0.1, so either add it now or check logs later.

We will now create a test project to make sure this works. Create a new file /var/lib/svn/conf/default_auth.conf with common parameters for mod_auth_pgsql:

AuthType Basic
Auth_PG_host localhost
Auth_PG_port 5432
Auth_PG_database subversion
Auth_PG_user subversion
Auth_PG_pwd secretpass
Auth_PG_pwd_table users
Auth_PG_uid_field username
Auth_PG_pwd_field pass
Auth_PG_grp_table groups
Auth_PG_grp_group_field memberof
Auth_PG_grp_user_field username
Auth_PG_cache_passwords on
Auth_PG_connection_reuse on
Auth_PG_log_table log
Auth_PG_log_uname_field uname
Auth_PG_log_date_field time
Auth_PG_log_uri_field uri
Auth_PG_log_addrs_field ip

Create a new file /var/lib/svn/conf/policies/blah intended for access control to a project, having following content:

<Location /svn/blah/>
        <LimitExcept GET PROPFIND OPTIONS REPORT>
                AuthName "Subversion repository for project blah"
                require group blah
        </LimitExcept>
</Location>

Next we need to create subversion repository in /var/lib/svn/repository/blah:

root @ box # mkdir /var/lib/svn/repository/blah
root @ box # svnadmin create /var/lib/svn/repository/blah

You can not authenticate user, if you don't have one, therefore we will create a 'test' user to see if we've got authentication set up properly:

root @ box # su postgres
postgres @ box $ psql subversion
subversion=# INSERT INTO users VALUES ('test',crypt('pass','seed'),'test@localhost');
subversion=# INSERT INTO groups VALUES ('test','blah');

We can now try and hopefully succeed in importing Revision 1 of our project.


If you are having trouble setting up, be sure to check Apache and PostgreSQL logs to see what is going on.

Share/Save/Bookmark


Posted by hruske (193.2.xx.xx) on Wed 10 Aug 2005 at 16:44
[ Send Message ]
uhm ... Apache directives with lessthan and greaterthan have been stripped?

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Wed 10 Aug 2005 at 16:47
[ Send Message | View Steve's Scratchpad | View Weblogs ]

Ugh.

I just checked my mailbox and they were never present - I don't know how I didn't notice.

If you give me updated bits I can fix it.

There are times when I really hate having to handle stuff like this :S

Steve
-- Steve.org.uk

[ Parent | Reply to this comment ]

Posted by hruske (193.2.xx.xx) on Wed 10 Aug 2005 at 23:31
[ Send Message ]
I've put a copy at http://www.kiberpipa.org/~hruske/subversion.html

Oh, and it would be smart to change the ip field from char to inet. ;) I forgot that earlier.

[ Parent | Reply to this comment ]

Posted by Anonymous (193.170.xx.xx) on Fri 12 Aug 2005 at 14:40
I also want to point you to mod_authz_svn, which allows you to use directory based access control within repositories:

http://svnbook.red-bean.com/en/1.1/ch06s04.html#svn-ch-6-sect-4.4 .2

[ Parent | Reply to this comment ]

Posted by Anonymous (148.136.xx.xx) on Fri 16 Sep 2005 at 07:36
Am I the only one who cannot make this work?

I have the default_auth.conf, but I havn't told apache anywhere to use it, and the guide doesn't tell me to. Is there a step missing? Or am I just not finding it...

[ Parent | Reply to this comment ]

Posted by Anonymous (193.2.xx.xx) on Fri 16 Sep 2005 at 08:25
How on earth did I miss that?

Yes, you have to add it to /var/lib/svn/conf/policies/blah, into the LimitExcept directive.

thanks for pointing this out.

[ Parent | Reply to this comment ]

Posted by Anonymous (148.136.xx.xx) on Fri 16 Sep 2005 at 11:18
Thanks for the reply, and all that...

But..

Exactly how do I type? I tried a few different things, but I have so far failed to figure out the magic word to use before the filename (yes, I've looked in the mod_auth_pgsql documentation, as well as som general apache auth documentation)

[ Parent | Reply to this comment ]

Posted by Anonymous (4.154.xx.xx) on Sat 15 Oct 2005 at 03:36
I too would like to know the answer to this question. Two months since publishing and there's no revision? These directions are really shady. They should have been checked before being posted. I appreciate the effort, but a half-written article almost does more harm than good.

[ Parent | Reply to this comment ]

Posted by Anonymous (81.152.xx.xx) on Sun 9 Sep 2007 at 11:35
Just add "Include /var/lib/svn/conf/default_auth.conf" as the first line in the <LimitExcept> bit.

[ Parent | Reply to this comment ]

Posted by hackeron (212.36.xx.xx) on Mon 24 Oct 2005 at 14:25
[ Send Message ]
If I were to write such a howto, I would go with pam (i.e. pam-mysql or pam-ldap) or even htauth for simplicity instead of auth-pgsql. Many WebDAV users dont want a separate password database.

I.E. register on a forum, and get read-only access to svn/webdav.

[ Parent | Reply to this comment ]

Posted by Anonymous (193.2.xx.xx) on Tue 25 Oct 2005 at 07:14
You could still use mod_auth_pam, this is basically just an example.

[ Parent | Reply to this comment ]

Posted by blackm (212.202.xx.xx) on Thu 15 Dec 2005 at 23:05
[ Send Message | View Weblogs ]
hruske, thanks for this article. Yesterday evening I set up my subversion server based on your article. I used MySQL instead of pgsql and also installed horde's version-controll frontend chora. This allows users to change their svn password.

Now I can start to develop software :-)

--
browse ManPages online!

[ Parent | Reply to this comment ]

Posted by kaybenleroll (66.250.xx.xx) on Tue 30 Oct 2007 at 21:22
[ Send Message ]
I keep getting a Server 500 error. There appears to be a problem with mod_auth_pgsql it seems.

I've looked at the error log, and the following error line get logged:

[Mon Oct 29 13:42:53 2007] [error] [client xxx.xxx.xxx.xxx] (9)Bad file descriptor: Could not open password file: (null)

The pgsql logs have not registered any kind of connection attempt.

Anyone got any idea what I am doing wrong?

[ Parent | Reply to this comment ]

Posted by Anonymous (213.165.xx.xx) on Sun 11 Nov 2007 at 21:49
I'm having the same problem with mod_auth_pgsql.

Apache/2.2.6 (Debian) mod_auth_pgsql/2.0.3 DAV/2 SVN/1.4.4 mod_python/3.3.1 Python/2.4.4 mod_ssl/2.2.6 OpenSSL/0.9.8g

I've isolated the problem to the mod_auth_pgsql module, I think, unless it is another auth module which is looking for an htaccess file or something similar.

Rory

[ Parent | Reply to this comment ]

Posted by Anonymous (194.204.xx.xx) on Fri 7 Dec 2007 at 14:33
This usually stands when, on that or another cause - your auth_pgsql cannot connect to the Postgres server.

No entry in pg_hba.conf ? Server down ? A typing mistake ?

[ Parent | Reply to this comment ]

Posted by kaybenleroll (87.192.xx.xx) on Wed 13 Feb 2008 at 13:44
[ Send Message ]
Yes, for some reason no connection is been made to the database.

I've made sure the module is enabled, and my pg_hba.conf is fine. The server is up and I have made no typos. I get the same error all the time (the bad password one, as before).

I even ran tcpdump on the network interface at the time I tried to view the page, and I noticed that no connection attempt appears to be made.

Anyone have any idea what I might be doing wrong, or suggest things I could look at to try to figure out what is happening?

[ Parent | Reply to this comment ]

Posted by Anonymous (68.42.xx.xx) on Sun 6 Apr 2008 at 20:50
I ran into the same problem and found that setting
AuthBasicAuthoritative Off

Corrects the issue with recent apache versions.

[ Parent | Reply to this comment ]

Posted by Anonymous (193.34.xx.xx) on Mon 8 Sep 2008 at 21:15
To prevent Apache from filling error.log with messages about being unable to open password file add this directive too:
AuthUserFile /dev/null

[ Parent | Reply to this comment ]

Posted by Anonymous (209.206.xx.xx) on Wed 7 May 2008 at 17:04
I wanted to say that these instructions are just fine. They are offered as a baseline by someone who knows what they are doing and how to get there. There's a bit of assumption that you, the user, are skilled enough to understand the steps he's omitted for brevity. I was able to get this system up and running - from building a scratch sys on XEN to logging with my firefox client to see Revision 0, in around 1.5 hours. Not super fast, but suitable. I thank you for the effort you put into this, and consider the instructions here nothing less than perfect.

[ Parent | Reply to this comment ]

Posted by Anonymous (84.10.xx.xx) on Tue 10 Mar 2009 at 19:59
Is this right to access svn repo via http in browser and apache doesn't require password? Cause when importing via svn it has asked for password. And i think it also should ask for password when accessing repo via http in browser...

[ Parent | Reply to this comment ]

 

 

Flattr