Setting up multiple Subversion repositories
Posted by hruske on Tue 9 Aug 2005 at 11:20
With subversion (and svn-buildpackage) becoming popular hosting multiple repositories with different access controls is something that is commonly required. Here we'll show how you can set this up in a very flexible manner using PostgreSQL
This article was inspired after reading the guide on LinuxJournal written by Cristiano Paris. However this introduction describes a more flexible setup.
Since this guide is for Debian Sarge, it may use different versions of the software.
We will be using PostgreSQL with Apache 2 WebDAV and mod-auth-pgsql. Packages you will need to install are:
- postgresql
- postgresql-contrib
- subversion
- apache2
- libapache2-mod-auth-pgsql
- libapache2-svn
Installing PostgreSQL is pretty easy. Be sure to select the proper encoding settings, ideally this would be the same as your system locale.
For storing password in PostgreSQL table we'd like some encryption, so we need to install pgcrypto too. Do this by switching to user postgres and running pgcrypto.sql:
root @ box # su postgres postgres @ box $ psql template1 < /usr/share/postgresql/contrib/pgcrypto.sql SET CREATE FUNCTION CREATE FUNCTION CREATE FUNCTION CREATE FUNCTION CREATE FUNCTION CREATE FUNCTION CREATE FUNCTION CREATE FUNCTION CREATE FUNCTION CREATE FUNCTION CREATE FUNCTION CREATE FUNCTION CREATE FUNCTION CREATE FUNCTION
This creates some new functions, including crpyt, which we will use to store encrypted passwords in database. First we need to create the database and new user:
postgres @ box $ createdb subversion CREATE DATABASE postgres @ box $ createuser subversion Shall the new user be allowed to create databases? (y/n) n Shall the new user be allowed to create more new users? (y/n) n CREATE USER
Now we can already map the database structure, which is mostly defined by mod_auth_pgsql. For the SQL queries to work, you need to connect to the database:
postgres @ box $ psql subversion subversion=#
Now we need a table for users ...
CREATE TABLE users (username char(32),pass text,email varchar(80)); GRANT SELECT ON users to subversion;
... and a table for groups ...
CREATE TABLE groups (username char(32),memberof char(64)); GRANT SELECT ON groups to subversion;
... and a table for logging authorizations. One can never know.
CREATE TABLE log (uname char(20), time timestamp(8), uri varchar(512),ip inet); GRANT INSERT ON log TO subversion;
We also need to change the subversion's password.
ALTER USER subversion WITH ENCRYPTED PASSWORD 'secretpass';Apache 2 and WebDAV
After getting Apache2 running make sure you enable mod_dav, mod_dav_svn and mod_auth_pgsql. On using the Debian tools for Apache2:
root @ box # a2enmod dav root @ box # a2enmod dav_svn root @ box # a2enmod auth_pgsql
When this is set up, we need to make the layout for repositories. Default Debian location is /var/lib/svn, and I'm going to use this default. In /var/lib/svn create this layout:
/var/lib/svn/ /var/lib/svn/conf /var/lib/svn/conf/policies /var/lib/svn/repository
We need some config files for Apache, so create /var/lib/svn/conf/mod_dav_svn.conf with following content:
Include /var/lib/svn/conf/default_policy.conf Include /var/lib/svn/conf/policies/* DavLockDB /var/lib/svn/repository/DavLock
Link location /etc/apache2/mods-available/dav_svn.conf to this file. Also create new file /var/lib/svn/conf/default_policy.conf:
<Location /svn/>
Dav svn
SVNParentPath /var/lib/svn/repository
<LimitExcept GET PROPFIND OPTIONS REPORT>
Order deny,allow
Deny from all
# for access from 'safe' ip, uncomment following line
# allow from 192.168.0.0/24
</LimitExcept>
</Location>
The policies for each project are then stored in folder /var/lib/svn/conf/policies/ and are included by Apache. This means Apache needs to be restarted when new project is added. The repository needs to be Apache writable, therefore we need to adjust permissions.
find /var/lib/svn/repository -type f -exec chmod 660 {} \;
find /var/lib/svn/repository -type d -exec chmod 2770 {} \;
chown -R root.www-data /var/lib/svn/repository
Apache needs one more setting set up properly. Add this stanza to /var/lib/svn/conf/mod_dav_svn.conf:
<Location /svn>
Order deny,allow
Allow from all
</Location>
Viewing files with browsers should now work.
Setting up the authentication with mod_auth_pgsqlFirst we need to allow connections for user subversion to PostgreSQL database subversion. Add this line to /etc/postgresql/pg_hba.conf:
host subversion subversion 127.0.0.1 255.255.255.255 md5
Be sure, to put it before line
host all all 127.0.0.1 255.255.255.255 ident sameuser
You may need to add your real IP too, not only 127.0.0.1, so either add it now or check logs later.
We will now create a test project to make sure this works. Create a new file /var/lib/svn/conf/default_auth.conf with common parameters for mod_auth_pgsql:
AuthType Basic Auth_PG_host localhost Auth_PG_port 5432 Auth_PG_database subversion Auth_PG_user subversion Auth_PG_pwd secretpass Auth_PG_pwd_table users Auth_PG_uid_field username Auth_PG_pwd_field pass Auth_PG_grp_table groups Auth_PG_grp_group_field memberof Auth_PG_grp_user_field username Auth_PG_cache_passwords on Auth_PG_connection_reuse on Auth_PG_log_table log Auth_PG_log_uname_field uname Auth_PG_log_date_field time Auth_PG_log_uri_field uri Auth_PG_log_addrs_field ip
Create a new file /var/lib/svn/conf/policies/blah intended for access control to a project, having following content:
<Location /svn/blah/>
<LimitExcept GET PROPFIND OPTIONS REPORT>
AuthName "Subversion repository for project blah"
require group blah
</LimitExcept>
</Location>
Next we need to create subversion repository in /var/lib/svn/repository/blah:
root @ box # mkdir /var/lib/svn/repository/blah root @ box # svnadmin create /var/lib/svn/repository/blah
You can not authenticate user, if you don't have one, therefore we will create a 'test' user to see if we've got authentication set up properly:
root @ box # su postgres
postgres @ box $ psql subversion
subversion=# INSERT INTO users VALUES ('test',crypt('pass','seed'),'test@localhost');
subversion=# INSERT INTO groups VALUES ('test','blah');
We can now try and hopefully succeed in importing Revision 1 of our project.
If you are having trouble setting up, be sure to check Apache and PostgreSQL logs to see what is going on.
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
Ugh.
I just checked my mailbox and they were never present - I don't know how I didn't notice.
If you give me updated bits I can fix it.
There are times when I really hate having to handle stuff like this :S
Steve
-- Steve.org.uk
[ Parent | Reply to this comment ]
Oh, and it would be smart to change the ip field from char to inet. ;) I forgot that earlier.
[ Parent | Reply to this comment ]
http://svnbook.red-bean.com/en/1.1/ch06s04.html#svn-ch-6-sect-4.4 .2
[ Parent | Reply to this comment ]
I have the default_auth.conf, but I havn't told apache anywhere to use it, and the guide doesn't tell me to. Is there a step missing? Or am I just not finding it...
[ Parent | Reply to this comment ]
Yes, you have to add it to /var/lib/svn/conf/policies/blah, into the LimitExcept directive.
thanks for pointing this out.
[ Parent | Reply to this comment ]
But..
Exactly how do I type? I tried a few different things, but I have so far failed to figure out the magic word to use before the filename (yes, I've looked in the mod_auth_pgsql documentation, as well as som general apache auth documentation)
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
I.E. register on a forum, and get read-only access to svn/webdav.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
Now I can start to develop software :-)
--
browse ManPages online!
[ Parent | Reply to this comment ]
I've looked at the error log, and the following error line get logged:
[Mon Oct 29 13:42:53 2007] [error] [client xxx.xxx.xxx.xxx] (9)Bad file descriptor: Could not open password file: (null)
The pgsql logs have not registered any kind of connection attempt.
Anyone got any idea what I am doing wrong?
[ Parent | Reply to this comment ]
Apache/2.2.6 (Debian) mod_auth_pgsql/2.0.3 DAV/2 SVN/1.4.4 mod_python/3.3.1 Python/2.4.4 mod_ssl/2.2.6 OpenSSL/0.9.8g
I've isolated the problem to the mod_auth_pgsql module, I think, unless it is another auth module which is looking for an htaccess file or something similar.
Rory
[ Parent | Reply to this comment ]
No entry in pg_hba.conf ? Server down ? A typing mistake ?
[ Parent | Reply to this comment ]
I've made sure the module is enabled, and my pg_hba.conf is fine. The server is up and I have made no typos. I get the same error all the time (the bad password one, as before).
I even ran tcpdump on the network interface at the time I tried to view the page, and I noticed that no connection attempt appears to be made.
Anyone have any idea what I might be doing wrong, or suggest things I could look at to try to figure out what is happening?
[ Parent | Reply to this comment ]
AuthBasicAuthoritative Off
Corrects the issue with recent apache versions.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]