Problems syncronising files with unison

Posted by kmlhk79 on Wed 8 Nov 2006 at 11:59

I want to sync my notebook home directory data to my Debian Sarge server using Unison. Unison is a file-synchronisation tool which allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.

I have tested the Unison on a Desktop PC running Sarge with no problem.

However, My notebook is running Debians testing release. I receive an "incorrect version" error when I try to sync my notebook data to my Debian Sarge Server with Unison.

I searched the web and found that Unison does not talk to each other on different version.

This is problem. I am using Debian Sarge ( stable ) on the server side. It was because stability is very important to me.

I am using Debian Testing on my notebook, because fast updates on new Desktop Apps is the key for desktops.

The Unison in Debian stable is on version 2.9.1 and version 2.13.16 on Debian Testing. And they are not talk with each other.

1. Are there any workarounds for this problem ?

(P.S. I am not going to down grade or upgrade the Debian on my notebook or server, I know that running the same Debian version will solve this problem, But I don't want to do that.).

2. Up to now, Unison does not support UniCode.

If the file/directory name is not ASCII , You will get a strange filename after syncing the files.

This is bad on multilingual system.

So, is there any better solution to synchronise files between two hosts? (P.S. I really do mean sync not backup.)

Any comment are welcome.

 

 


Posted by Anonymous (83.5.xx.xx) on Wed 8 Nov 2006 at 12:15
Take a look at the unison2.9.1 package, made specifically for this purpose: http://packages.debian.org/unison2.9.1

-- Shot

[ Parent | Reply to this comment ]

Posted by dom (217.147.xx.xx) on Wed 8 Nov 2006 at 14:53
[ Send Message ]
Aha, great stuff.

This does have the advantage (if the server is the one using unison2.9.1) that you need to modify the client's command to use the explicitly named newer binary on the server side.

Another option which I have been using up until now is to install the version of unison from sarge on testing, which works fine. You can use the dpkg/apt/aptitude 'hold' function to prevent it from getting upgraded to the etch version.

[ Parent | Reply to this comment ]

Posted by kmlhk79 (222.167.xx.xx) on Fri 10 Nov 2006 at 14:52
[ Send Message ]
But how about the Apt source list ?

Should I mix the testing and sarge in the Apt sources list ?

I have never try to mix both of them.

When I use apt-get install apps , what will happen ?

Will it install testing or sarge version ?

Can you give me some example source-list which mix up the two ?

K. M. Lau

[ Parent | Reply to this comment ]

Posted by jrms (200.1.xx.xx) on Wed 8 Nov 2006 at 12:21
[ Send Message ]
You can try with rsync.

[ Parent | Reply to this comment ]

Posted by jaddle (66.36.xx.xx) on Wed 8 Nov 2006 at 14:11
[ Send Message ]
Rsync only goes in one direction though - in other words, if you have two machines, A and B, with a copy of some files on both, and make changes on both of them, rsync won't work properly. It will just copy A to B, and wipe out the changes made to B, and vice versa.

Unison is designed to prevent that, by taking into account the changes in both directions. SVN does the same thing, it should be noted, though I think it's optimized for source trees, and not random files. I've found it rather annoying to work with that with anything else... I haven't tried unison, though I might look into it.

[ Parent | Reply to this comment ]

Posted by Anonymous (218.190.xx.xx) on Thu 9 Nov 2006 at 19:21
I guess you can run rsync -u once in each direction.

[ Parent | Reply to this comment ]

Posted by jaddle (66.36.xx.xx) on Wed 8 Nov 2006 at 14:32
[ Send Message ]
Though probably not useful to you, I thought I'd mention the offlineimap program. It only works for IMAP mailboxes, (syncing to maildir, or to other IMAP servers), but seems to work in the right way - you can make all kinds of changes to both accounts, and they're both be propagated in both directions. It's much faster and more reliable than using the offline mode in any mail client I've tried.

It seems that this is the sort of thing that unison does, though unison works in a generalized way - any files containing any sort of data can be synced, instead of just IMAP folders.

[ Parent | Reply to this comment ]

Posted by Anonymous (84.191.xx.xx) on Wed 8 Nov 2006 at 15:41
Hey,

can't help you with your second question. sry.

but what about installing the "stable" version of unison on your laptop?
on testing you should already meet the needed dependencys:
- libc6 (>= 2.3.2.ds1-21)
- libc6.1 (>= 2.3.2.ds1-21)
- libncurses5 (>= 5.4-1) ...a.s.o.

so putting a stable source in your sources.list, something like "APT::Default-Release "testing";" in the apt.conf, getting an update und apt-get install unison/stable should be an idea, isnt it?

mika
http://michbeck.homeftp.org/blog/

[ Parent | Reply to this comment ]

Posted by Anonymous (192.38.xx.xx) on Thu 9 Nov 2006 at 10:07
I would download the tgz-source and compile it. Instead of make_install I would use checkinstall (apt-get install checkinstall) which makes a debian-package and install your compiled package in the debian-system. That is, you can remove it with dpkg -r bla-bla in case you want to update or libraries are update so that you have to recompile.

[ Parent | Reply to this comment ]

Posted by spiney (85.127.xx.xx) on Thu 9 Nov 2006 at 17:03
[ Send Message ]
Actually, this is not that easy, since unison is written in ocaml, so you drag in a whole lot of packages as build depends (re-building all the stuff for the maemo platform was a rather lengthy procedure, for the result see the link below).

So it's definitely easier to use a binary package in this case.
--
Unison for the Nokia 770

[ Parent | Reply to this comment ]

Posted by kmlhk79 (222.167.xx.xx) on Fri 10 Nov 2006 at 16:23
[ Send Message ]
For my second question , Unison does not support Unicode.

It is normall , as most Open Source developer who did not consider or don't care to supporting multiLangauge on their program/apps . That I can understand.

However , the problem for Unison not supporting Unicode is on the file system level not on the language side.

I would like to high light this problem here and alert the Open Source developer to aware of the problem.

I found the following statement from the Unison home page FAQ:

"There are some known limitations (discussed in many messages on the unison-users mailing list) in the way that Unison handles non-ASCII characters in the names of files and folders. So if you use diacritics / accent marks, umlaute or anything more exotic in your file and folder names, you'll have to take special care. The word "exotic" is based on a 7-bit ASCII environment ... you get the point!

On 6/1/06, Benjamin Pierce <bcpierce@cis.upenn.edu> wrote:
[snip]
> Unfortunately, I'm less sanguine about the ease of fixing things.
> There are lots of ways in which Unison doesn't deal well with Unicode
> and other character encoding issues -- basically, Unison itself just
> ignores all such issues and takes whatever it gets from the lower-
> level OCaml / Posix filesystem libraries, string libraries, etc.
> Doing all of this right would be very valuable, but at the moment no
> one is signed up to do it. (Volunteers welcome, of course! :-)

The best way to avoid problems is to avoid any kind of non-ASCII characters in file names. "

The most interesting statement is that "to avoid any kind of non-ASCII characters in file names. "

For non english country, it will be impossible to avoid using non-ASCII characters in file names.

So, It only means that " the applicaton is not for non-english country".

Hope someone can help to solve Unsion unicode problem.

If not , I hope other developer may aware the problem and include Unicode support on thier apps as default.

Regards,
K. M. Lau

[ Parent | Reply to this comment ]

Posted by Anonymous (217.155.xx.xx) on Sun 17 Dec 2006 at 20:45
Its worth making sure that your systems are all correctly setup with UTF8 support.

To Linux and probably other kernels, file names are just a string of raw 8-bit bytes, and they allow the use of absolutely any character in a file name. I'm English and I use symbols from Unicode in file names, but they still get synchronized without problem between Linux and Mac systems.

In the standard C library (usually the lowest level before the kernel) the old solution used that extra bit above ASCII to create many code pages suited to each language, problem is it often results in mismatches when converting between them. UTF8 is the ideal solution and represents every available Unicode character by using that extra bit in a smart way.

UTF8 needs to become the universal default, thats what people using characters beyond ASCII need to push for, as it'll resolve many of todays inconsistencies.

--
Alan
coincident@gmail.com

[ Parent | Reply to this comment ]

Posted by Anonymous (217.83.xx.xx) on Sun 11 Mar 2007 at 08:50
In fact, UTF-8 and Unicode works for me in certain situations:

* Syncing between two GNU/Linux boxes over ssh, both with UTF-8 locale works.
* Syncing between two Windows XP boxen over ssh also works.
*Syncing between Windows and GNU/Linux works via SMB, i.e. when the share is assigned a drive letter. In that case, from Unison's point of view it is a syncronisation between two local directories. Samba is setup
*What does *not* work is syncronisation between Windows and GNU/Linux over ssh. Here, the UTF-8 file names from the Linux file system are always misrepresented as two-character sequences on the Windows side.

[ Parent | Reply to this comment ]

Posted by kmlhk79 (222.166.xx.xx) on Sun 11 Mar 2007 at 09:50
[ Send Message ]
What if the Samba shared directory or file name were created by Windows XP ( i.e. Traditional Chinese Version) ?

Have you try to syncronising files with unison between two Debian Sarge with that directory ?

Let me assume the following case for the study:

1. Assume there are a Debian Sarge server - "Master" running Samba(Sarge Version) with a shared directory named "public" .

The public directory are R/W by all users.

2. There are two type of users who will access and R/W that public directory.

Type A. user are running Debian 3.1 Sarge with Gnome/KDE.
Type B users are running Windows XP .

3. There are anther Debain Sarge server - "Slave" with same configuration as Server - "Master".

The Debain Sarge server - "Slave" will running Unsion to Sync the Server - "Master" each day.

The question :

Will the UTF-8 file names from the Linux file system misrepresented as two-character sequences on the server - "Slave" side after sync with the server -"Master" ?

Notes : The Samba were designed to be able to access by Window and Linux . It may be the problem on the Samba not on the Unsion.

K. M. Lau

[ Parent | Reply to this comment ]

Posted by mvanbaak (80.126.xx.xx) on Wed 8 Nov 2006 at 20:17
[ Send Message ]
I'm using subversion for this (and a lot of other things as well)
One of the added benifits of using subversion is that you can sync whenever you have a net connection. It uses https so almost every firewall/router/proxy setup allows you to sync.
And, you get versioned changes. So if you ever mess up (we all do regulary right) you can simply go back to a past version.

[ Parent | Reply to this comment ]

Posted by tsykoduk (67.185.xx.xx) on Fri 10 Nov 2006 at 06:44
[ Send Message ]
Yeah... I svn across two macs and two linux boxes. Could not live with out it :)

[ Parent | Reply to this comment ]

Posted by b88zhou (70.137.xx.xx) on Sat 11 Nov 2006 at 17:19
[ Send Message ]
If you want to go the version control route, you're probably better off with distributed version control tools - such as darcs or mercurial. I can recommend mercurial, for text files.

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

Which init system are you using in Debian?






( 1045 votes ~ 6 comments )