Saturday, November 22, 2008

autofs #2

Well, I'm back from vacation... the Aspire One survived the journey well, it really is light and compact enough, that you can travel with it without noticing it. The biggest hassle was the lack of a 'global' WiFi network. In Amsterdam it costs you 10 euro's to log in... Melbourne down to $5, in Kuala Lumpur free. At cafe's and hotels a similar story, sometimes free, sometimes expensive... but you find yourself wondering is it worthwhile digging out the credit card. A 3G modem would in an ideal world avoid these hassles. In the real world, there's still the cost of global roaming, and the fact that UMTS hasn't yet completely replaced GSM. Occasionally you'll find yourself barely getting 10kbps.

Anyway back to autofs... two annoyances remain -
- I don't have a dns or mdns or zeroconfig solution on my home network
- On linpus, /etc/hosts is replaced at each boot
- autofs needs the hostname for my NAS (could be avoided using the IP address I guess)
- for some reason autofs isn't starting automatically, although chkconfig shows it configured

Well here's my somewhat ugly solution to the host name problem

[user@localhost ~]$ cat /etc/NetworkManager/dispatcher.d/02-hosts
#!/bin/bash
if [ "$2" = "up" ];
then
FOUND=`grep dlink-EEF76A /etc/hosts | wc -l`

if [ $FOUND = "0" ];
then
echo "Updating /etc/hosts"
DATE=`date`
echo "#updated by $0 at ${DATE}" >> /etc/hosts
echo "192.168.1.10 dlink-EEF76A" >> /etc/hosts
fi;
fi

Sunday, October 19, 2008

autofs

Install autofs (yum install autofs), then follow the link below
Auto-mounting the NAS using autofs -
Accessing Windows Or Samba Shares Using AutoFS
add an entry for the NAS in /etc/hosts

Something fundamental I've missed is that autofs gets the key on demand.
so -
[user@localhost ~]$ ll /cifs
total 0

which isn't promising... but

[user@localhost ~]$ ll /cifs/DLINK-EEF76A/
total 0
drwxrwxrwx 24 user user 0 2008-10-16 01:43 Volume_1
drwxr-xr-x 2 root root 0 2008-10-20 19:21 web_page

is what we want :)

A symbolic link simplifies access -
sudo ln -s /cifs/DLINK-EEF76A /mnt/nas

udev

After some compilation and installation following this post FC8 HP LaserJet M1005 MFP installation -step-by-step-, I managed to get my scanner to work on Linpus. Unfortunately only as root. To get past this, I needed to dig into the world of udev. Here's what I found -

The scanner is found, as follows -
[user@localhost udev]$ sane-find-scanner
found USB scanner (vendor=0x03f0, product=0x3b17) at libusb:001:004

and after updating /etc/udev/rules.d/60-libsane.rules, udev is creating a symlink for the scanner -

[user@localhost udev]$ ll /dev/scanner-1-2.3
lrwxrwxrwx 1 root root 15 2008-10-20 16:33 /dev/scanner-1-2.3 -> bus/usb/001/004
[user@localhost udev]$ ll /dev/bus/usb/001/004
crw-r--r--+ 1 user root 189, 3 2008-10-20 16:54 /dev/bus/usb/001/004

and magically after a restart, it's even creating them with the right owner. A lot of playing around yesterday (perhaps without restarting) had me frustrated, as it was consistently created with root:root as the owner.

Thursday, October 16, 2008

DAR backup

Well I started looking at the different file base alternatives, but relatively quickly put many of them on hold. They each have there benefits, and I'll try to get back to them soon - but here in bugwhine style, I'll just explain why I put them on hold.

unison
Interesting file synchronizing tool.
Unfortunately on hold, as it's written in Objective Caml, so building for Linpus may require a bunch of work, I don't have time to follow this path right now.

rdiff-backup
In brief - File based backup directory -> directory. Hard links and special files, permissions etc. are preserved.
Incremental diffs are preserved in a special directory.
rpm available at livna (rdiff-backup.i386)

Why on hold -
- requires own server on NAS
- backup to/from cifs/samba mount not recommended
- file based

rsnapshot
A filesystem snapshot tool, using rsync.
rpm available at livna - rsnapshot.noarch
Why on hold -
- snapshots may become 'messy' on the backup filesystem
- written in perl (*)

duplicity
Uploads encrypted tar volumes to remote system. Uses librsync.
rpm available at livna - duplicity.i386

Why on hold -
- doesn't handle hard links
- written in python (*)

DAR - Disk Archive
In the end DAR was the first utility I made a successful full and incremental backup with (ie. without putting it on hold). It's available at livna -

[root@localhost ~]# yum install dar.i386

My first impressions from the documentation and the command line switches was that it looked a bit primitive, and that maybe I should have not given up so easily on the other tools.

I tried a complete backup first as follows -

[root@localhost ~]# dar -v -z -R / -P dev -P proc -P mnt -P sys -P tmp -c /mnt/nas/aspire/full

and remained skeptical. It took quite some time, but the resulting file was a compact 915MB.
To put it further to the test, I tried an incremental backup -

[root@localhost ~]# dar -v -z -R / -P dev -P proc -P mnt -P sys -P tmp -A /mnt/nas/aspire/full -c /mnt/nas/aspire/diff1
...
--------------------------------------------
59 inode(s) saved
with 2324 hard link(s) recorded
0 inode(s) changed at the moment of the backup
80778 inode(s) not saved (no inode/file change)
0 inode(s) failed to save (filesystem error)
5 inode(s) ignored (excluded by filters)
0 inode(s) recorded as deleted from reference backup
--------------------------------------------
Total number of inode considered: 80842
--------------------------------------------

At this point my skepticism had almost completely died. The incremental backup was very fast, and efficient. Being 'tar' style (single file on the destination device), there aren't issues with cifs/samba permissions. A clean result, but still with the possibility to quickly restore individual files.

[root@localhost ~]# du -h /mnt/nas/aspire/*
3.0M /mnt/nas/aspire/diff1.1.dar
915M /mnt/nas/aspire/full.1.dar

The remaining thing bothering me, is manually specifying the paths to exclude from the backup. Perhaps a small thing, and perhaps something I should tune further.

I need to look further into a backup schedule, recovery methods and management of the archives with dar_manager - but this feels like a good start.

Thumbs up DAR.

(*) Perl and Python fans - I'm not against these languages at all. I just find that scripted solutions tend to be less well integrated than compiled solutions. Output messsages, exception handling etc, is often unprofessional in solutions based on scripting languages - and it was fairly clear that this is true also for rsnapshot and duplicity.

Tuesday, October 14, 2008

Backup of Aspire One to DNS-323

I want to backup my Aspire One to my DNS-323 NAS, there are a plethora of ways to do this, so I thought I'd evaluate them here.

The following methods came to mind immediately -
- copy the whole hard disk/partition image (ghost style)
- copy at a file level, rsync style

Initially I like the whole image method. It speeds up the recovery process, and with a Linux style implementation, it's probably possible to loop mount the image, if one just wants to recover some files anyway. Or would this require 120GB or RAM? :)

The first obstacle of the image method is that you probably need to have the file system either not mounted or mounted read-only. This could be a hassle - I certainly don't want to have to boot on an external drive to perform a backup... but maybe dropping down to single user mode would be ok.

It's worth noting that the Aspire one comes with a recovery DVD (bootable via external drive or copy to USB key), that can restore the system to it's factory install. This should be a reasonable starting point for then restoring changed applications/data/configuration from the backup mechanisms evaluated here.

Partimage
Well I liked the look of partimage, and decided to start here. Rather than setup partimaged on the NAS, it seems mounting the filesystem was an easier way to go. See below -

[user@localhost ~]$ su - Password:
[root@localhost ~]# modprobe cifs
[root@localhost ~]# mkdir /mnt/nas
[root@localhost ~]# mount -t cifs //192.168.1.10/Volume_1 /mnt/nas
Password:
[root@localhost ~]# wget "http://dag.wieers.com/rpm/packages/partimage/partimage-static-0.6.5-1.rh7.rf.i386.rpm"

[root@localhost ~]# rpm -i partimage-static-0.6.5-1.rh7.rf.i386.rpm
warning: partimage-static-0.6.5-1.rh7.rf.i386.rpm: Header V3 DSA signature: NOKEY, key ID 6b8d79e6
[root@localhost ~]# partimage-static


I then added an fstab entry for the partition
//192.168.1.10/Volume_1 /mnt/nas cifs auto,rw,user=guest,password=guest

but soon wandered off into issues with mounting it as a normal (non root user). For now I'm relying on sudo.
The big challenge is to unmount the root partition, or remount it in read-only mode, such that partimage can be safely used to produce a consistent image.

As the standard Linpus install only has one partition + swap, this challenge stumped me. The only options I see are unfortunately booting off external media, or doing some repartitioning. This is where I give up on this method for now.

Next I'll take a look at unison or rdiff-backup -
http://www.cis.upenn.edu/~bcpierce/unison/
http://www.nongnu.org/rdiff-backup/

ssh and telnet

Well, what's a Linux install without ssh and telnet? I'm trying to resist hacking my new toy before I've learned it's pitfalls.. but ssh is something I need.

I'm starting as suggested by "The road to Elysium", let's hope it's a good road.

Alt-F2, Check 'Run in terminal', Run

[user@localhost ~]$ su -
Password:
[root@localhost ~]# yum -y install yum-priorities
fedora 2.1 kB 00:00
primary.sqlite.bz2 4.9 MB 00:07
updates 100% |=========================| 2.6 kB 00:00
primary.sqlite.bz2 100% |=========================| 2.2 kB 00:00
Setting up Install Process
Parsing package install arguments
Resolving Dependencies
--> Running transaction check
---> Package yum-priorities.noarch 0:1.1.8-1.fc8 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================
Package Arch Version Repository Size
=============================================================================
Installing:
yum-priorities noarch 1.1.8-1.fc8 fedora 8.1 k

Transaction Summary
=============================================================================
Install 1 Package(s)
Update 0 Package(s)
Remove 0 Package(s)

Total download size: 8.1 k
Downloading Packages:
(1/1): yum-priorities-1.1.8-1.fc8.noarch.rpm 8.1 kB 00:00
warning: rpmts_HdrFromFdno: Header V3 DSA signature: NOKEY, key ID 4f2a6fd2
Importing GPG key 0x4F2A6FD2 "Fedora Project " from /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora
Importing GPG key 0xDB42A60E "Red Hat, Inc " from /etc/pki/rpm-gpg/RPM-GPG-KEY
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing: yum-priorities ######################### [1/1]

Installed: yum-priorities.noarch 0:1.1.8-1.fc8
Complete!


Check the contents of yum-priorities, edit it to match the below -

[root@localhost ~]# cat /etc/yum/pluginconf.d/priorities.conf
[main]
enabled = 1
check_obsoletes = 1

Install the livna repository -
[root@localhost ~]# rpm -Uvh http://www.fedorafaq.org/f8/yum http://rpm.livna.org/livna-release-8.rpm
Retrieving http://www.fedorafaq.org/f8/yum
Retrieving http://rpm.livna.org/livna-release-8.rpm
warning: /var/tmp/rpm-xfer.N4ZUob: Header V3 DSA signature: NOKEY, key ID a109b1ec
Preparing... ########################################### [100%]
1:livna-release ########################################### [ 50%]
2:yum-fedorafaq ########################################### [100%]

Some quick checking with yum shows openssl is ready installed, but not openssh.

[root@localhost ~]# yum list openssh*
Loading "priorities" plugin
119 packages excluded due to repository priority protections
Available Packages
openssh.i386 4.7p1-2.fc8 fedora
openssh-askpass.i386 4.7p1-2.fc8 fedora
openssh-clients.i386 4.7p1-2.fc8 fedora
openssh-server.i386 4.7p1-2.fc8 fedora
[root@localhost ~]# yum install openssh*
Loading "priorities" plugin
119 packages excluded due to repository priority protections
Setting up Install Process
Parsing package install arguments
Resolving Dependencies
--> Running transaction check
---> Package openssh-askpass.i386 0:4.7p1-2.fc8 set to be updated
---> Package openssh-clients.i386 0:4.7p1-2.fc8 set to be updated
---> Package openssh.i386 0:4.7p1-2.fc8 set to be updated
---> Package openssh-server.i386 0:4.7p1-2.fc8 set to be updated
filelists.xml.gz 100% |=========================| 173 kB 00:01
filelists.sqlite.bz2 8.0 MB 00:22
filelists.sqlite.bz2 100% |=========================| 224 kB 00:01
filelists.sqlite.bz2 100% |=========================| 3.0 kB 00:00
filelists.sqlite.bz2 100% |=========================| 394 kB 00:00
filelists.sqlite.bz2 100% |=========================| 1.1 kB 00:00
filelists.sqlite.bz2 100% |=========================| 76 kB 00:00
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================
Package Arch Version Repository Size
=============================================================================
Installing:
openssh-askpass i386 4.7p1-2.fc8 fedora 38 k
openssh-clients i386 4.7p1-2.fc8 fedora 463 k
openssh-server i386 4.7p1-2.fc8 fedora 264 k
Installing for dependencies:
openssh i386 4.7p1-2.fc8 fedora 284 k

Transaction Summary
=============================================================================
Install 4 Package(s)
Update 0 Package(s)
Remove 0 Package(s)

Total download size: 1.0 M
Is this ok [y/N]: y
Downloading Packages:
(1/4): openssh-server-4.7p1-2.fc8.i386.rpm 264 kB 00:00
(2/4): openssh-4.7p1-2.fc8.i386.rpm 284 kB 00:00
(3/4): openssh-clients-4.7p1-2.fc8.i386.rpm 463 kB 00:00
(4/4): openssh-askpass-4.7p1-2.fc8.i386.rpm 38 kB 00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing: openssh ######################### [1/4]
Installing: openssh-askpass ######################### [2/4]
Installing: openssh-clients ######################### [3/4]
Installing: openssh-server ######################### [4/4]

Installed: openssh-askpass.i386 0:4.7p1-2.fc8 openssh-clients.i386 0:4.7p1-2.fc8 openssh-server.i386 0:4.7p1-2.fc8
Dependency Installed: openssh.i386 0:4.7p1-2.fc8
Complete!

And, oh how I love it when things work first go!

[root@localhost ~]# ssh root@192.168.1.10
The authenticity of host '192.168.1.10 (192.168.1.10)' can't be established.
RSA key fingerprint is 06:86:21:cb:61:3d:4f:4d:02:7f:41:22:0a:2b:52:0d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.10' (RSA) to the list of known hosts.
root@192.168.1.10's password:
root@dlink-EEF76A:~# uname -a
Linux dlink-EEF76A 2.6.12.6-arm1 #29 Wed Apr 30 10:03:59 CST 2008 armv5tejl unknown
root@dlink-EEF76A:~# exit
Connection to 192.168.1.10 closed.

and for completeness -

[root@localhost ~]# yum install telnet.i386
...
Installed: telnet.i386 1:0.17-41.fc8
Complete!
[root@localhost ~]# telnet 192.168.1.10
Trying 192.168.1.10...
Connected to 192.168.1.10.
Escape character is '^]'.



BusyBox v1.12.0.svn (2008-07-11 12:43:14 CEST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

# exit
Connection closed by foreign host.

Monday, October 13, 2008

It has arrived

All the way from Finland, shiny and small :)