Saturday, January 19, 2013

SSH and performance overhead

One of the first things I do when I come home – I open a port on the router with upnpc, log into my office's PC and set up a reverse ssh connection to my notebook, then I use sshfs to mount the remote filesystem.

I know that ssh uses encryption (thanks, Cap) and that comes at the price of CPU time. I guess, modern CPUs have instructions to accelerate encryption.

What's the price?

So, let's begin the experiment

$ dd if=/dev/urandom of=bigfile bs=8M count=50

gives me a big (400M) file. By the way, it took 107.675 s (3.9 MB/s) on i7-2630QM (only 1 core was loaded).

Let's just read it. First time, after a reboot:
$ dd if=bigfile of=/dev/null bs=8M
50+0 records in
50+0 records out
419430400 bytes (419 MB) copied, 1.7027 s, 246 MB/s


Consecutive runs:
$ dd if=bigfile of=/dev/null bs=8M
50+0 records in
50+0 records out
419430400 bytes (419 MB) copied, 0.0819643 s, 5.1 GB/s


Let's read that via ssh:
$ sudo sshfs -o allow_other,uid=1000 user@localhost:/home/user/ /home/user/self/
$ dd if=self/bigfile of=/dev/null bs=8M
50+0 records in
50+0 records out
419430400 bytes (419 MB) copied, 6.985 s, 60.0 MB/s


The same experiment, on an old AMD Athlon(tm) 64 Processor 3000+ (1 Ghz): 38.663 s, 10.8 MB/s

Ok, here is the price.



Tuesday, August 14, 2012

Partimage & Windows

I do backups.

Backups by themselves are not useful. Joel said it well: "don't ask about backups, ask about restores".

This is a long (and perhaps boring) story log about restoring an image of a Windows XP, made by partimage. The restored image didn't boot...

Intro
I use ubuntu and windows. On that PC, ubuntu has it's own partition for system files and another one as a swap. WinXP has it's own partition, for system files and programs. And there is a data-only ntfs partition.

One day I had a powercut, XP was running at the moment, for the last time. It wouldn't boot again. I didn't go on and restore it, since it wasn't needed :) My daily activity was to write my thesis in TeX, and ubuntu is just perfect for that.

Then one day I wanted to see what a few linux distros look and feel like, so I went on removing the windows partition, created an extended one, with enough partitions for every distro. I did a bad thing: I reduced the size of the data partition for that.

Time went on, I finished my experiments and I needed the WinXP back. So, all the distros were deleted and a new primary partition was created.

Log
The backup was 20Gb in size, and it took ~12m to restore. At this time, to the second partition, /dev/sdb2.

I restored the partition, it didn't boot. Just a blank screen and a blinking cursor.

Booted into linux, mounted the partition, all the files were there. Checked all the things. Found that despite it was ntfs in nature, I forgot to set the proper code in partitions table. Changed the code to 7 (HPFS/NTFS/exFAT). Rebooted the pc. Same result: blank screen...

Found the installation CD and booted it. For some reason, it didn't suggest to boot into recovery console...

Booted into linux, found a tiny recovery console iso (~7MB), burned it onto a blank cd, booted it. The recovery console didn't see the windows partition...

Booted into linux. Checked all things again. I noticed (in partimage info) that the original partition was the third one, sdb3. Googled and found that that may be the reason. So I deleted sdb2, created the new small one sdb2 and another one & large sdb3. (And, for the second time, I needed to look into the image, so that I would create sdb3 of exactly the same size, in sectors). Restored the image into sdb3. Booted it again, no luck: blank screen.

Booted into linux. Mounted sdb3 to ensure all is there...

At this moment restoring the backup image became the primary goal. Of course, I could drop it, install the OS anew and all the software. After all, my data was intact on a separated partition. But, why do I do backups? I needed to know where was the mistake.
After all, I did restores with partimage. And each time the restoration went fine, both with linux and windows. What was wrong this time? I took care to re-create the partition of exactly the same size, at exactly the same partition number. But the windows' loader didn't start. So, the problem much be with the boot loader. And the final difference was the start of the partition, which probably was different.

Deleted sdb2, sdb3. Created a 10Gb sdb2. Took the original WinXP CD and installed the OS onto sdb2.


After installing WinXP, booted into ubuntu Live CD, chroot'ed into my installed ubuntu and restored the grub loader.

Booted into linux. Made a backup of the new OS with partimage :) Made a screenshot of a window with fdisk's printout. Deleted sdb2, created new one of the same size, but with a different start. Restored the OS from the image.

Booted and saw a blank screen. Hurra! At least some success. Now I know that the start of partition affects window's loader.

Booted into linux, took a look at hexview of sdb2. I found that in there the original start of the partition is stored at offset 1C, double-word little-endian. Installed ghex. Saved the first 512 bytes into a file boot1, copied it into boot2, in boot2 changed the start (at offset 1C), and wrote boot2 back to sdb2.

Rebooted. Hurra! Windows booted.

Rebooted into linux. Created the sdb3 and restored the WinXP again. Again, saved the first 512 bytes into a file boot1, copied it into boot2, in boot2 changed the start (at offset 1C), and wrote boot2 back to sdb3.

Rebooted. The loader went fine, but windows itself hanged.

A strange thing. Windows' bootloader stores the start in its own place, now I know. But one installation of windows worked fine if I changed the start manually, the other one didn't.

Booted into linux. Looked at the original start (that was saved in boot1), deleted sdb3 and re-created at the start it was back then. Restored the WinXP.

Rebooted ... and it was running fine.

Hurra! I do restores.

Bottom line

When restoring a bootable windows partition into a different place, you need to
1) run restoration in simulation mode. partimage will tell the exact size.
2) ensure the target partition is of the right size
3) run restoration but quit early. The first sector will be there anyway, you need to look at offset 1C of what was the original start.
4) ensure the target partition has the same start.
5) restore =)

Update:  out of curiosity, I booted into linux and made the windows partition a second one. WinXP didn't boot. That means, the exact partition number is also important.