Category: Issues

Linux copy GPT partition table with dd

I recently had to copy the partition table of a 3TB disk in a situation where tools such as sfdisk could not be installed.

Since GPT table length is dependant on the number of partitions, you need to do some investigation.

In this case, it was a ‘QNAP’ server that had fdisk (no GPT support) and parted.

On a working drive, run

parted -ms /dev/sda print

Note the number of partitions.

Formula = (128*N)+1024

Where N is the number of partitions you have. In this case I had 4, so I end up with a value of 1536

dd if=/dev/sda of=GPT_TABLE bs=1 count=1536

You now have a backup of a valid partition table you can apply to another drive

dd if=GPT_TABLE of=/dev/sdb bs=1 count=1536

Once this was done, you can manually re-add the drive.

mdadm –manage /dev/md0 –add /dev/sdb3

If you are wondering how we determined the sd[a-z], we accomplished this through hot-swapping the drive to generate logs indicating the drive.

Now why this supposedly automated RAID product required this…

View processes (PID) using disk I/O

Install the sysstat package

apt-get install sysstat

# iostat -m
Linux 2.6.27-14-generic (slowaris)      07/28/2009      _x86_64_

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
4.88    6.65    3.03    1.49    0.00   83.95

Device:            tps    MB_read/s    MB_wrtn/s    MB_read    MB_wrtn

md0             221.24         0.64         0.78    1765002    2159490

We can see that I/O isn’t terribly high here, but there are 221 transfers per second going on.

To check what processes are causing I/O

echo 1 > /proc/sys/vm/block_dump

tail -f /var/log/syslog | grep md0

Jul 28 08:20:12 server kernel: [2752582.434647] kvm(17362): READ block 1017744552 on md0
Jul 28 08:20:12 server kernel: [2752582.502401] kvm(17362): READ block 615283608 on md0
Jul 28 08:20:13 server kernel: [2752582.634622] kvm(17362): READ block 1017744576 on md0
Jul 28 08:20:14 server kernel: [2752583.964709] kvm(17362): READ block 1017744608 on md0
Jul 28 08:20:14 server kernel: [2752584.372889] kvm(1868): dirtied inode 17367041 (live-default-32.img) on md0
Jul 28 08:20:14 server kernel: [2752584.372908] kvm(1868): dirtied inode 17367041 (live-default-32.img) on md0

We see that kvm is causing some disk I/O; now where know where to start investigating!

To turn off these messages

echo 0 > /proc/sys/vm/block_dump

Enable network virtio (Gigabit) in libvirt/kvm Virtual Machine Ubuntu 8.10

If you create a domain with virt-manager, it does not always add the appropriate ‘virtio’ entry to the xml file for your VM.

If this is the case, you will notice that network speeds on the VM will not break 10/100 speeds.

To fix this, edit the xml file

vi /etc/libvirt/qemu/vm.xml

Make sure your network block looks like this (keep your MAC address the same)

<interface type='network'>
<mac address='00:16:36:0b:4c:9c'/>
<source network='default'/>
<model type='virtio'/>
</interface>

Shut down the VM and confirm it’s shut down

virsh shutdown vm

# virsh list
Connecting to uri: qemu:///system
Id Name                 State
----------------------------------

If you do not see your vm “running” you can proceed. If it’s still running you need to shut it down another way.

Update the xml definition

virsh define /etc/libvirt/qemu/vm.xml

Start the domain

virsh start vm

Log into the VM and test with a 1GB file

dd if=/dev/zero of=/tmp/test bs=1M count=1000
scp /tmp/test [email protected]:/tmp

You should see transfer speeds above 11-12 megabytes/second.

Apache2 hangs with ‘Digest: generating secret for digest authentication’

I’m running Apache2+SSL inside a libvirtd (kvm) Virtual Machine.

After enabling SSL and trying to add another SSL site, apache should refuse to restart and the following error would show up in /var/log/apache2.log

Digest: generating secret for digest authentication

After doing some research, it turns out that the VM did not have enough entropy to generate much of anything. Increasing this is easy, but may not be completely secure.

# cat /proc/sys/kernel/random/entropy_avail
139

# apt-get install rng-tools

# rngd -r /dev/urandom -o /dev/random

# cat /proc/sys/kernel/random/entropy_avail
2220

After running rngd, the entropy will increase at a gradual rate.

If you want this to survive a reboot, you’ll need to put it in a startup script.

Recover from accidental mkfs using fsck

Recently I accidentally ran mkfs.ext3 on the wrong RAID1 array.

As soon as I realized what had happened, I shut power off from the PC.

Booting to a live CD, I ran

mdadm –assemble /dev/md0 /dev/sda1 /dev/sdb1

fsck -y /dev/md0

I was able to then mount the drive

mkdir /media/md0

mount /dev/md0 /media/md0

ls /media/md0/lost+found

And the majority of my files were saved.

Check to see if your filesystem is ext2 or ext3

mount | grep md0

If it’s ext2, convert it to ext3

umount /media/md0

tune2fs -j /dev/md0

WARNING: If you choose to run fsck manually (without -y), choosing “n” to Clear? and “y” to Fix? will cause you to lose that file.

Debian: IPtables and net protection should be enabled by default

As I come across network security issues in Linux, I can’t help but wish that Debian had decent default firewall rules (and kernel network protection in general) like many other distros.

This applies to both Debian 4.0 and Ubuntu 7.10.

By default, a Ubuntu or Debian install of Linux is vulnerable to a SYN flood. If behind a router, chances are there won’t be a problem. But if it’s a standalone Linux machine connected to the internet, the machine is vulnerable to a DoS attack.

It’s a simple fix.

echo 1 > /proc/sys/net/ipv4/tcp_syncookies

or

iptables -A INPUT -i eth0 -p tcp –tcp-flags ALL ACK,RST,SYN,FIN -j DROP
iptables -A INPUT -i eth0 -p tcp –tcp-flags SYN,FIN SYN,FIN -j DROP
iptables -A INPUT -i eth0 -p tcp –tcp-flags SYN,RST SYN,RST -j DROP

The second should probably be used if it’s a server. The first way may still have quite a few open SYN connections.

You may just want to block the IP(s) that are conducting a SYN flood on you (keep an eye on netstat or iptables.log). A script that does this would be pretty useful; I wonder if there are any out there (or if I have to write one?).

iptables -I -s ip.to.ban -j DROP

Along with rules to help protect against flooding, it seems like it would be a good idea to block all ports by default.

iptables -A INPUT -m limit –limit 100/second –limit-burst 150 -j RETURN

One solution could be to have a folder called /etc/security/iptables that contains files that get passed to iptables at startup (in the same way /etc/rc2.d gets read in numeric order). So you could have files like 22ssh, 23ftp, etc. with iptable rules in each file. You could also have an ‘ENABLED’ variable like some files in /etc/default have (so that ports wouldn’t be opened by default; the user would have to manually enable them for the port to be opened).

Then they’d just run /etc/init.d/iptables restart and the port would be opened (flush the rules, reapply).

I’ve put together a proof-of-concept for this; feel free to check it out. Extract it somewhere, and check out the source. Then try the following

tar xvzf debian-iptables.tar.gz -C /

Which will create and fill /etc/security/iptables and put an init file in /etc/init.d and a link in /etc/rc2.d

It hasn’t been tested as much as I’d like, but it should work.

Please do this at the computer itself; if you do it over the network you might be locked out.

http://bbis.us/debian-iptables.tar.gz

st0rage.org outages

I figured I’d post some information here about recent outages.

I plan on moving st0rage.org over to a new server soon, and there are going to be some changes (user quotas for the first time ver, yay!). I’ll also be offering a paid service, along with an increased free service.
It’s been a lot of work, and has caused some downtime (usually less than 30 minutes). People using irssi I’m sure havn’t been happy. 😉

Sorry about all the trouble, things will be much nicer soon (moving to a much more stable connection).

Vostro 1400 Wireless Problems (ipw3945)

A problem with Debian and Ubuntu (and possibly other distros) is that the ipw3945 module fails after a large amount of activity.

The solution is very simple, but it took a lot of work to find it.

Edit /etc/modules and put the following in there

iwl3945

Edit /etc/modprobe.d/blacklist and add

blacklist ipw3945

That’ll do it. The iwl3945 module performs just as well on sftp transfers, and I havn’t noticed any problems with it.