GlobalMegaBlog

Parkinson’s Law states that:

Work expands so as to fill the time available for its completion.

I was about to propose a corollary to this and apply it to data storage, but if you check the Wikipedia link above, you’ll find they’ve already got it covered:

Data expands to fill the space available for storage.

Our fileserver had a 120G drive in it up until about a year ago. It ran out, and I replaced it with a new RAID5 setup, and today we have:

Filesystem            Size  Used Avail Use% Mounted on
/dev/md1              1.4T  1.3T   73G  95% /data

That’s 1.3 Terrabytes out of 1.4 used, for my non computer geek readers (which is.. what, no-one?). An interesting comparison is my very first computer hard drive which I bought around 1992 or so, which was 50MB. The above partition is (approximately) 1402519MB, or a frankly ridiculous 28,050 times larger than my first drive, and it’s still running out of space.

My next upgrade will probably be 4*2T drives, which in a RAID5 arrangement will give me about 6T of usable space, hopefully. Of course, I’m going to have to Ebay a kidney to pay for it, so in the meantime, I may have to find some stuff to delete.

Anyway, my point seems to be that we’re basically coming up with new ways to fill data that is keeping pace with the growth in hard drive technology. 17 years ago (wow, is that when 1992 was?), 50MB was plenty of space for operating system, applications, data, etc.. These days, we’ve got video files, just one of which would easily fill 50MB for a few seconds of footage. Add in HD and you’re talking serious growth. Where does it stop? I fully expect that another 17 years from now we’ll be measuring our storage usage in Exabytes, as we download our daily brain backup. Gulp.

This is one those “for future reference” posts, mostly for myself, but perhaps it will help some other poor unfortunate from running into the same problems I had tonight.

Specifically, setting up bridged networking under Linux to enable a KVM virtual machine. I had this working just fine up until the point where I upgraded this server to hardy, whereupon everything fell apart. Thanks to the folks on freenode.net/#kvm, I discovered that my interface configuration was broken.

To summarise, I have:

1 Physical network interface
3 IP addresses (2 of which are assigned to the host OS, one to the guest VM).

It turns out that if you have a bridge set up on the interface, assigning a second IP address should be an alias to the bridge, not the first IP’s interface.

Initially, I had something like this:

auto eth0
iface eth0 inet manual

auto br0
iface br0 inet static
        address xxx.xx.xxx.xxx
        netmask xxx.xxx.xxx.xxx
        broadcast xxx.xx.xxx.xxx
        gateway xxx.xx.xxx.xxx
        bridge_ports eth0
        bridge_fd 9
        bridge_hello 2
        bridge_maxage 12
        bridge_stp off

auto eth1
iface eth0:1 inet static
        address xxx.xx.xxx.xxx
        netmask xxx.xxx.xxx.xxx
        broadcast xxx.xx.xxx.xxx
        gateway xxx.xx.xxx.xxx

There are several things wrong with this. The first is that the gateway should not be specified on the second interface. It wasn’t a problem in gutsy, but hardy did things somewhat differently, with the result that networking was completely disabled on reboot. IPMI to the rescue, fortunately.

Having fixed that, things seemed to work ok, but I was informed that the secondary IP should more accurately be assigned to the bridge, like this:

auto eth0
iface eth0 inet manual

auto br0
iface br0 inet static
        address xxx.xx.xxx.xxx
        netmask xxx.xx.xxx.xxx
        broadcast xxx.xxx.xxx.xxx
        gateway xxx.xx.xxx.xxx
        bridge_ports eth0
        bridge_fd 9
        bridge_hello 2
        bridge_maxage 12
        bridge_stp off

iface br0:0 inet static
	address xxx.xx.xxx.xxx
        netmask xxx.xxx.xxx.xxx

One swift edit, and things are working swimmingly well. One of the folks on freenode #kvm hinted that he might spend some time soon writing some, you know, documentation on this process, so in future it won’t require reading dozens of blog posts (like this one) to understand what the hell you’re doing. Sounds like a great plan to me. The most defining characteristic of the current documentation is that it’s almost as shallow as Paris Hilton, and decidedly less thick too.

I love the internet. It’s an explosion of information. It allows anyone from anywhere to learn anything about anything (more or less). The old adage used to be that you learn something new every day. With the advent of the internet, the “day” part of that is becoming hour, or maybe minute.

Anyway, all this is a roundabout way of getting to talk about TED. TED is a conference which has been going on since 1984. Their site explains their mission much more thorougly than I am here, but the basic gist is that they gather together people from the worlds of Technology, Entertainment and Design to give 18 minute lectures on basically anything. Since April 2007, they’ve been posting videos of conference sessions on their site, and the output is nothing short of remarkable.

If you’re a science geek, you could spend hours watching videos on this site. Some of the ideas and technologies on display are breathtaking, and it makes you feel like we’re actually going somewhere as a species. In particular, I want to highlight the video I just watched. Go ahead, take the 18 minutes and watch this:

Isn’t that just the most amazing thing ever? Not just the fact that this group of scientists are figuring out such incredible stuff, or what this could mean for humanity, but the fact that it’s available for anyone to watch and learn about. It feels like this is what the internet was built for – to spread ideas, share knowledge, to advance our civilization. Ok, maybe that’s a bit lofty, but it things like this that help me counteract all the doom and gloom in the world today. On the one hand, the planet may be in an environmental death spiral, but on the other, we’ve got insanely smart people figuring out stuff that will maybe help save us in the end.. and thanks to the internet, we all get to participate in it, or at the very least watch.