XLAB Tech

“Back in the early days of the web there was this wonderful Perl library called CGI.pm , many people only learned Perl because of it.”
— Quote from: http://mojolicious.org/

And so true….

It can happen, that you need to add new hard drive – HDD to your Citrix Xen machine. After installing the hardware, you start searching for appropriate action to be taken in XenCenter and you see, that there is no way to add newly attached HDD as a Local storage device.

You need to do the following:

Let assume, that your disk is recognized by system as /dev/sdb.

1) Find scsi ID as shown:

# ls -l /dev/disk/by-id/
lrwxrwxrwx 1 root root 9 Oct 22 12:00 scsi-XXXXXXXXX -> ../../sdb

2) Create the SR:
xe sr-create type=lvm content-type=user device-config:device=/dev/disk/by-id/scsi-XXXXXXX name-label=”Local Storage D1″

Go to Xencenter – you should see your HDD, and you should be able to create an LVM based SR on that HDD.

http://www.activestate.com/activeperl/

Following the link the programmer will find nice windows perl

People – go grab your copy

As with anything new coming from Google, there has been a lot of expectation regarding the new Google “operating system” for PCs based on the Chrome browser.

Truth is that, as the mobile-devices-oriented Android, ChromeOS can be hardly called an Operating system but rather a Linux distribution.

ChromeOS’s concept is simple; remove absolutely everything from GNU/Linux that is not essential for browsing the internet. That lets ChromeOS boot quickly, in about 10 seconds, a bit more time than of Moblin or the tuned-up version of Fedora by Intel. Google Chrome has still a lot of glitches, but I’m sure they will fix’em all in time for production.

As soon as we boot ChromeOS we are prompted not for a user on the computer but for our Google account and password. On log-on you are presented with your gmail and calendar in the Chrome browser/interface.

There are no programs available in the machine, and you are expected to use Google Docs for office tasks, gtalk for chat, gmail for your mail, or actually any web based application (e.i. meebo or Photoshop Express or Zimba). And that’s pretty much it; you cannot install any additional desktop programs, services, and you are not even supposed to get to the shell)

Another advantage is the low memory usage, since there are almost no services running in the background (except for network, sound, and printing sometime in the future), and the only application running is the browser.

Apart from not being able to install any desktop application, the main problem with ChromeOS is that, at least so far, it’s so cloud-oriented, that your computer will be absolutely useless if you are not connected to the internet.

Now, my desktop computer is connected to the internet all day, no big deal; but why would I cap-down my inexpensive powerful desktop computer with ChromeOS if I can have the Chrome browser together with any desktop application I want?

Of course, ChromeOS is probably oriented to MIDs, UMPCs, nettops, laptops, and other energy efficient devices to have on the go, right? But those are not connected to the internet 24/7, at least not yet.

On the other hand, there are other non-bloated Linux projects to choose from, from DSL and Puppy to Moblin and Ubuntu Mobile, that provide the full desktop experience.

It’s success will be very much connected with the hype Google produces, but we are still to see if it will let Google jump the threads-hold from something that sounds cool to something that actually is, for instance, by allowing the installation of at least ChromeOS applications. For one, when I thought they were reinventing the wheel with Chrome, they did manage to create something different and useful, so we will see if they’ll succeed again or not.

The review ends here, you can stop reading now.

It’s clear that Google, in spite of its “Don’t be evil” motto has decided to spread its tentacles as wide as possible, competing with other open-source like FireFox, Moblin or NASA’s World Wind, and closed-source initiatives like GPS routing or Photo-albums. Intelligently, whenever competing with a strong open-source option Google decided to open the source to their own projects, at least partially (Andriod, Chrome, ChromeOS), but not with other projects. Let’s not forget that you can use Google Earth, Google Maps, Sketch-up and several other pieces of software at home for personal use, but they are still closed-sourced and not free for commercial use, unlike their open-source counterparts.

Thus, by making users dependant on their software, Google can affect open-source initiatives (World Wind, OpenStreetMaps, Blender, etc.). We’ve already seen how Google sent a cease-and-desist letter to avoid a custom Andriod installation from including gmail and other Google applications. With GoogleOS, they have already said that it will not be available for machines with traditional hard disk drives; a completely artificial restriction.

Our solution was this way for each OSS VM:

* create new VM, boot it and assign a temporary IP
* install rsync on the old and the new VM
* enable SSH root login on the old VM
* run the following rsync command the first time:

rsync -av –numeric-ids –delete –progress –exclude /sys –exclude /boot –exclude /dev –exclude /proc –exclude /etc/mtab –exclude /etc/fstab –exclude /etc/udev/rules.d –exclude /lib/modules root@my-old-vm:/ /

* this copies most files from the old VM, but excludes the running kernel, special filesystems, fstab etc.
* downtime begins: switch the old VM to runlevel 1, disable remote logins, startup the sshd again
* do a final rsync with the command from above
* shutdown the old VM
* downtime ends: reboot the new VM
* install the linux XenTools on the new VM
* cleanup old kernels from the package manager

This way we migrate several VM’s with short downtimes, aslong there are not to many inodes (files) in use. One VM had more than 2 million inodes on a 20 GB partition in use (maildir) – using rsync would take several hours just for creating the file list.

After several months of hard work and testing we are pleased to announce the second public release of XtreemOS, a Linux-based grid operating system. It is GPL/BSD licensed, based on Mandriva Linux, and can be downloaded from http://www.xtreemos.eu/software/downloading.  The source code is available here.

Version 2.0 includes the following functionalities:

• Creation and management of dynamic virtual organizations, management of credentials, permissions and resources.

• Application execution management handles selection of suitable resources, execution of applications on the selected resource. Job and resource monitoring provides the user with a Linux-like view of their applications.

• XtreemFS is a high-performance replicated and distributed file system. By default an auto-mountable XtreemFS home volume is created for each grid user.

• Xosautoconfig is a new tool for quick deployment of XtreemOS.

For more information please refer to the  project web site, the contact e-mail, or join the lively IRC developer channel #xtreemos-dev on irc.freenode.net.

With all the regression bugs in recent kernel releases I had my doubts whether to update my desktop Ubuntu 9.04 Jaunty Jackalope installation to the latest 9.10 Karmic Koala. I’ve read several comparisons between 9.04 and 9.10, but usually those benchmarks use the default (and clean) installations, which in the case of 9.10 brings the EXT4 file-system instead of the EXT3, which yells expected improvements. I wanted to know how would Ubuntu 9.10 compare to the previous version after an update, without updating the file-system.

To do that I used a very recommendable open-source benchmark solution for linux named Phoronix Test Suite, by the Phoronix technical site I really like. I choose the stable 2.0.0 version instead of the 2.2.0 beta because I didn’t feel like investing too much time in any possible eventuality. Even though these are somewhat synthetic tests, there is currently no other way to measure desktop performance.

Thus, I ran the “linux-system” suite (a predefined compound of general purpose tests) before and after upgrading my box. Let me also note that my Linux box was clean installed with Ubuntu 8.04 when it was released, and the upgraded to 8.10 and 9.04 before finally upgrading to 9.10.

My box is an Intel Core 2 Duo CPU E8400 @ 2.99GHz, Intel DP35DP, 82G33/G31/P35/P31 + ICH9R, with 2GB RAM and a 500GB WDC WD5000AACS-0 , and a GeForce 8600 GT 512MB (540/400MHz). My Ubuntu 9.04 install had kernel 2.6.28-16-generic, gcc 4.3.3, GNOME 2.26.1 and X.Org Server 1.6.0, plus I was using the NVIDIA’s 180.44 propietary driver. With 9.10 it now uses kernel 2.6.31, GNOME 2.28.0, X.Org 1.6.4 and GCC 4.4.1, with NVIDIA 185.18.36.

Here are the raw results:

The results show one big regression (we’ll get back to it), 5 minor ones (<7%), 12 minor gains (<15%) and some impressive gains in 7, totalizing 25 test.

The most important gain is in dbench with a 206% improvement that, together with the 36% improvement with desktop-oriented PostMark, somehow invalidating the more synthetic 32-threaded 64MB reads regression. Apache static web page serving boast a 89% improvement.

Processing also seams to have improved with GMPBench yelling a 35% gain, 21% faster GnuPG encryptions and somewhat shorter multimedia encoding times (~7%). SQLite, used by several programs including FireFox, showed a bit of a regression by 7%, but professional database PostgreSQL showed 23% faster transactions.

The rest of the results are below 10% difference, and other losses are within 3%.

Conclusions? Seams like it's worth updating, even though, be warned, you often loose some configuration options in the process, I had to re-configure a couple of things, including Compiz options. Other unavoidable annoyances are changes in the default behaviour of programs, but if it was painless, then it wouldn't be Linux, right?

I'm going to dig a bit on the threaded-reads regression; I think I'll mount my partition with "noatime” and try running the tests again; will keep you updated.

On a mounted partition filesystem check (fsck) of course cannot be done and many tutorials suggest using a live CD of some sort. Here is a solution to force fsck on the next reboot:

1. touch /forcefsck
2. reboot

How to make a DVD from Sony camcoder MTS files in Ubuntu

First I would like to explain what is our goal. Sony camcorder records the movies in *.mts files. Those are not compatible with all DVD players. Our goal is to make a DVD from Sony camcorder movies that can be played with almost every DVD player. First we shoot some movies and transfer them to the PC. Than we need to install mplayer. To get the latest version is good to get it right from the repository.

svn checkout svn://svn.mplayerhq.hu/mplayer/trunk mplayer
./configure
make
make install

To get the more popular AVI file from a MST file just go to the folder where the MTS file is saved and run the next line in the console.

mencoder INPUT.MTS -o OUTPUT.avi -oac copy -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=10000 -fps 50 -vf scale=1280:720

Ok. As you can see I scaled the movie to the lower resolution. You can skip that if you want and you will have the full AVI resolution. But if you would like to make a DVD that could be run on all DVD players, than your movie resolution will have to be lowered during the DVD production. You can also downscale the frames per second (fps) to the desired value.

Now translate one file from MTS to AVI. If there are more files to do, it is good to have a script that will run on all files in the folder:

#!/bin/bash
for file in *.MTS
do
outfile=${file%.MTS}.avi
echo $file ” ” $outfile
mencoder $file -o $outfile -oac copy -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=10000 -fps 60000/1001 -vf scale=1280:720
done

Now we have all files in AVI format. The next step is to use the DeVeDe program to make the DVD. DeVeDe is a nice and easy program to make a DVD. At the beginning we see the window where we can chose what we would like to do(Screenshot 1). The first option “Video DVD” is the best if we would like to make our movies playable on all DVD players.

DeVeDe Screenshot 1

After we choose the first option, we get the window with the settings for our DVD (Screenshot 2). At the top there are two text-boxes. In the first box we can add the titles that will be shown in the menu, in the second we can add files that belong to each title.

DeVeDe screenshot 2

When we have added all the movies to the DVD, we make the DVD menu. Press the “menu options” button and adjust the menu settings and see the preview. For a demonstration I have put the XLAB logo as background and pressed the preview button (Screenshot 3).

DeVeDe screenshot 3

When you are satisfied with your creation you just make the DVD. DeVeDe makes an iso-image that can be burn to your DVD or DVDs with your favorite DVD burn program.

… or the long way to get multi-core machine work.

In one of the previous posts by Tesla, Gregor announced a few stats that are produced by the new multi-core computer. We have two multi-core machines for image processing or parallel computing. The briefing stats are:

• Core i7 processor
• 12 gigs of RAM
• GeForce GTX 275
• Tesla 1060C card

While composing the machine we learned a lot of things that have to be considered. First of all is the power consumption. I newer thought that I would need more than one kilowatt (1kW) of power supply for the computer. First we connected all the components with a 750W power supply and the machine didn’t start… but the 1,25kW PS did the job!

Graphic controller

One of the Tesla machines that we have put together worked great with no problems. Then, few days later, when we got another 1,25kW power supply and finally put together the second computer, we couldn’t make it run. I installed Ubuntu on it, I did some updates and restarted it a few times. Than I disconnected the machine from the power and moved it to a prearranged place, where it would be available to our mind-masters and… it didn’t boot at all! It stuck at the GRUB Error 18 and the story of bringing life to the number-cruncher began. The error 18 hasn’t any description that could be compared with our problem or would show us where the problem is. We also had another machine with the same characteristics and it worked perfectly. I moved the machine back to my place and reinstalled everything and it worked. I plugged the power supply off and on and the machine refused to work. I tried all different disk configurations in the BIOS, but it didn’t help. I saw that BIOS had some problems with the disk, because the size of the 1Tb disk wasn’t properly displayed. It was 33MB or 0MB, depending on the BIOS configurations of AHCI or IDE enhanced. I found out, that I can wake up the disk with the Ubuntu install CD if I boot from the CD, mount the disk with installed operating system and than restart the machine. After this procedure the machine booted from the disk as it should. All worked perfectly until the machine was shut down.

Tesla 1060C

We wondered that maybe the first disk partition is to big, or that maybe there is a problem with a SATA cable or a power cable connected to the disk. So we made separate boot partitions, changed the SATA cable and the SATA channel and replaced the power cable of the disk. All the same. We reinstalled the GRUB, but the result was still the same.

Tesla 1060C

Then the conclusion was – there must be something wrong with the disk. We tried another one. After a one-hour procedure of installing and configuring, the machine still refused to work. So what now? Where is the problem? BIOS? Motherboard?

Everything together

Finally we tried with another disk. This time it was a 2Tb disk from the third manufacturer. Again we put in the Ubuntu CD, installed the software, crossed our fingers … and it WORKED! Even after shutdown or replugging everything worked without a glitch.

We happily concluded that there are some problems with some large disks. We don’t know why. Maybe it is old firmware. We couldn’t say and we just don’t bother till the Tesla machine works…