Create a full featured and compact Oracle Linux Virtualbox VM in 15 minutes

AKA: “Honey, I shrunk my Linux”

Overview

Linux VMs such as Oracle Linux 6.6 start with an install footprint of about 5GB. This makes it inconvenient to hand off to other people and you are constantly looking for bigger drives to hold the VM in.

How about we just cut out all the bits that are irrelevant to a VM - such as bluetooth, printing, wireless, firmwares, additional kernels and the likes?
Here is one way to do it.

• Get a list of rpms that is an absolute minimum
• Create a KickStart configuration file
• Host the kickstart cfg file on HFS or similar server
• Now, copy the drive over using CloneZilla - which only copies occupied bytes.
• All should be done in about half hour tops.

Pre install checks

Have the following ready:

1. VirtualBox
2. Oracle Linux 6.6
3. HFS server from: http://www.rejetto.com/hfs/?f=dl
4. CloneZilla: http://clonezilla.org/ has an iso that can be mounted to boot with.
5. You can also use PartedMagic - which contains a copy of CloneZilla from http://partedmagic.com/doku.php?id=start#.UXhGlFeOOjg
6. KickStart cfg file :
• OL6.6:  https://raw.github.com/venkman69/vmshrink/master/VBOX/OL66/ks66vbox-2disk.cfg

Notes about Kickstart: 

• Kickstart file is sensitive to the CR/LF in the file. Make sure that the file is saved as unix LF only otherwise the shell commands within will fail. 
• Another user called 'oracle' is setup by the installation of the oracle-rdbms preinstall package.
• The kickstart cfg will:
• Allow you to set the hostname
• Set the root and oracle user passwords to 'password'
• Setup the second drive as /opt
• Create /opt/oracle and set the owner to 'oracle'
• Add user 'oracle' to sudoers file
• And importantly install only those bits that are needed.
• The kickstart file also does not setup the logical volume. For things like clonezilla, partitionmagic etc LVM is just beginning to be supported or not supported. This keeps setup friendly and easy.
• You can add a new hard drive once this is all done to host all your 'software'. This is a super simple step especially if you are not setting up LVM. It is just a simple mount setup in your fstab.
• The 6.6 Kickstart file uses 2 disks. Other than that all instructions are identical.

Installation:

Setup a new VirtualBox VM and pre-install steps

• Create a new VM image for the Oracle Linux VM: (img. not updated for 6.6)
• Setup a single dynamically allocated 20gb drive. AND one 100GB drive. The 100GB drive will be mounted as /opt/

• I have started calling the first disk “Pre” since it will be swapped out by the skinny disk1 at the end of this process. I name the skinny one correctly as d1.

• Setup 2 network adapters, one for VM to look towards the internet and one to allow your host to access the VM (VirtualBox does not provide the translation that VMware does and many posts use this configuration to setup the VM).
• First network card is "NAT"

• Second is "Host Only"

• Set the Oracle Linux ISO file in the 'drive'.

• It should finally look like
  
• Start HFS

• Add the kickstart configuration file (drag and drop)

This completes the initial prep.

Installing Oracle Linux Using Kickstart

• Start the VM and hit the TAB button when you get the install screen.
  Now, add to the options:
  ks=http://192.168.56.1/ks66vbox-2disk.cfg

• Next select the first network card

 

 The kickstart config should ask for a hostname, hit enter to accept kickstart cfg default and move on. (image not updated for 6.6)

  

• The installation should commence and about 705 packages will be installed. 

• After packages are installed, the post install script should kick in and setup the following:
• Add 'oracle' as a sudo user
• Create /opt/oracle and chown to oracle
• Add vboxsf group to oracle (virtualbox shared folder group)
• Setup the RPMFORGE repository

• Next it should install the packages htop, meld and the kernel headers. The kernel headers are important to setup the virtualbox tools.
• Now, kickstart will remove some more unnecessary kernel items and some firmware items (enter a y)

• When the vm reboots it will still launch the installer, but you can safely poweroff the VM and remove the Oracle Linux iso from the drive.

Take it for a spin 

You should be able to login as 'oracle' and password as 'password' which was setup in the kickstart file. Also, root password is 'password' as well. Again this is purely for a VM install for testing therefore there is no fear of an insecure environment etc. Also, ora user should be able to 'sudo'.

NOTE: Do not run yum yet. This will add about 300MB of downloaded index files and such.

Disable YUM Refresh!

Disable YUM refresh in preferences otherwise right after you login you will gain 300MB. Change the updates to "Never"

Screen saver is annoying in a VM. Disable it as well. Preferences>ScreenSaver

Shrinking the VM drive

Now, setup the clonezilla iso in the drive. And add a new 20GB harddrive as shown below

• Add clonezilla in the drive

• Start the VM back up

• Select defaults to get going

• Select device to device

• Beginner mode is just fine

• Disk to disk

• Select first disk

• Select second disk

• You can skip this, but if things fail, select the auto fix (I am truly lazy)

• yes, yes, and yes yes and yes.

• Choose 0 to Shutdown the VM after it is done.

• And take a look at your handiwork

• Notice that the new disk is about 2GB. That is a solid 600MB savings. (I called my disk1 'pre1' and my final clonezilla disk 'd1')
  
• You can now edit the settings and remove the original disk

The ova is compressed and is around 700MB
Hope you enjoyed that.

Jython swing UI similar to zenity

For simple UI to request user information. Such as input dialogs with lists, input boxes, file/dir chooser, tree list.

http://sourceforge.net/projects/jythonsimpledialog/

Eclipse + PyDev + PortablePython or Jython + Local install

Portable and client installable packages are so much easier with Jython since it is a small jar and better yet very portable and has the ability to call java jars into its fold.
I also like pure-python implementations for the same reason as there is no concern with portability.

So, I am recording steps for a local install and how to use a package.
In this example I will use xlrd and xlwt.

Prerequisites:

• MingW or MSYS tools so I can run shell scripts. This is obviously far more usable than just for this one reason. You can get it from http://www.mingw.org/wiki/MSYS . You can also install MobaXterm a one-stop-shop that has cygwin stuffed into it.
• A way to launch python or jython. 
• Python: This is reasonably simple: python portable has to be added to $PATH then you can run 'python' from command line.
• Jython: This is slightly more work but simple:

#!/bin/bash 
#export JAVA_HOME=/usr/java/latest

export JYTHON_HOME=`msys2winpath "/PortableApps/Jython2.7b1"`
#export JYTHON_HOME=`msys2winpath "/PortableApps/jython2.5.3"`
export JVMOPTS="$JAVA_OPTIONS $JVMOPTS"

export JYTHONCACHEDIR="./jythoncache"
echo $JYTHONPATH
export PATH=$JYTHON_HOME/bin:$PATH
export CLASSPATH="$CLASSPATH;$JYTHON_HOME/jython.jar"
export JVMOPTS="$JVMOPTS -Dpython.home=$JYTHON_HOME -Dpython.cachedir=$JYTHONCACHEDIR" 

export PYTHONPATH=$JYTHONPATH

#echo $JVMOPTS
echo $CLASSPATH
set -x
$JAVA_HOME/bin/java.exe -cp "$CLASSPATH" $JVMOPTS org.python.util.jython $*

 
And, msys2winpath is a simple script:

#!/bin/bash --login
if [ $# -eq 0 ]
then
 pwd -W
else
 cd "$1"
 pwd -W
fi

Now, create a shell script, I called it 'dopylocalinstall.sh', to make it easier to setup the local site-packages.

#!/bin/bash

CPWD=`pwd -W | sed 's/\//\\\\/g'`
if [ ! -d 'dist/Lib/site-packages' ]
then
 mkdir -p dist/Lib/site-packages
fi
export PYTHONPATH="$CPWD\\dist\\Lib\\site-packages"
export JYTHONPATH="$PYTHONPATH"
pycmd="runjython.sh"
#pycmd="python"
$pycmd setup.py build
$pycmd setup.py install --prefix="$CPWD\\dist"

Now, change to the xlrd unzipped folder and run the dopylocalinstall.sh
This should have now created a dist folder with Lib and site-packages within.

The contenets of the site-packages/xlrd can now be copied into a folder in Eclipse and you should be able to use the package in your projects.

Regex to remove version information from RPM strings

Overview

Removing version strings helps compare packages between two installations using simple diff/winmerge like tools.
Later using unix commands such as 'comm' we can obtain just what is unique in one install and patch the other install.

The Regex

The following regex works within vim/vi editors:

-[-0-9]\+\.[-0-9A-z\.]*\.

Effect of using the regex is as shown below:

Note the highlighted strings.

Effect of using a global replace is as shown below:

VI/VIM global search and replace with a single 'dot':

:g/-[-0-9]\+\.[-0-9A-z\.]*\./s//\.

Replaced result:

Using 'comm' to generate unique packages from one or the other system:

• Sort both the files that you wish to diff.
  The following command will sort the file in-place
  
  sort -o vm1rpm.txt vm1rpm.txt

• Run the comm command as below:
  comm -23 vm1rpm.txt vm2.rpm.txt

This will generate a list of only what is in vm1rpm.txt (I am not providing output as it is reasonably obvious).