by lunarg on October 1st 2007, at 22:06
6 pages

Installing Gentoo

The very foundation of our system will be the linux system. It doesn't really matter which distribution you plan to use, but you should favor a system that's very optimized for the whole media thing. I myself, use Gentoo for just about everything (accept servers, which are Debian). Since I'm using an Athlon64 X2, and I'm a bit adventurous, I tried out the 64bit version, which works pretty well with MythTV (a bit to my surprise, though).

For the installation, I'll be using the latest release, which is 2007.0, and roughly followed the instructions from the Gentoo Handbook. Of course, where needed, I adapted things a bit.
However, do note that I don't use the media of Gentoo itself. Because of my requirement of reiser4, I had to use a LiveCD with reiser4 support. The one I use virtually everytime I have to do a Gentoo install is RiP-Linux. Once booted, I can download the stage tarballs and portage snapshot from any Gentoo mirror (as explained in the handbook). If you're not comfortable with this method, use reiserfs wherever I used reiser4, and you'll be just fine.

Disk layout

Partitioning

My partition layout is very specific to my needs, so it's highly unlikely you will be following this to the letter. For partitioning, and according to the handbook, I use fdisk.
An fdisk -l reveals the following:

Disk /dev/sda: 251.0 GB, 251000193024 bytes
255 heads, 63 sectors/track, 30515 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1           3       24066   83  Linux
/dev/sda2   *           4         993     7952175    7  HPFS/NTFS
/dev/sda3             994        2211     9783585   83  Linux
/dev/sda4            2212       30515   227351880    5  Extended
/dev/sda5            2212        2274      506016   82  Linux swap / Solaris
/dev/sda6            2275       30515   226845801   8e  Linux LVM

A bit more explanation would be neat:

  1. Following the handbook, I have a seperate boot partition where the bootloader (GRUB) and the kernel resides.
  2. My second partition is an NTFS partition for my Windows XP system, which I use strictly for printing, scanning, and managing my PDA.
  3. Third partition is the root partition. It's limited in size, because it will only have to hold the software.
  4. Naturally, I also have a swap partition. It will most likely never use more than a few MB of it, but since we're going for optimalization, a swap partition is always a good idea. If you're planning on using flash as filesystem, don't use swap though.
  5. The remainder of the space will be made a PV for use with LVM. Notice the special partition type flag for that.

Configuring the LVM

The reason for using LVM is the ease of configuration, and moreover, if we ever have to change volume sizes, it will be far more easier. For more information about LVMs, read Manual configuration of LVM.
In my case, I need two volumes: one for the MythTV storage (you probably want this too), and one for miscellaneous data (music, transcoded video, etc.).

First, format our PV so we can use it for LVM:

# pvcreate /dev/sda6

After that, create our VG, which I conveniently called store:

# vgcreate store /dev/sda6

And next up, I create two LVs, respectively tv and data:

# lvcreate -n tv -l 24576 store
# lvcreate -n data1 -l 30806 store

The storage for MythTV is 96GB (= 24576 extents) in size. The remaining space becomes our miscellaneous data partition.
A note about the size of MythTV though. I took a rough estimate of about 1.5 to 2GB for every hour of MPEG2. My size of 96GB would thus provide me with about 48 hours of video. Note that most likely, you'll have to substract a few hours of it, overhead and all that.
I'll probably won't be getting near the limit, though. I'll wipe most recordings after I watched them, and things I do want to keep will be transcoded to MPEG4 and moved to the data volume.

Filesystems

Once partitioning is completed, it's time to move on to some mkfsing. Based upon the usage of each partition/volume, I select (in my opinion) the best filesystem.
For more information though, check out About linux filesystems.

  • For the boot partition, I use ext2 with sparse_super, and no reserved space (see below for the commandline).
  • For root I prefer high speed reading; writing is less important here because the only time we'll be writing is when we update the system, or install new software. Therefore, I choose reiser4 which beats virtually all filesystems on that account. The fact that it's experimental is of no consequence, precisely because we'll only be reading from it for most of the time.
  • The swap partition is a swap partition (logical, no?)

I mkfs those partitions with following parameters:

# mke2fs -O sparse_super -m 0 /dev/sda1
# mkfs.reiser4 /dev/sda3
# mkswap /dev/sda5

For the LVs, I use XFS. Using XFS (or JFS) for the MythTV storage area is highly recommended: XFS and JFS are the only two filesystems that have excellent performance with the large MPEG2 files that are generated by MythTV. Other filesystems are more sluggish and slow when it comes to handling these large files.

From the MythTV wiki:

Quote
JFS was originally developed by IBM for their AIX operating system, and was later donated to Linux. JFS is incredibly good at dealing with the huge files that MythTV generates, and can delete pretty much any file in under a second (ext3 can take as long as 15 seconds to delete really big files). JFS is a very good file system to use for storing your videos on, and it is very conservative with CPU usage.

XFS is another "foreign" filesystem, developed by SGI for their IRIX operating system, and once again donated to Linux. Like JFS, it is exceptionally good at dealing with large files, and has the highest throughput of any Linux filesystem, albeit at a higher CPU loading. XFS also makes an excellent choice as storage for your movie files. (Note that XFS filesystems can be grown, but not shrunk, at the present time; this can occasionally be problematic. Note also that file system cleanings are forced using xfs_repair, not fsck; if you are going to use XFS, and Bay Link recommends that you do, read about it first.)

I don't have a whole lot of experience with JFS (and frankly, I've read other stuff about JFS which wasn't all that good), so I'm using XFS. The fact that I can't shrink the filesystem is something I don't mind. I won't be shrinking the filesystem anyway; if any, it would be growing. If you do mind it, don't use XFS, but use JFS instead.
If you don't want to use JFS or XFS, you have to enable delete files slowly in MythTV. More about that later.

XFS is a complex filesystem, with many parameters at creation time. I followed the instructions of this forum thread to create the filesystem:

# mkfs.xfs -l internal,size=128m -d agcount=2 /dev/store/tv

Take particular note to the agcount parameter. According to the thread, this parameter should be set to 2x the number of CPUs you have. Thus, if you have a dual-core CPU, the agcount would be 4.

My other data partition will contain mostly media files, with all files being at least over a few MBs. For XFS, these already count as fairly large files, so for this volume, I'll be using XFS as well:

# mkfs.xfs -l internal,size=128m -d agcount=2 /dev/store/data

USE flags, and other options from make.conf (Gentoo only)

Aside of the USE flags, the make.conf is pretty subjective to the hardware you'll be using.

Notice
I've added my make.conf. I suggest not to use it as-is, but rather keep it as a template and adjust properties. Do note that it's not a whole lot different from the default make.conf, so it's up to you to decide whether this is useful.

My USE-flags

My USE-flags are these:

Quote
USE="-gnome -qt4 -esd 3dnow 3dnowext 7zip X a52 aac aalib acpi aiglx akode alsa apm -arts asf audiofile browserplugin bzip2 cdparanoia cdr cups curl divx divx4linux dri dts dv dvb dvd dvdnav dvdr dvdread dxr3 encode ffmpeg -firebird flac ftp gd gif gimpprint gphoto2 gs gtk gtk2 hal ieee1394 imap imlib ipod iproute2 ivtv java jbig joystick jpeg jpeg2k -kde lcd lcms lirc mad matroska mmx mmxext modplug mp2 mp3 mp4 mpeg mplayer multiuser musepack mysql ncurses network nfs njb nptl nsplugin ogg openal opengl oss pdf pdflibs png qt qt3 quicktime rar rdesktop real realmedia samba sasl scanner sdl srt sse subtitles svg threads tiff truetype usb v4l v4l2 vcd vidix visualization vorbis wavpack wifi win32codecs wma wmf wxgtk1 wxwindows x264 xcomposite xine xml xmms xpm xv xvid xvmc zlib"

I know it's quite a list, but the fact is that, in comparison with a few releases back, more and more functions (like support for codecs and such) are becoming optional, instead of included by default. The flags you see here provide a fairly completed support, in particular to the MPlayer and Xine-lib backends (used by MythDVD, MythVideo, and probably some others). Also, note that some of these flags occur more than once (e.g. support for RealMedia is specified through real and realmedia): these are either relics of how flags were named in the past, or because another application uses another name to reference to the same thing.

The flags marked red are specifically excluded, because I really don't need it. Take particular note to esd and arts: I don't use a sound daemon because I'm using an Audigy (which has hardware mixing). If you don't have a card with hardware mixing and hardware multi-channel support, you may need to enable one of these. In that case, I suggest to use esd and not arts, as the latter was designed for KDE.
The flags in green are architectural flags. The one I've added are for my AMD AthlonXP 2100+: I've added support for 3DNow, Extended 3DNow, MMX, Extended MMX, and SSE. Depending on your CPU, additional flags may be possible:

  • sse2: adds SSE2 support;
  • pni and/or sse3: adds SSE3 support (PNI is used with AMD CPUs that support SSE3);
  • ssse3: adds SSSE3 support (note that this is not the same as SSE3).

The flags in blue are codec flags. These add support for various codecs to MythTV and other backends. Take particular note to ffmpeg: it is a library which holds support for quite a few codecs, like MPEG2 and MPEG4 (which are probably the important ones). If you're running 32bit, you can also add win32codecs to your flags: it is a bunch of codecs for Windows. In particular the MPlayer backend (in MythVideo) has the ability to use these libraries to playback files (similar to ndiswrapper being able to use NDIS based drivers for networking).
The flags marked in yellow are flags that add device based support to the system. Things like ivtv (support for Hauppauge TV tuners), but also things like cdr and dvdr (CD and DVD writing support), dvd (DVD reading support), and vcd (support for (S)VCD).
The flags that are in magenta are flags which you should have enabled, regardless of the system. These flags add support for Xv (video overlay support for hardware-accelerated video playback), alsa (the elementary sound system), etc.
If you have an nVIDIA Geforce card, a VIA card, or an Intel card in your system (ATI is currently not supported, according to wikipedia), you can also use the XvMC extension.
Finally, also take note to some of the other flags:

  • srt and subtitles: add support for subtitles and the so-called SRT format to MPlayer and Xine backends.

My CFLAGS and CXXFLAGS

My CFLAGS and CXXFLAGS are these:

Quote
CFLAGS="-march=k8 -mtune=k8 -msse3 -O3 -pipe -fomit-frame-pointer"
CXXFLAGS="${CFLAGS}"

These flags make sure the compiled code (= installed software) is optimized to my system (and in particular CPU). The flags are pretty much the same for 32bit and 64bit, because the architecture flag will determine the resulting code. Therefore, using k8 will still optimize for Athlon64, but the code will be 32bit.

Notice

More information about CFLAGS, CXXFLAGS and compiler optimalizations can be found in the Gentoo Handbook, and here:

Audio and video support flags

For X (and various other applications), there are additional parameters that have to be set in order for the application to have support for your hardware.

AUDIO_CARDS

The AUDIO_CARDS variable tells which audio devices have to be build. Mine are:

Quote
AUDIO_CARDS="emu10k1 hda_intel"

Note that the flag emu10k1 is an alternate name for SB Live, Audigy, and Audigy 2 cards.

Notice
The AUDIO_CARDS flag may be obsolete, starting from kernel 2.6.
If anyone can confirm this, please let me know.
VIDEO_CARDS

The VIDEO_CARDS flag tells the system to build support for these video cards. This is in particular important for X where adding these flags will automatically merge the proper drivers. MythTV also seem to have specific support for this parameter, though I'm not sure what it does.
Since I have an ATI Radeon, my flags are:

Quote
VIDEO_CARDS="vga vesa nvidia ati radeon fglrx v4l nvidia"

I've added support for standard VGA, VESA, and support for video4linux (required for TV tuners). I also added support for my primary video card (nVIDIA), and if ever needed, I also added the open-source drivers for ATI and Radeon cards, the proprietary (closed-source) drivers for ATI Radeon and FireGL cards.
More information about which hardware these flags depict, can be found at The Simple Guide to a Media PC (Installing and configuring X).

INPUT_DEVICES

This flag tells which input devices to build support for in X. You probably want to add keyboard and mouse. Optionally, you can add support for joysticks (includes gamepad), drawing tablets, touchscreens, etc.

My flags are:

Quote
INPUT_DEVICES="keyboard mouse joystick"

More information about which hardware these flags depict, can be found at The Simple Guide to a Media PC (Installing and configuring X).

Additional, miscellaneous flags

Additional flags may be set to optimize the installation process a bit. Note that more info about this can be found in the Gentoo Handbook.

  • GENTOO_MIRRORS: this is a space-seperated list of URLs where the sources can be downloaded from. It's recommended to add servers which are fast and close to your location.
  • MAKEOPTS: the number of parallel builds during a merge. Set this to 2x the number of CPUs (as explained in the Gentoo Handbook).
  • SYNC: set the URL of a Portage server. Set it to a fast server, close to your location, or use a rotation server (like I did).

In my make.conf, the parameters mentioned above, are set to:

Quote
GENTOO_MIRRORS="http://ftp.snt.utwente.nl/pub/os/linux/gentoo/ http://gentoo.virginmedia.com/"
MAKEOPTS="-j2"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"

Setting the local Portage Overlay

Since we'll be installing a custom kernel (and possibly some other custom stuff) for which there's no ebuild in the official portage tree, we'll have to create a local Portage Overlay. For this, we have to set the PORTDIR_OVERLAY parameter in make.conf. I prefer to keep the overlay in /usr/local/portage, so the parameter becomes:

Quote
PORTDIR_OVERLAY=/usr/local/portage

Don't forget to create the folder, or you'll be hounded each time you do a merge:

# mkdir -p /usr/local/portage && chown portage:portage /usr/local/portage

Locales

As explained in the Gentoo Handbook, we need to properly set up for foreign languages. For this, we have to adjust the locales, which is done in two little steps.

First adjust the file /etc/locale.gen. I've added these locales (for Belgium, Europe):

nl_BE@euro ISO-8859-15
nl_BE@euro ISO-8859-1
en_US ISO-8859-15
en_US ISO-8859-1
en_US.UTF-8 UTF-8

To apply them, run locale-gen (in your installation environment of course).

Kernel installation and configuration

Because of the need for reiser4, but not wanting to give up on the Gentoo patchset either, I roughly followed the guide here to get the right kernel sources. I chose to use the latest stable Gentoo sources, which, at the time of writing (and installing), is 2.6.22-gentoo-r5. A plus for this version of the kernel (though higher would also work), is the presence of the IVTV driver in the kernel. While I won't be using the built-in version (the ebuild provides more up-to-date versions), selecting the driver as a module (this is important), will also enable the many dependencies of the driver (which makes installation reeaally easy).

For more detailed configuration, read this page The Simple Guide to a Media PC (Installing and configuring X) (which you probably have to do anyway).

Install the ebuild

First, make sure your portage overlay is properly created and configured in your make.conf. Then, create the necessary directory structures:

# mkdir -p /usr/local/portage/sys-kernel/reiser4-gentoo-sources

Next up, download the file reiser4-gentoo-sources-2.6.22-r5.ebuild, and place it in the directory you just created. After that, cd to the dir and run:

# cd /usr/local/portage/sys-kernel/reiser4-gentoo-sources
# ebuild reiser4-gentoo-sources-2.6.22-r5.ebuild digest

This will download the sources and build the MD5 sums and digests for the ebuild. Once this is completed, you will be able to merge the sources like you would do with any kernel:

ACCEPT_KEYWORDS="~x86" emerge -v =reiser4-gentoo-sources-2.6.22-r5

Consult the Gentoo Handbook for more information about installing the kernel, and for general configuration instructions. The ACCEPT_KEYWORDS is required because the ebuild requires it (it is in testing, because it's not a very thourough tested setup).

Continuing the installation

After configuring the kernel (and some of the other hardware), continue with your installation like you normally would (for Gentoo, this means roughly follow the handbook).