My third book is released! Learn what you'll need to know in order to become an embedded engineer.

Check out my second book; learn practical stuff about building robots and control systems around Linux PCs and the Atmel AVR.

My first book gives you all the intro you need on developing 32-bit embedded systems on a hobbyist budget.

# Installing Linux on Geode-based Single-Board Computers

Thursday 9/11/2003 - This document has been revised, cleaned-up and extended. Most of this material will eventually appear in my second book, which you can expect to see around the middle of 2004. Feel free to buy my first embedded systems book, while you wait :)

My daytime employer builds embedded Linux appliances based around the Advantech PCM-5820-G0xx 3.5" miniature SBC, and I've found several issues with it that require tweaking. Over the last two weeks, in fact, I have spent almost all my time reverse-engineering the BIOS and creating hybrid firmware to fix some ongoing issues with it.

Before continuing with the meat of this information, let me pause to mention that several readers have commented on the relevance, or lack thereof, of this document to other Geode SBCs. By and large, everything you read here is relevant to most Geode boards with an Award BIOS, which includes the following specific products (all of which are biscuit PCs in the 3.5" or 5.25" form factors):

- Acrosser AR-B-1551, AR-B1552 and AR-B1565.
- Advantech PCM-5820, PCM-5822, PCM-5823, PCM-5824 and PCM-5825.
- e-valuetech (BCM) EBC-3410 and EBC-5410.
- Wafer-5820 and Wafer-5823 boards (it isn't clear to me who actually makes these).

As of the time of writing, we are using Linux kernel 2.4.21. I have heard positive results from people using the 2.6 kernel betas, but we don't wish to ride the bleeding edge in our application, so we will defer testing on 2.6 until it is primetime. Our distribution has very heavy "eye candy" requirements, so it is based on Red Hat 8.0, with significant customizations. In fact, there is not much Red Hat left.

Our chosen SBC is based around a 300MHz National Semiconductor Geode processor with the CS5530A companion IC providing interface to CRT SVGA monitors and/or TFT LCD panels. It also features an IrDA port, two OHCI-compliant USB ports, PS/2 keyboard/mouse port, one parallel port, two RS-232 ports (one of which can be configured for RS-485), floppy controller, single IDE bus, CompactFlash slot residing on the second IDE bus, 10/100 Ethernet via a Realtek RTL8139 and analog audio I/O. Some of the other devices listed above differ in peripherals; dual or more LANs, TV-out, etc, but they are all the same core hardware and share a lot of the same quirks.

I'll get some of the on-board peripherals out of the way immediately by stating that they work without special modifications:

- The PS/2, parallel and serial ports are vanilla and work normally.
- The CS5530's PCI bridge and the I/O chip are supported by kernel 2.4.x (and probably earlier versions).
- The USB interface is a standard OHCI controller and works well with the 2.4.x kernel tree. To date I have successfully used it with Atmel-based 802.11b wireless LAN adapters, mice, keyboards, USB "pen disks", ST-based "pencam" webcams, and various flash media card-reading devices.
- The Realtek Ethernet chip is supported using the "8139too" module. Note that you wil probably get a warning about unexpected IRQ steering when loading the module; this can safely be ignored. Also note that I have been unable to get this chip to work in full-duplex mode, at least in a Linux environment.

The first problem you may encounter if you install your Linux distribution on another computer is that you'll get an i686 kernel installed, and it will not boot. It seems to have problems finding a Pentium-style TSC when booting on a Geode. GX1 does in fact have a TSC at MSR 10, but the i686 kernel installed on an Intel/Via/AMD on 7.x's install CD will fail to find it, and will panic. To rectify this, use the kernel-x.x.x-i586.config configuration as a starting point, then make config and make bzImage to build your kernel.

Geode is descended from the Cyrix MediaGX processor. It is probably most directly comparable with a Pentium-MMX, but it is by no means 100% identical. Overall performance of a 300MHz Geode is very roughly equivalent to a 200MHz Pentium-MMX, but any meaningful comparison must be made on a function-by-function basis. In particular, Geode's performance is significantly limited by the fact that a lot of the functionality, particularly video, is emulated in what NS calls "Virtual System Architecture"; special firmware accessed in System Management mode. But, more on that topic later.

The audio feature built into the CS5530 is supported using the SoundBlaster 16 driver with the "mpu_io=0" parameter. Note that you must disable ISA Plug'n'Play in the kernel. Recent(ish - i.e. contemporaneous Red Hat 7.x and later) versions of sndconfig will detect the 5530 as "CS5530 [Kahlua]", but they will inform you that the chip is disabled. On some boards (perhaps those using National's XPressROM) it may be possible for you to fix this error with CMOS settings and get the native driver loaded, but this doesn't seem to be possible on the PCM-5820. Just use the part in SB16 emulation mode.

Now for the longest part of this discussion - how to get XFree86 running. Our application is graphics-intensive. For various reasons, it is advantageous for us to implement our embedded app as an X client application, but getting X to work on this board is quite a chore. I'll skip the story of every combination I tested, and just give you the answers here.

Before going any further, note that there is a special issue with the BIOS on this board, which affects all the Geode boards I have tested. This board supports two displays directly (CRT and LCD). The BIOS allows you to select the LCD resolution - 640x480, 800x600 or 1024x768. Even if you have the LCD disabled in BIOS and only use the CRT outputs, you MUST set the LCD resolution option in CMOS setup at least as high as the highest resolution you want to use with X. Otherwise the system will hang with a black screen when you attempt to enter graphics mode. Also ensure that you allocate enough video RAM (in CMOS setup) to accommodate your desired video mode.

Now, using out-of-box Linux components from major distributions, there are basically three ways to get XFree86 working on the CS5530:

Method 1. Use the framebuffer console and the XF86_FBDev server.

This method might not work on all Geode-based systems because it relies on the presence of a VESA BIOS extension. It works on the PCM-5820 (but note the important drawbacks below). For testing purposes, you can simply type "video=vesa vga=xxx" at the LILO boot: prompt, or make whatever changes are appropriate to your bootloader if you're not using LILO. This will allow you to check various video modes (xxx = video mode; look in Documentation/vb/vesafb.txt inside your Linux kernel source directory for more information on this). Once you've established which mode works best for you, adding these lines to the appropriate paragraph in lilo.conf will make your choice permanent:

`vga=xxx
append="video=vesa"`

Note that this feature requires kernel framebuffer console support. Assuming you have everything working for the framebuffer console, simply install the XF86_FBDev server and link it to /usr/X11R6/bin/X and you'll be set. An example XF86Config file for framebuffer operation is below. Using this file, X will start up in the resolution you selected at boot time.

`Section "Files"
` `RgbPath "/usr/X11R6/lib/X11/rgb"
` `FontPath "unix/:7100"
` `EndSection`

`Section "ServerFlags"
` `EndSection`

`Section "Keyboard"
` `Protocol "Standard"
` `AutoRepeat 500 5
` `LeftAlt Meta
` `RightAlt Meta
` `ScrollLock Compose
` `RightCtl Control
` `XkbKeycodes "xfree86"
` `XkbTypes "default"
` `XkbCompat "default"
` `XkbSymbols "us(pc101)"
` `XkbGeometry "pc"
` `XkbRules "xfree86"
` `XkbModel "pc101"
` `XkbLayout "us"
` `EndSection`

`Section "Pointer"
` `Protocol "PS/2"
` `Device "/dev/mouse"
` `Emulate3Buttons
` `Emulate3Timeout 50
` `EndSection`

`Section "Monitor"
` `Identifier "Panel"
` `VendorName "Unknown"
` `ModelName "Unknown"
` `HorizSync 31-90
` `VertRefresh 40-160
` `# Modelines aren't actually used
` `Modeline "640x480" 31.5 640 656 720 864 480 488 491 521
` `EndSection`

`Section "Device"
` `Identifier "Geode"
` `VendorName "Unknown"
` `BoardName "Unknown"
` `VideoRam 4096
` `EndSection`

`Section "Screen"
` `Driver "FBDev"
` `Device "Geode"
` `Monitor "Panel"
` `Subsection "Display"
` `Depth 16
` `Modes "default"
` `ViewPort 0 0
` `EndSubsection
` `EndSection`

There are two major disadvantages to this setup (and some minor ones). The two big problems are (a) it's slow, because you have no access to any accelerated hardware features, and (b) you can't switch resolutions on the fly. In our application, we want to run at 1024x768 for still images, but use lower resolutions for full-screen motion video (MPEG-1 playback) since the source material is lo-res anyway and stretching it out to fill a large screen is a complete waste of CPU bandwidth.

Important note: If you are using the VESA framebuffer console, it is not possible to use non-FBDev X servers, nor graphics libraries like SVGAlib. The system will go into an undefined video mode as soon as you attempt to switch modes away from the boot-time default.

Method 2. Use the XF86_SVGA server from version 3.x.

The best match I have found so far is the XF86_SVGA server, version 3.3.6a. If you install Red Hat 7.2 and select Geode, this server will be installed but it will _NOT_ be enabled! The Red Hat 7.2 install process by default links /etc/X11R6/bin/X to /etc/X11R6/bin/XFree86, which is the XFree86 4.1.0 "mega-wrapper"; it attempts to install the Cyrix MediaGX driver, which does not work with Geode.

Note that the 3.3.6a server I mention above is NOT the same version Advantech supplies on their driver CD-ROM. The Advantech version seems to calculate dot clocks and other timings quite bizarrely, and I've had trouble getting properly centered video with their server. Unfortunately, there's a catch-22 lurking here, which I describe below.

One essential point to note is that there is a bug in the CS5530 which can make the SVGA server extremely unstable. You can work around this bug by specifying an odd virtual screen size using the Virtual x y keyword (I use 1024 769 because our system normally runs at 1024x768. If you are using a lower screen resolution like 640x480 and want to conserve video memory requirements, use an appropriately odd-sized virtual buffer such as 640x481). The cause of this bug is a combination of XF86_SVGA's behavior and a quirk of the CS5530. The Geode system relies heavily on I/O traps and "faked" hardware emulation for some of its functionality, particularly video. In the "standard" resolutions, XF86_SVGA attempts to enable display compression, which causes problems. In the best case, you will get some garbage on the display; in the worst case, the system will lock hard and require a hard reset or power-cycle. Specifying the strange virtual resolution implicitly disables the compressed display feature and works around the problem. This workaround may or may not be ideal for you depending on your system setup; in our case, the system has no pointing device so it's not possible for the user to scroll the display window and see the extra "phantom" scanline.

Using XF86_SVGA you will have access to some hardware acceleration features such as a hardware mouse cursor. More importantly, you will be able to change resolutions on the fly. Here is a suitable XF86Config file for the three standard resolutions, 640x480, 800x600 and 1024x768. It is possible to get the low-res video modes working also, but only with the Advantech-supplied server. (Argh! This is the catch-22 I mentioned) If you don't need low-res I suggest you stick to the Red Hat-supplied server. It's much easier to get stable, easily- centered video with this server.

`Section "Files"
` `RgbPath "/usr/X11R6/lib/X11/rgb"
` `FontPath "unix/:7100"
` `EndSection`

`Section "ServerFlags"
` `EndSection`

`Section "Keyboard"
` `Protocol "Standard"
` `AutoRepeat 500 5
` `LeftAlt Meta
` `RightAlt Meta
` `ScrollLock Compose
` `RightCtl Control
` `XkbKeycodes "xfree86"
` `XkbTypes "default"
` `XkbCompat "default"
` `XkbSymbols "us(pc101)"
` `XkbGeometry "pc"
` `XkbRules "xfree86"
` `XkbModel "pc101"
` `XkbLayout "us"
` `EndSection`

`Section "Pointer"
` `Protocol "PS/2"
` `Device "/dev/mouse"
` `Emulate3Buttons
` `Emulate3Timeout 50
` `EndSection`

`Section "Monitor"
` `Identifier "Panel"
` `VendorName "Unknown"
` `ModelName "Unknown"
` `HorizSync 31-90
` `VertRefresh 40-160
` `# 640x480 @ 72 Hz, 36.5 kHz hsync
` `Modeline "640x480" 31.5 640 656 720 864 480 488 491 521
` `# 800x600 @ 72 Hz, 48.0 kHz hsync
` `Modeline "800x600" 50 800 816 976 1040 600 637 643 666 +hsync +vsync
` `# 1024x768 @ 70 Hz, 56.5 kHz hsync
` `Modeline "1024x768" 75 1024 1040 1184 1328 768 771 777 806 -hsync -vsync
` `EndSection`

`Section "Device"
` `Identifier "Geode"
` `VendorName "Unknown"
` `BoardName "Unknown"
` `VideoRam 4096
` `EndSection`

`Section "Screen"
` `Driver "svga"
` `Device "Geode"
` `Monitor "Panel"
` `DefaultColorDepth 16
` `Subsection "Display"
` `Depth 16
` `Modes "1024x768" "800x600" "640x480"
` `Virtual 1024 769
` `EndSubsection
` `EndSection`

One final note on XFree86 3.x - National has supplied numerous subtly different versions of the 3.x SVGA server for Geode; some with sourcecode, some as binary-only. Since 3.x is officially dead, I have not experimented with all of these. If you're desperate to fix some particular issue, feel free to go the trial-and-error route!

Method 3. Use XFree86 4.x

Modern distributions of Linux, such as Red Hat 9.0, ship with XFree86 4.x. However, the installation process doesn't correctly detect the GX/5530 chipset; it installs the Cyrix MediaGX driver, again. To get the system working, use this XF86Config, which uses the nsc_drv.o driver. Note: This XF86Config has only been tested on XFree86 4.2.99.3 beta and the current release, 4.3.0.

`Section "ServerLayout"`
`Identifier "XFree86 Configured"`
`Screen 0 "Screen0" 0 0`
`InputDevice "Mouse0" "CorePointer"`
`InputDevice "Keyboard0" "CoreKeyboard"`
`EndSection`
`Section "Files"`
`RgbPath "/usr/X11R6/lib/X11/rgb"`
`ModulePath "/usr/X11R6/lib/modules"`
`FontPath "/usr/X11R6/lib/X11/fonts/misc/"`
`FontPath "/usr/X11R6/lib/X11/fonts/Speedo/"`
`FontPath "/usr/X11R6/lib/X11/fonts/Type1/"`
`FontPath "/usr/X11R6/lib/X11/fonts/CID/"`
`FontPath "/usr/X11R6/lib/X11/fonts/75dpi/"`
`FontPath "/usr/X11R6/lib/X11/fonts/100dpi/"`
`EndSection`
`Section "Module"`
`Load "extmod"`
`Load "dbe"`
`Load "dri"`
`Load "glx"`
`Load "record"`
`Load "xtrap"`
`Load "speedo"`
`Load "type1"`
`EndSection`
`Section "InputDevice"`
`Identifier "Keyboard0"`
`Driver "keyboard"`
`EndSection`
`Section "InputDevice"`
`Identifier "Mouse0"`
`Driver "mouse"`
`Option "Protocol" "auto"`
`Option "Device" "/dev/mouse"`
`EndSection`
`Section "Monitor"`
`Identifier "Monitor0"`
`VendorName "Monitor Vendor"`
`ModelName "Monitor Model"`
`HorizSync 31.5 - 50.0`
`VertRefresh 50.0 - 75.0`
`EndSection`
`Section "Device"`
`#Option "SWcursor" # [bool]`
`#Option "HWcursor" # [bool]`
`#Option "NoCompression" # [bool]`
`#Option "NoAccel" # [bool]`
`#Option "TV" # [str]`
`#Option "TV_Output" # [str]`
`#Option "TVOverscan" # [str]`
`#Option "ShadowFB" # [bool]`
`#Option "Rotate" # [str]`
`#Option "FlatPanel" # [bool]`
`#Option "ColorKey" # i`
`#Option "OSMImageBuffers" # i`
`Identifier "Card0"`
`Driver "nsc"
Option "NoAccel" "True"`
`VendorName "Cyrix Corporation"`
`BoardName "5530 Video [Kahlua]"`
`BusID "PCI:0:18:4"`
`EndSection`
`Section "Screen"`
`Identifier "Screen0"`
`Device "Card0"`
`Monitor "Monitor0"`
`DefaultDepth 16`
`SubSection "Display"`
`Depth 16`
`Modes "1024x768" "800x600" "640x480" "400x300" "320x240" "320x200"`
`EndSubSection`
`EndSection```

There are many good reasons to switch to using XFree86 4 if you can, including:

- Xv and DGA accelerated graphics support.
- An end to the need to specify weird virtual screen sizes (the Option DisplayCompression setting achieves this).
- More flexible server options.
- The default video modes have "friendlier" syncrates than the 3.x server.
- Translucent mouse cursors.

Now, that's only half the picture. Due to a bug, the XFree86.org official release code does not support scandoubled modes (e.g. 320x240) on Geode. I have generated a patch for this bug; you can download a precompiled version of the patched nsc_drv.o here. If you are interested in the details of the patch, for example if you wish to apply it to a newer sourcecode release, please refer to my posting on the topic, at devel@xfree86.org.

Using my patched driver enables 400x300, 320x240 and 320x200 graphics modes, which are useful if you need to play VCD content on a Geode.

- The NSC driver does not, apparently, fully support autoprobing. This means that XFree86 -configure will not generate a completely valid XFree86Config file (it will work, but it won't give you a full range of resolutions and will require some manual tweaking).
- It appears that the Geode, or at least the X driver for it, doesn't support DDC so the monitor syncrates in an auto-generated XFree86Config will be arbitrary.
- If you're running with display compression enabled, you may see minor video glitches onscreen, particularly if your application writes directly to display memory. This phenomenon appears to be a momentary loss of sync, like a skipped v-sync pulse, and it is yet another of the problems caused by the ridiculous "video compression" feature of the CS5530. The line:
  `Option "NoCompression" "True"
  in the Device stanza in your XFree86Config file fixes this.`
- UI rotation is supported using the Option "Rotate" "CW" or Option "Rotate" "CCW" switches. However, these will fail catastrophically unless you also use Option "ShadowFB" "true". This has something of a performance downside and I don't recommend it.
- Flat-panel support appears to be broken, at least on the Advantech board we are using. If you need to use a direct-connect parallel or LVDS LCD, then for the time being you should use the VESA driver. Neither the vanilla XFree86 driver nor my patched driver will work correctly on any of the LCDs we have tested. The National Semiconductor server does work, but it doesn't support scandoubled video modes. (Note, by the way, that you need to specify Option FlatPanel True if you are using XFree86 4.x with an LCD system).
- The nsc_drv.o driver does not correctly save/restore the entire video subsystem state with some BIOS versions. This makes it impossible to switch from X to a different virtual console. It also means that the system will lose sync and go into an undisplayable video mode if you exit X. There is no workaround for this issue at this time; use XFree86 3.x if this is a problem for you. This problem is known to exist on the Wafer-582x products and the e-valuetech EBC-3410. It does not affect the EBC-5410.

A final note: Of BIOSes and Quirks

Recently, we discovered that our PCM-5820-based systems would intermittently fail to go into X properly, mostly after a warm boot. There was no pattern to this behavior; the symptom would happen once. The system would actually be running - we could hear audio features playing, and communicate with the device over Ethernet - but the CS5530 was putting out very odd syncrates. Further investigation linked this problem to the currently-shipping BIOS (as of September, 2003) - version 1.23. Flashing the boards down to version 1.14 fixed the problem. Unfortunately, coincident with BIOS release 1.23, Advantech also changed the AC97 audio codec on the board from an Analog Devices chip to a Realtek ALC201 part. BIOS code 1.22 and earlier will play back audio approximately 10% too fast on new hardware. BIOS 1.22 also shows the video hanging problem. The last BIOS that doesn't have the video problem is 1.21, which can't be used on current hardware because of the audio issue.

This was the start of panic mode. Our mechanical design is based around the PCM-5820, so we couldn't quickly adapt to a new board. So, I experimented with numerous "alien" BIOSes from other vendors. All of them either had the audio problem, or had the intermittent hanging problem, or didn't work correctly in some other way (e.g. no network support, USB ports not working, &c). Clearly, the shotgun approach wasn't going to work. So, it was time to hack a little deeper.

The first step is to extract various parts of the BIOS code to compare them. The PCM-5820 uses Award BIOS 4.51G. This is comprised of a tiny decompressor/bootloader module, and an internal LZH-compressed filesystem containing various modules. One tool to manipulate this internal filesystem is AwardMod (a GNU tool); you can download it here. AwardMod (and the official Award utilities) work with raw dumps of the BIOS ROM file; if you don't have a copy of your BIOS, you can write one out using the /SY command-line switch to the AWDFLASH flash-update utility (search for this utility with Google or download it from BiosMods.com).

Splitting out the BIOSes into their component modules reveals that they contain the following parts:

- Main system BIOS
- Video BIOS extension
- Award BIOS extension (PnP features, probably)
- Geode VSA emulation code (not normally accessible; this code lies in the SMM address space)
- PXE ROM
- LCD parameters
- EPA logo

The EPA logo, LCD parameters, Award PnP extension and PXE ROM are identical across all affected versions, so they're not part of our problem. Using AwardMod, I created various hybrid BIOSes with combinations of modules from different Advantech releases. I quickly determined that both the video problem and the audio fix reside in the VSA code.

My attack on this problem was three-pronged. Success along any of these routes would satisfy us:

- Hack the 1.21 VSA code to play audio at the correct speed on the ALC201 codec.
- Borrow VSA code from a different board to fix the problem, or
- Find out what, exactly, is being munged by the bad VSA code and hack the XFree86 driver to disable the culprit feature.

In pursuit of the first attack, I started to reverse-engineer the VSA code from 1.21 and 1.23 side by side. I quickly determined that there are major differences between the two; it's not just a timing constant or two that needs to be hacked. Due to the large number of Geode-specific instructions and the fact that I would basically need to create a full reverse-engineered disassembly of the entire 128K of VSA code, I decided that I would never be able to finish this inside a useful timeframe.

The second attack method has yielded an apparently working BIOS. It is comprised of Advantech's 1.21 code, with VSA code from a different board. This has also fixed another problem with Advantech's code - high interrupt latency causing glitches in audio playback. There are political reasons why this code is not a desirable solution, and for copyright reasons I obviously can't post it for download.

The third attack seems to be the safest. I have had promising results by disabling SMA trapping on the VGA text-mode addresses. (Note that this will partly break text video modes. This is not an issue for us). This code has not yet been through rigorous testing, but it seems to be working. Once I have verified that it is "the" fix, I will post the sourcecode and precompiled binary here.

One final note: Yes, I am available for contract help with custom BIOS modifications and Linux distributions for embedding on these SBCs ;)

larwe.com and all original content herein is © Copyright 2003 by Lewin A.R.W. Edwards. "larwe.com" is a trademark protected under U.S. and international law. Infringement or attempted dilution of the intellectual property rights held by Lewin A.R.W. Edwards will be prosecuted to the fullest possible extent.