XD1000
From PEL Wiki
We have hardware!
Contents |
Status and Links to other pages
Right now I'm working on a few things:
- LinuxBIOS support - mostly done I've made a build tutorial on the LinuxBIOS wiki
- Linux Kernel Modifications - recognizes large memory area on FPGA
- Device Driver for ATA (or SATA) on FPGA Disk on DiskRAM
- Support for RAM on FPGA DIMMs on DiskRAM
- My FPGA design page
- General understanding of the Altera HyperTransport MegaCore
- Debugging my XD1000 Freezing Problem
- Testing DiskRAM
- DiskRAM Optimizations for later
- Booting Windows on DiskRAM
I have a few questions which need to have some debugging/profiling information collected for concrete answers:
- How often do I really get hit by Data Hazards in DiskRAM? (How much temporal locality is there?)
- I'm assuming that the processor's caches will filter much of the locality out before I see the references
- How often would a read buffer have helped me?
- I'm assuming that things won't be re-read because of the caches, so write buffers are needed, but not read buffers.
- Reference counts at each level of the hierarchy (buffer, DRAM, disk)
I'd like to have it run a (close to) standard kernel, so I'm recompiling. Unfortunately LinuxBIOS doesn't like ACPI, so that at least causes some warnings as it boots, and makes it so that I can't see the start-up messages from Linux, or the shutdown messages. It looks like it's just as robust as XDI's, other than that, though.
The ACPI messages were a red herring. See VGA Bios section below.
In order to make a kernel
- download the source from Fedora
- add the pci_ht.c file from XDI
- subtract the MSI quirk
- configure as single processor
VGA Bios
It turns out that since the board has built in video, the BIOS is supposed to contain the video BIOS as well. I think that the graphics problems are more related to the graphics card not being initialized correctly than they are to the kernel.
Since I didn't get the original BIOS with my system, I went to Tyan's web site and downloaded the newest version. Then I found (after trying in vain to use the BIOS update utility under qemu) PhoenixDeco, which decodes the BIOS and allows you to save out different parts. Strings helped me find which part was the Video BIOS, and I was on my way.
Now I'm going to build LinuxBIOS with the Video BIOS included, and see if that helps.
It turns out that you not only need to add the Video BIOS, you also need to:
- Disable BTEXT in etherboot and LinuxBIOS
- Make Sure CONSOLE_VGA and CONSOLE_DIRECT_VGA are enabled
LinuxBIOS actually runs the VGA Bios so late in the process that I don't get any output anyway. I don't know if it would be worth moving it, but for now I'll use the serial console. I see Linux boot up now and X doesn't hang anymore.
Slow RAM (aka my DDR controller)
I started to make a memory controller today. I went into the XD1000's HyperTransport core and set up the Base Address Registers (BARs) so that now there are four distinct memory regions to be mapped by the BIOS. There are a total of 6 BARs, but when you use a 64-bit address it uses two of them.
Now the BARs look like this: BAR 0/1 64k map of the I/O registers of the device BAR 2/3 256GB of memory space which is prefetchable BAR 4 32B of I/O space to be used as two legacy IDE controllers or SATA devices BAR 5 8k of memory space to be used for SATA
I'm going to test it up to that point now, to see if LinuxBIOS maps it into the address space without problems.
My machine hung. LinuxBIOS wasn't configured to allow the mapping of RAM or I/O above 4GB, so it didn't map all that it needed to and died. See the section on address space and mapping.
The next part is to find the parts of the VHDL which allow me to respond to writes to those spaces.
Address spaces and Memory mapping
I need to allow LinuxBIOS to map the board above 4GB. I decided to track down an existing error first. It turns out that every time LinuxBIOS boots, it gives the error: ERROR: PNP: 002e.1 74 drq size: 0x0000000001 not assigned
I'm going to try to assign it in Config.lb and see if it goes away, then I think I'll disable it anyway because it is taking a DMA resource for the parallel port. That seems like a questionable use of a resource. Then maybe I'll disable the floppy for the same reason (I don't have one in this box.)
It worked, and I've disabled both in case there is some better use for the resources. I think the config file should be changed so that if the device is enabled, the resources are assigned. It would also be nice if this were a compile-time instead of run-time error.
DRAM vs. MMIO
It seems like the natural way to map my board into the system would be to say that it is the DRAM attached to the processor replaced by the XD1000. I don't know if that's possible, because it may depend on the fact that there is a coherent link between the two. I need to get to the point where I can test that soon, though.
If I can't use that, then I'll have to do what I'm already doing (map it into the address space as MMIO.) I guess I don't know all of the complications that could await me because of that choice.
Memory Hole
I tried enabling MEMORY_HOLE and K8_MEM_HOLE_SIZEK=1024*1024, but it didn't make any difference. Linux booted with the same memory map it always had. I was interested in this in case I need to move memory around when I enable DiskRAM.
It looks like etherboot may be the culprit. I'm tired of the chain of events that loads a kernel. It seems tortuous.
- LinuxBIOS
- Elfboot
- Etherboot
- Filo
- Final Linux kernel image in elf format
The problem is that changes that I make in LinuxBIOS don't always get picked up by the next stage. Since I already need to become familiar with the Linux kernel and LinuxBIOS, I'm going to try the OLPC (One Laptop Per Child) method and boot into a tiny Linux kernel, then a normal kernel. Hopefully this will make my changes more coherent.
All right, I'm done with that. Now my machine boots a Linux kernel from the ROM as LinuxBIOS's payload, then uses kexec to start the next kernel.
Now I need to go back and figure out why my device is being disabled, and where. It's not clear to me if LinuxBIOS is successfully enabling it or not. I need to try using an old FPGA image to see if that helps. If not I need to start digging in the kernel. I had hoped that enabling 64-bit PCI resources would allow it to work. Oh well.
Linux Kernel Modifications is where I'm tracking my progress.
OLPC tries
I tried to build the OLPC ROM image.
- It needs to be built in a 32-bit environment (no problem, I'll use my P4 box)
- It obviously uses a different motherboard (I'll just take its payload, and build LinuxBIOS for myself)
When I tried to build a single image, nothing happened. I think I broke the config file. I'm going to go back and try using a "normal" and a "fallback" image.
All right, I got it to build by building LinuxBIOS on the XD1000 and building the kernel and the rest on my P4.
I can't tell where it is dying now, but at least I get it to start. I need to try compressing a simple program and running that, so I know that the problem is my tiny kernel, not the compression.
It turns out that it is dying in the BIOS still, not getting to the payload. It dies complaining that it can't find the AMD 8111, the southbridge in charge of reset. I tracked it down to a change in bus enumeration between versions of LinuxBIOS. I backed out one change, and we'll see. It was using pci_locate_device_on_bus instead of pci_locate_device.
That wasn't the problem. It seems that LinuxBIOS added some #defines to organize the HT bus, but they break the XD1000. I just turned them off, and it works.
There are several steps to getting the OLPC idea to work.
LinuxBIOS
This part is done. I can boot with LinuxBIOS and use a lzma-encoded payload.
I have a BIOS Savior which allows me to switch between two different BIOS chips. The nice thing about this is that I can develop without worrying (too much) about ruining my board.
Unfortunately the built-in ROM for the BIOS Savior is only 512K, and my ROM is 1MB. This means that I can't have Linux in that ROM. So the current status is Linux tiny kernel that Kexecs to Linux in the 1 MB ROM, and Linux loading FILO, which loads the kernel in the 512K ROM. When it comes up it asks for a boot location. Right now that is hdb1:/boot/linuz.elf, which is my kernel with SATA support so that I can still get to my development stuff on the main drive. hda is going to be the windows drive when I boot from this ROM. We'll see how that works :)
Hopefully that's clear.
tiny kernel
I've configured a tiny kernel. It seems that the one I downloaded from OLPC had some broken things (linux-2.6.18-rc4) so I updated to linux-2.6.18 which is a stable release, and they went away. I applied the patches for the tiny kernel, took out jffs support and added ext3 and a driver for my sata devices.
irqs
I need to figure out how to assign these so that my devices can function properly. In the normal kernel, these are the IRQs that are assigned to some of the devices
- 01:02.1 SMBus nv CK804 - pin A routed to IRQ 10
- 01:03.0 USB Controller - pin A routed to IRQ 185
- 01:03.1 USB Controller - pin B routed to IRQ 187
- 01:08.0 IDE CK804 SATA - pin A routed to IRQ 201
- 01:09.0 IDE CK804 SATA - pin A routed to IRQ 193
- 02:06.0 VGA ATI Rage XL- pin A routed to IRQ 161
- 02:08.0 Ethernet Intel - pin A routed to IRQ 161
- 08:09.0 Ethernet 5704 - pin A routed to IRQ 50
- 08:09.1 Ethernet 5704 - pin A routed to IRQ 58
As I suspected, the IRQ's are not set correctly when I boot into the kernel.
memory maps
I need to figure out what to change to pass this to the next running kernel. It's possible that I should be taking care of it in the same place as the IRQs.
initrd-fs
I hope I don't have to mess with this very much. It's the filesystem that's packed in the flash with the kernel.
devices.txt and initrd-rootfs
They create devices with makedevs, a program that takes a formatted text file and makes device nodes. I need to add devices for anything my kernel may need. Right now that includes the hard drives I want to boot from, and I need to make sure the console is set up correctly.
in bin/boot.functions they set up the terminal and define $VT=1 for my machine it needed to be $VT=0 This changes whether it opens /dev/tty0 or /dev/tty1
I made it open the right console and then start the shell instead of starting their graphical boot menu.
boot menu
I decided to ignore the boot menu, since I should never have to use it. That frees me from having to make the frame buffer work or write my own text menu.
I need to add an option so that it can boot my normal kernel here. Right now it only has support for ethernet, flash, and wireless boot.
pciutils
I needed to build pciutils against uclibc so that I could use it for debugging. I downloaded their pciutils.mk from buildroot.uclibc.org/ and put that in packages/pciutils/ after some customization so that it allowed the makefile to pass through the needed compile flags, it builds. pci.ids probably won't fit in the ROM eventually. It takes up quite a bit of space.
kexec-tools
It turns out that the kexec tools included with OLPC aren't equipped for handling 64-bit kernels. That's not too surprising since they're trying to economize on space.
I built my own statically-linked version of kexec-tools and put them on the hard drive (they're large.)
Now I get the error:
No filesystem could mount root, tried: ext2 iso9660" Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(3,0)
I tried to copy the files I needed off the hard drive and then unmount it first, but no such luck.
At least it tries to boot. When I try to kexec from anything but single user mode, it just reboots.
I need to try to figure out if it isn't trying ext3 for some good reason.
I'll try using FILO and Etherboot to boot into a larger ext3 enabled kernel.
It turns out that kexec works fine when you give it an elf instead of the vmlinux file. My problems vanished once I fixed that.
Categories: Pel | Myles | LinuxBIOS | LinuxKernel
