Compiler Tips for uClinux

uClinux beginners often get frustrated when things don't work.
The documentation is rather outdated, and googling isn't that helpful.


First off, lets look at whats needed for Linux 2.4 compiler wise. (from http://www.linux-mips.org/wiki/Kernel_Build )

The recommended compiler for building Linux 2.4 is gcc 2.95 however the large majority of developers is probably user a more modern GCC. New compilers up to gcc 3.3 have been used successfully to build the kernel and only the latest 2.4 kernels have fixed the issues with gcc 3.4. GCC 4.x can not be used and with Linux 2.4 being in deep freeze mode we generally don't intend to fix such problems but rather recommend to use an older compiler for the kernel.

The latest official GCC 2 is version 2.95.3[1]. GCC 2.96[2] is an unofficial, massively patched version. It provides more efficient optimization with -O2, more strict syntax checking and supports more recent C++ standards. Unfortunately the C++ ABI is not fully compatible[3] with the official 2.95.3 nor 3.0. This is not a concern for building the kernel or modules as those are written in plain C but is a consideration if using a single toolchain for an entire distribution or product.

We have an existing working gcc compiler chain - aka arm_tools which comes with gcc 3.0, this can compile our kernel 2.4.20 version, but its quite old. We also have a gcc 3.3.4 version with the BSP which is supposed to be used to compile the userland (romfs) code.

Ideally we should be using one compiler chain to do it all, but doesn't work straight out of the box.
There is a mailing list for uClinux, but they really need better documentation (which is pretty much what I'm doing here )

So, lets talk about how things compile under uClinux.

If you've been following my previous examples on how to setup a BSP, you'll note that we have 2 compiler environments, and 2 uClinux distributions.

The smarter readers will probably have a question at this point.

How does each distribution know what compiler to use?

Good question 

uClinux has a number of folders in its structure.
The main one we'll be looking at in this post is the vendors folder.

In both the BSP and the current uClinux we see a list of vendors in the vendors folder.
The vendors folder contains the build instructions for each given architecture and cpu type.

Lets take a look at the BSP folder first, as its cleaner.

Code: [Select]

cd /home/N745/W90N745-uClinux/uClinux-dist/vendors/
minty vendors # ls -al
total 16
drwxr-xr-x  4 root root 4096 May 23  2008 .
drwxr-xr-x 10 root root 4096 May 26  2008 ..
drwxr-xr-x  3 root root 4096 May 23  2008 config
drwxr-xr-x  3 root root 4096 May 23  2008 Winbond

As you see, we have 2 folders.  One for the manufacturer, and one called config.
The config folder is the one we'll look at next

Code: [Select]

cd config/
minty config # ls -al
total 12
drwxr-xr-x 3 root root 4096 May 23  2008 .
drwxr-xr-x 4 root root 4096 May 23  2008 ..
drwxr-xr-x 2 root root 4096 May 23  2008 armnommu
minty config # cd armnommu/
minty armnommu # ls -al
total 24
drwxr-xr-x 2 root root 4096 May 23  2008 .
drwxr-xr-x 3 root root 4096 May 23  2008 ..
-rwxr-xr-x 1 root root 4401 May 23  2008 config.arch
-rwxr-xr-x 1 root root 4198 May 23  2008 config_old.arch

As you can see, we have a subfolder called armnommu.
armnommu refers to our cpu type.

We're running on an ARM7TDMI based chip, which is arm based, and has no mmu (no memory management unit).
Hence the cunningly named armnommu folder

The config.arch file is what tells the compiler how to build for our architecture.
Lets take a look inside

Code: [Select]

cat config.arch

...
MACHINE       = arm
ARCH          = armnommu
CROSS_COMPILE = arm-elf-
CROSS         = $(CROSS_COMPILE)

CC        = $(CROSS_COMPILE)gcc $(CPUFLAGS)

I've excerpted the relevant bits we need to look at.

CROSS_COMPILE = arm-elf-
CROSS = $(CROSS_COMPILE)
CC = $(CROSS_COMPILE)gcc $(CPUFLAGS)

Those 3 lines mean that our compiler call will look something like this

arm-elf-gcc

Essentially we've added a prefix to all the compiler code so it looks for arm-elf-XXX

Lets look at the config.arch for our ucLinux latest.

Code: [Select]

cd /home/uClinux-dist/vendors/config/armnommu
cat config.arch

This is a little more complicated, but it boils down to this.

It looks for these versions of the arm compiler in order of preference:
 arm-linux-20061213-gcc
 arm-linux-20070808-gcc

If it can't find them, it defaults to using this as a prefix.
 arm-linux-

We can find appropriate compiler chains which match those preferences over here -  http://ftp.snapgear.org/pub/snapgear/tools/arm-linux/

Lets take a look at our /usr/local/ folder now, and see whats in there.
If you remember from above, the BSP is looking for arm-elf-XXX filenames
...and the latest uClinux is looking for arm-linux-XXX filenames.

Our path should include the folders for both sets of build utilities.  Lets check.

Code: [Select]

echo $PATH
/usr/local/arm_tools/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

The BSP put our build tools in /usr/local/arm_tools/bin
It should contain arm-elf-xxx files, lets check

Code: [Select]

cd /usr/local/arm_tools/bin
 ls -al
total 23140
drwxr-xr-x 2 root root    4096 May 26  2008 .
drwxrwxr-x 9 root root    4096 May  7  2003 ..
-rwxrwxr-x 1 root root 1043382 Aug  3  2001 arm-elf-addr2line
-rwxrwxr-x 2 root root  943208 Aug  3  2001 arm-elf-ar
-rwxrwxr-x 2 root root 1479964 Aug  3  2001 arm-elf-as
-rwxrwxr-x 2 root root  267769 Aug  8  2001 arm-elf-c++
-rwxrwxr-x 1 root root  195997 Aug  8  2001 arm-elf-c++filt
-rwxrwxr-x 1 root root  267880 Aug  8  2001 arm-elf-cpp
-rwxrwxr-x 1 root root   70692 Aug  7  2003 arm-elf-depmod
-rwxr-xr-x 1 root root  592520 Mar 24  2005 arm-elf-elf2flt
-rwxr-xr-x 1 root root  390012 Mar 24  2005 arm-elf-flthdr
-rwxrwxr-x 2 root root  267769 Aug  8  2001 arm-elf-g++
-rwxrwxr-x 1 root root  272426 Aug  8  2001 arm-elf-g77
-rwxrwxr-x 1 root root  211109 Aug  3  2001 arm-elf-gasp
-rwxrwxr-x 1 root root  264177 Aug  8  2001 arm-elf-gcc
-rwxrwxr-x 1 root root  274191 Aug  8  2001 arm-elf-gcj
-rwxr-xr-x 1 root root    4746 Mar 24  2005 arm-elf-ld
-rwxrwxr-x 1 root root 1416491 Nov  1  2001 arm-elf-ld.real
-rwxrwxr-x 2 root root 1037487 Aug  3  2001 arm-elf-nm
-rwxrwxr-x 1 root root 1442773 Aug  3  2001 arm-elf-objcopy
-rwxrwxr-x 1 root root 1573662 Aug  3  2001 arm-elf-objdump
-rwxrwxr-x 2 root root  943207 Aug  3  2001 arm-elf-ranlib
-rwxrwxr-x 1 root root  445502 Aug  3  2001 arm-elf-readelf
-rwxrwxr-x 1 root root  884019 Aug  3  2001 arm-elf-size
-rwxrwxr-x 1 root root  883237 Aug  3  2001 arm-elf-strings
-rwxrwxr-x 2 root root 1442772 Aug  3  2001 arm-elf-strip
-rwxrwxr-x 1 root root 2685109 Nov  1  2001 elf2flt
-rwxrwxr-x 1 root root 1708896 Nov  1  2001 flthdr
-rwxrwxr-x 1 root root   16764 Aug  8  2001 gccbug
-rwxrwxr-x 1 root root  285104 Aug  8  2001 gcjh
-rwxrwxr-x 1 root root  109914 Aug  8  2001 gcov
-rwxrwxr-x 1 root root   19562 Jun 12  2003 genbin
-rwxrwxr-x 1 root root   15568 Feb 10  2003 genromfs
-rwxrwxr-x 1 root root  182342 Aug  8  2001 jar
-rwxrwxr-x 1 root root  305247 Aug  8  2001 jcf-dump
-rwxrwxr-x 1 root root  187871 Aug  8  2001 jv-scan
-rwxrwxr-x 1 root root    2006 Nov  2  2001 ld-elf2flt
-rwxrwxr-x 1 root root 1416491 Nov  1  2001 ld-elf2flt.real
-rwxrwxr-x 1 root root   16708 Feb 19  2003 makeimg
-rwxrwxr-x 1 root root   23279 Jun 18  2003 mkrom
-rwxrwxr-x 1 root root   19594 Oct 31  2003 mkupgfw


Yup, that looks good.

Now lets check our other folder (/usr/local/bin)

Code: [Select]

cd /usr/local/bin
minty bin # ls -al
total 9228
drwxr-xr-x  2 root root    4096 Feb  3 15:53 .
drwxrwsr-x 15 root staff   4096 Feb  3 17:25 ..
-rwxr-xr-x  1 root root    4275 Oct  4 16:30 apt
-rwxr-xr-x  1 root root  433320 Jun 23  2008 arm-linux-addr2line
-rwxr-xr-x  1 root root    1022 Jun 20  2008 arm-linux-addr2name.awk
-rwxr-xr-x  2 root root  454852 Jun 23  2008 arm-linux-ar
-rwxr-xr-x  2 root root  787160 Jun 23  2008 arm-linux-as
-rwxr-xr-x  2 root root  178660 Jun 23  2008 arm-linux-c++
-rwxr-xr-x  1 root root  433752 Jun 23  2008 arm-linux-c++filt
-rwxr-xr-x  1 root root  177956 Jun 23  2008 arm-linux-cpp
-rwxr-xr-x  1 root root  430000 Jun 23  2008 arm-linux-elf2flt
-rwxr-xr-x  1 root root   10064 Jun 23  2008 arm-linux-flthdr
-rwxr-xr-x  2 root root  178660 Jun 23  2008 arm-linux-g++
-rwxr-xr-x  2 root root  176612 Jun 23  2008 arm-linux-gcc
-rwxr-xr-x  2 root root  176612 Jun 23  2008 arm-linux-gcc-4.2.4
-rwxr-xr-x  1 root root   15934 Jun 20  2008 arm-linux-gccbug
-rwxr-xr-x  1 root root  181796 Jun 23  2008 arm-linux-gcj
-rwxr-xr-x  1 root root   77056 Jun 23  2008 arm-linux-gcjh
-rwxr-xr-x  1 root root   22788 Jun 23  2008 arm-linux-gcov
-rwxr-xr-x  1 root root  179780 Jun 23  2008 arm-linux-gfortran
-rwxr-xr-x  1 root root   77060 Jun 23  2008 arm-linux-gjnih
-rwxr-xr-x  1 root root  488840 Jun 23  2008 arm-linux-gprof
-rwxr-xr-x  1 root root   94764 Jun 23  2008 arm-linux-jcf-dump
-rwxr-xr-x  1 root root   68004 Jun 23  2008 arm-linux-jv-scan
-rwxr-xr-x  1 root root    5467 Jun 23  2008 arm-linux-ld
-rwxr-xr-x  2 root root  837008 Jun 23  2008 arm-linux-ld.real
-rwxr-xr-x  2 root root  442952 Jun 23  2008 arm-linux-nm
-rwxr-xr-x  2 root root  571412 Jun 23  2008 arm-linux-objcopy
-rwxr-xr-x  2 root root  685736 Jun 23  2008 arm-linux-objdump
-rwxr-xr-x  2 root root  454884 Jun 23  2008 arm-linux-ranlib
-rwxr-xr-x  1 root root  226000 Jun 23  2008 arm-linux-readelf
-rwxr-xr-x  1 root root  435032 Jun 23  2008 arm-linux-size
-rwxr-xr-x  1 root root  434600 Jun 23  2008 arm-linux-strings
-rwxr-xr-x  2 root root  571412 Jun 23  2008 arm-linux-strip
-rwxr-xr-x  1 root root   27126 May 20  2003 genext2fs
-rwxr-xr-x  1 root root   11384 Jun 26  2008 genromfs

Yup, that also looks good.

Now, lets try to compile the BSP with a newer toolchain, and see what happens.

Code: [Select]

cd /home/N745/W90N745-uClinux/uClinux-dist/vendors/config/armnommu
pico config.arch

Change the CROSS_COMPILE line as follows:

Code: [Select]

CROSS_COMPILE = arm-linux-

Save, and exit

Lets clean up our build folder in preparation for our build.

Code: [Select]

cd /home/N745/W90N745-uClinux/uClinux-dist
make clean
make dep

Finally, lets try to make the build

Code: [Select]

make

What happened?

Lets look at the output

Code: [Select]

make
make ARCH=armnommu CROSS_COMPILE=arm-linux- -C linux-2.4.x  || exit 1
make[1]: Entering directory `/home/N745/W90N745-uClinux/uClinux-dist/linux-2.4.x'
arm-linux-gcc -D__KERNEL__ -I/home/N745/W90N745-uClinux/uClinux-dist/linux-2.4.x/include -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fno-strict-aliasing -fno-common -fno-common -pipe -fno-builtin -D__linux__ -D__WB_EVB__ -DNO_MM -mapcs-32 -march=armv4 -mtune=arm7tdmi -mshort-load-bytes   -DKBUILD_BASENAME=main -c -o init/main.o init/main.c
cc1: error: unrecognized command line option "-mapcs-32"
cc1: error: unrecognized command line option "-mshort-load-bytes"
make[1]: *** [init/main.o] Error 1
make[1]: Leaving directory `/home/N745/W90N745-uClinux/uClinux-dist/linux-2.4.x'
make: *** [linux] Error 1

Looks like our newer gcc compiler doesn't support the -mapcs-32 or mshort-load-bytes command.

Lets take a look at the compiler documentation for GCC
http://gcc.gnu.org/onlinedocs/gcc-3.3.6/gcc/ARM-Options.html

Seems thats only supported in older versions of GCC.

Lets take a look at our original compiler binary.

Code: [Select]

./arm-elf-gcc  -v
Reading specs from ./../lib/gcc-lib/arm-elf/3.0/specs
Configured with: ../gcc-3.0/configure --target=arm-elf --host=i686-pc-linux --build=i686-pc-linux --prefix=/home/clyu2/gcc3.0/arm_com --with-gnu-ld --with-gnu-as : (reconfigured) ../gcc-3.0/configure --target=arm-elf --host=i686-pc-linux --build=i686-pc-linux --prefix=/home/clyu2/gcc3.0/arm_com --with-gnu-ld --with-gnu-as --with-newlib : (reconfigured) ../gcc-3.0/configure --target=arm-elf --host=i686-pc-linux --build=i686-pc-linux --prefix=/home/clyu2/gcc3.0/arm_com --with-gnu-ld --with-gnu-as --with-newlib : (reconfigured) ../gcc-3.0/configure --target=arm-elf --host=i686-pc-linux --build=i686-pc-linux --prefix=/home/clyu2/gcc3.0/arm_com --with-gnu-ld --with-gnu-as --with-newlib : (reconfigured) ../gcc-3.0/configure --target=arm-elf --host=i686-pc-linux --build=i686-pc-linux --prefix=/home/clyu2/gcc3.0/arm_com --with-gnu-ld --with-gnu-as --with-newlib : (reconfigured) ../gcc-3.0/configure --target=arm-elf --host=i686-pc-linux --build=i686-pc-linux --prefix=/home/clyu2/gcc3.0/arm_com --with-gnu-ld --with-gnu-as --with-newlib : (reconfigured) ../gcc-3.0/configure --target=arm-elf --host=i686-pc-linux --build=i686-pc-linux --prefix=/home/clyu2/gcc3.0/arm_com --with-gnu-ld --with-gnu-as --with-newlib : (reconfigured) ../gcc-3.0/configure --target=arm-elf --host=i686-pc-linux --build=i686-pc-linux --prefix=/home/clyu2/gcc3.0/arm_com --with-gnu-ld --with-gnu-as --with-newlib : (reconfigured) ../gcc-3.0/configure --target=arm-elf --host=i686-pc-linux --build=i686-pc-linux --prefix=/home/clyu2/gcc3.0/arm_com --with-gnu-ld --with-gnu-as --with-newlib
Thread model: single
gcc version 3.0

Yikes, its ancient - gcc 3.0!

What version of gcc is the newer compiler version?

Code: [Select]

./arm-linux-gcc -v
Using built-in specs.
Target: arm-linux
Configured with: ../configure --target=arm-linux --disable-shared --prefix=/usr/local --with-headers=/home/gerg/src/gnu/arm-linux/linux-2.6.25/include --with-gnu-as --with-gnu-ld --enable-multilib
Thread model: posix
gcc version 4.2.4

Hmm, the newer version is 4.2.4, much newer. (and also not going to work, as its made for the 2.6.x kernel, not 2.4!).  Lets continue anyway and follow through this example.

So, what to do?

Lets go back to the error

Code: [Select]

cc1: error: unrecognized command line option "-mapcs-32"
cc1: error: unrecognized command line option "-mshort-load-bytes"

The gcc compiler doesn't recognize those commands anymore, so lets see what replaced those.

Some googling shows that this can be fixed by using -mabi=apcs-gnu instead of -mapcs-32
and -mshort-load-bytes can fixed by changing it to -malignment-traps

Would be nice if the gcc compiler documentation mentioned this, but, it doesn't.  Sigh.
Now we know that,  lets go find where its adding that and fix it.

Code: [Select]

cd /home/N745/W90N745-uClinux/uClinux-dist/linux-2.4.x
grep mapcs * -R
arch/arm/Makefile:CFLAGS :=$(CFLAGS:-fomit-frame-pointer=-mapcs -mno-sched-prolog)
arch/arm/Makefile:apcs-y :=-mapcs-32
arch/arm/Makefile:apcs-$(CONFIG_CPU_26) :=-mapcs-26 -mcpu=arm3 -Os
arch/armnommu/Makefile:apcs-$(CONFIG_CPU_26) :=-mapcs-26 -mcpu=arm3 -Os
arch/armnommu/Makefile:apcs-$(CONFIG_CPU_32) :=-mapcs-32
drivers/block/doc/Makefile.sa: -Wno-trigraphs -mapcs -fno-strict-aliasing -fno-common -pipe -mapcs-32 \

Looks like its set in a few places.  Lets start with the appropriate one - arch/armnommu/Makefile

Change

apcs-$(CONFIG_CPU_32) so it uses the new compiler command.

Code: [Select]

apcs-$(CONFIG_CPU_32)           :=-mabi=apcs-gnu         

and change

Code: [Select]

CFLAGS          += $(apcs-y) $(arch-y) $(tune-y) -mshort-load-bytes
to

Code: [Select]

CFLAGS          += $(apcs-y) $(arch-y) $(tune-y) $(call cc-option,$(call cc-option,-malignment-traps,))
Save, and try to compile.

You'll get further this time, but it will still break.
However, we have fixed that particular compiler build issue.

Now you have an idea of why things don't work, and what to do to fix them.

If you want to continue with this, the next error is this:

Code: [Select]

init/do_mounts.c:56: error: static declaration of ‘root_device_name’ follows non-static declaration
/home/N745/W90N745-uClinux/uClinux-dist/linux-2.4.x/include/linux/fs.h:1488: error: previous declaration of ‘root_device_name’ was here

...

Google shows this as the issue - http://lkml.org/lkml/2005/6/12/58

So, which file are you going to edit, and why?

I know the answer, but do you?