Linuxdoc Linux Questions
Click here to ask our community of linux experts!
Custom Search

2. Linux Makefiles

Before perusing Linux code, we should get some basic idea about how Linux is composed, compiled and linked. A straightforward way to achieve this goal is to understand Linux makefiles. Check Cross-Referencing Linux if you prefer online source browsing.

2.1. linux/Makefile

Here are some well-known targets in this top-level makefile:

  • xconfig, menuconfig, config, oldconfig: generate kernel configuration file linux/.config;

  • depend, dep: generate dependency files, like linux/.depend, linux/.hdepend and .depend in subdirectories;

  • vmlinux: generate resident kernel image linux/vmlinux, the most important target;

  • modules, modules_install: generate and install modules in /lib/modules/$(KERNELRELEASE);

  • tags: generate tag file linux/tags, for source browsing with vim.

Overview of linux/Makefile is outlined below:

include .depend
include .config
include arch/i386/Makefile

vmlinux: generate linux/vmlinux
        /* entry point "stext" defined in arch/i386/kernel/head.S */
        $(LD) -T $(TOPDIR)/arch/i386/vmlinux.lds -e stext
        /* $(HEAD) */
        + from arch/i386/Makefile
                arch/i386/kernel/head.o
                arch/i386/kernel/init_task.o
        init/main.o
        init/version.o
        init/do_mounts.o
        --start-group
        /* $(CORE_FILES) */
        + from arch/i386/Makefile
                arch/i386/kernel/kernel.o
                arch/i386/mm/mm.o
        kernel/kernel.o
        mm/mm.o
        fs/fs.o
        ipc/ipc.o
        /* $(DRIVERS) */
        drivers/...
                char/char.o
                block/block.o
                misc/misc.o
                net/net.o
                media/media.o
                cdrom/driver.o
                and other static linked drivers
                + from arch/i386/Makefile
                        arch/i386/math-emu/math.o (ifdef CONFIG_MATH_EMULATION)
        /* $(NETWORKS) */
        net/network.o
        /* $(LIBS) */
        + from arch/i386/Makefile
                arch/i386/lib/lib.a
        lib/lib.a
        --end-group
        -o vmlinux
        $(NM) vmlinux | grep ... | sort > System.map
tags: generate linux/tags for vim
modules: generate modules
modules_install: install modules
clean mrproper distclean: clean up build directory
psdocs pdfdocs htmldocs mandocs: generate kernel documents

include Rules.make

rpm: generate an rpm
"--start-group" and "--end-group" are ld command line options to resolve symbol reference problem. Refer to Using LD, the GNU linker: Command Line Options for details.

Rules.make contains rules which are shared between multiple Makefiles.

2.2. linux/arch/i386/vmlinux.lds

After compilation, ld combines a number of object and archive files, relocates their data and ties up symbol references. linux/arch/i386/vmlinux.lds is designated by linux/Makefile as the linker script used in linking the resident kernel image linux/vmlinux.

/* ld script to make i386 Linux kernel
 * Written by Martin Mares <mj@atrey.karlin.mff.cuni.cz>;
 */
OUTPUT_FORMAT("elf32-i386", "elf32-i386", "elf32-i386")
OUTPUT_ARCH(i386)
/* "ENTRY" is overridden by command line option "-e stext" in linux/Makefile */
ENTRY(_start)
/* Output file (linux/vmlinux) layout.
 * Refer to Using LD, the GNU linker: Specifying Output Sections */
SECTIONS
{
/* Output section .text starts at address 3G+1M.
 * Refer to Using LD, the GNU linker: The Location Counter */
  . = 0xC0000000 + 0x100000;
  _text = .;                    /* Text and read-only data */
  .text : {
        *(.text)
        *(.fixup)
        *(.gnu.warning)
        } = 0x9090
/* Unallocated holes filled with 0x9090, i.e. opcode for "NOP NOP".
 * Refer to Using LD, the GNU linker: Optional Section Attributes */

  _etext = .;                   /* End of text section */

  .rodata : { *(.rodata) *(.rodata.*) }
  .kstrtab : { *(.kstrtab) }

/* Aligned to next 16-bytes boundary.
 * Refer to Using LD, the GNU linker: Arithmetic Functions */
  . = ALIGN(16);                /* Exception table */
  __start___ex_table = .;
  __ex_table : { *(__ex_table) }
  __stop___ex_table = .;

  __start___ksymtab = .;        /* Kernel symbol table */
  __ksymtab : { *(__ksymtab) }
  __stop___ksymtab = .;

  .data : {                     /* Data */
        *(.data)
        CONSTRUCTORS
        }
/* For "CONSTRUCTORS", refer to
 * Using LD, the GNU linker: Option Commands */

  _edata = .;                   /* End of data section */

  . = ALIGN(8192);              /* init_task */
  .data.init_task : { *(.data.init_task) }

  . = ALIGN(4096);              /* Init code and data */
  __init_begin = .;
  .text.init : { *(.text.init) }
  .data.init : { *(.data.init) }
  . = ALIGN(16);
  __setup_start = .;
  .setup.init : { *(.setup.init) }
  __setup_end = .;
  __initcall_start = .;
  .initcall.init : { *(.initcall.init) }
  __initcall_end = .;
  . = ALIGN(4096);
  __init_end = .;

  . = ALIGN(4096);
  .data.page_aligned : { *(.data.idt) }

  . = ALIGN(32);
  .data.cacheline_aligned : { *(.data.cacheline_aligned) }

  __bss_start = .;              /* BSS */
  .bss : {
        *(.bss)
        }
  _end = . ;

/* Output section /DISCARD/ will not be included in the final link output.
 * Refer to Using LD, the GNU linker: Section Definitions */
  /* Sections to be discarded */
  /DISCARD/ : {
        *(.text.exit)
        *(.data.exit)
        *(.exitcall.exit)
        }

/* The following output sections are addressed at memory location 0.
 * Refer to Using LD, the GNU linker: Optional Section Attributes */
  /* Stabs debugging sections.  */
  .stab 0 : { *(.stab) }
  .stabstr 0 : { *(.stabstr) }
  .stab.excl 0 : { *(.stab.excl) }
  .stab.exclstr 0 : { *(.stab.exclstr) }
  .stab.index 0 : { *(.stab.index) }
  .stab.indexstr 0 : { *(.stab.indexstr) }
  .comment 0 : { *(.comment) }
}

2.3. linux/arch/i386/Makefile

linux/arch/i386/Makefile is included by linux/Makefile to provide i386 specific items and terms.

All the following targets depend on target vmlinux of linux/Makefile. They are accomplished by making corresponding targets in linux/arch/i386/boot/Makefile with some options.

Table 1. Targets in linux/arch/i386/Makefile

Target Command
zImage [a] @$(MAKE) -C arch/i386/boot zImage [b]
bzImage @$(MAKE) -C arch/i386/boot bzImage
zlilo @$(MAKE) -C arch/i386/boot BOOTIMAGE=zImage zlilo
bzlilo @$(MAKE) -C arch/i386/boot BOOTIMAGE=bzImage zlilo
zdisk @$(MAKE) -C arch/i386/boot BOOTIMAGE=zImage zdisk
bzdisk @$(MAKE) -C arch/i386/boot BOOTIMAGE=bzImage zdisk
install @$(MAKE) -C arch/i386/boot BOOTIMAGE=bzImage install
Notes:
a. zImage alias: compressed;
b. "-C" is a MAKE command line option to change directory before reading makefiles;
Refer to GNU make: Summary of Options and GNU make: Recursive Use of make.

It is worth noticing that this makefile redefines some environment variables which are exported by linux/Makefile, specifically:

OBJCOPY=$(CROSS_COMPILE)objcopy -O binary -R .note -R .comment -S
The effect will be passed to subdirectory makefiles and will change the tool's behavior. Refer to GNU Binary Utilities: objcopy for objcopy command line option details.

Not sure why $(LIBS) includes "$(TOPDIR)/arch/i386/lib/lib.a" twice:

LIBS := $(TOPDIR)/arch/i386/lib/lib.a $(LIBS) $(TOPDIR)/arch/i386/lib/lib.a
It may be employed to work around linking problems with some toolchains.

2.4. linux/arch/i386/boot/Makefile

linux/arch/i386/boot/Makefile is somehow independent as it is not included by either linux/arch/i386/Makefile or linux/Makefile.

However, they do have some relationship:

  • linux/Makefile: provides resident kernel image linux/vmlinux;

  • linux/arch/i386/boot/Makefile: provides bootstrap;

  • linux/arch/i386/Makefile: makes sure linux/vmlinux is ready before the bootstrap is constructed, and exports targets (like bzImage) to linux/Makefile.

$(BOOTIMAGE) value, which is for target zdisk, zlilo or zdisk, comes from linux/arch/i386/Makefile.

Table 2. Targets in linux/arch/i386/boot/Makefile

Target Command
zImage
$(OBJCOPY) compressed/vmlinux compressed/vmlinux.out
tools/build bootsect setup compressed/vmlinux.out $(ROOT_DEV) > zImage
bzImage
$(OBJCOPY) compressed/bvmlinux compressed/bvmlinux.out
tools/build -b bbootsect bsetup compressed/bvmlinux.out $(ROOT_DEV) \
        > bzImage
zdisk
dd bs=8192 if=$(BOOTIMAGE) of=/dev/fd0
zlilo
if [ -f $(INSTALL_PATH)/vmlinuz ]; then mv $(INSTALL_PATH)/vmlinuz
        $(INSTALL_PATH)/vmlinuz.old; fi
if [ -f $(INSTALL_PATH)/System.map ]; then mv $(INSTALL_PATH)/System.map
        $(INSTALL_PATH)/System.old; fi
cat $(BOOTIMAGE) > $(INSTALL_PATH)/vmlinuz
cp $(TOPDIR)/System.map $(INSTALL_PATH)/
if [ -x /sbin/lilo ]; then /sbin/lilo; else /etc/lilo/install; fi
install
sh -x ./install.sh $(KERNELRELEASE) $(BOOTIMAGE) $(TOPDIR)/System.map
        "$(INSTALL_PATH)"
tools/build builds boot image zImage from {bootsect, setup, compressed/vmlinux.out}, or bzImage from {bbootsect, bsetup, compressed/bvmlinux,out}. linux/Makefile "export ROOT_DEV = CURRENT". Note that $(OBJCOPY) has been redefined by linux/arch/i386/Makefile in Section 2.3.

Table 3. Supporting targets in linux/arch/i386/boot/Makefile

Target: Prerequisites Command
compressed/vmlinux: linux/vmlinux @$(MAKE) -C compressed vmlinux
compressed/bvmlinux: linux/vmlinux @$(MAKE) -C compressed bvmlinux
tools/build: tools/build.c $(HOSTCC) $(HOSTCFLAGS) -o $@ $< -I$(TOPDIR)/include [a]
bootsect: bootsect.o $(LD) -Ttext 0x0 -s --oformat binary bootsect.o [b]
bootsect.o: bootsect.s $(AS) -o $@ $<
bootsect.s: bootsect.S ... $(CPP) $(CPPFLAGS) -traditional $(SVGA_MODE) $(RAMDISK) $< -o $@
bbootsect: bbootsect.o $(LD) -Ttext 0x0 -s --oformat binary $< -o $@
bbootsect.o: bbootsect.s $(AS) -o $@ $<
bbootsect.s: bootsect.S ... $(CPP) $(CPPFLAGS) -D__BIG_KERNEL__ -traditional $(SVGA_MODE) $(RAMDISK) $< -o $@
setup: setup.o $(LD) -Ttext 0x0 -s --oformat binary -e begtext -o $@ $<
setup.o: setup.s $(AS) -o $@ $<
setup.s: setup.S video.S ... $(CPP) $(CPPFLAGS) -D__ASSEMBLY__ -traditional $(SVGA_MODE) $(RAMDISK) $< -o $@
bsetup: bsetup.o $(LD) -Ttext 0x0 -s --oformat binary -e begtext -o $@ $<
bsetup.o: bsetup.s $(AS) -o $@ $<
bsetup.s: setup.S video.S ... $(CPP) $(CPPFLAGS) -D__BIG_KERNEL__ -D__ASSEMBLY__ -traditional $(SVGA_MODE) $(RAMDISK) $< -o $@
Notes:
a. "$@" means target, "$<" means first prerequisite; Refer to GNU make: Automatic Variables;
b. "--oformat binary" asks for raw binary output, which is identical to the memory dump of the executable; Refer to Using LD, the GNU linker: Command Line Options.
Note that it has "-D__BIG_KERNEL__" when compile bootsect.S to bbootsect.s, and setup.S to bsetup.s. They must be Place Independent Code (PIC), thus what "-Ttext" option is doesn't matter.

2.5. linux/arch/i386/boot/compressed/Makefile

This makefile handles image (de)compression mechanism.

It is good to separate (de)compression from bootstrap. This divide-and-conquer solution allows us to easily improve (de)compression mechanism or to adopt a new bootstrap method.

Directory linux/arch/i386/boot/compressed/ contains two source files: head.S and misc.c.

Table 4. Targets in linux/arch/i386/boot/compressed/Makefile

Target Command
vmlinux[a] $(LD) -Ttext 0x1000 -e startup_32 -o vmlinux head.o misc.o piggy.o
bvmlinux $(LD) -Ttext 0x100000 -e startup_32 -o bvmlinux head.o misc.o piggy.o
head.o $(CC) $(AFLAGS) -traditional -c head.S
misc.o
$(CC) $(CFLAGS) -DKBUILD_BASENAME=$(subst $(comma),_,$(subst -,_,$(*F)))
        -c misc.c[b]
piggy.o
tmppiggy=_tmp_$$$$piggy; \
rm -f $$tmppiggy $$tmppiggy.gz $$tmppiggy.lnk; \
$(OBJCOPY) $(SYSTEM) $$tmppiggy; \
gzip -f -9 < $$tmppiggy > $$tmppiggy.gz; \
echo "SECTIONS { .data : { input_len = .; \
        LONG(input_data_end - input_data) input_data = .; \
        *(.data) input_data_end = .; }}" > $$tmppiggy.lnk; \
$(LD) -r -o piggy.o -b binary $$tmppiggy.gz -b elf32-i386 \
        -T $$tmppiggy.lnk; \
rm -f $$tmppiggy $$tmppiggy.gz $$tmppiggy.lnk
Notes:
a. Target vmlinux here is different from that defined in linux/Makefile;
b. "subst" is a MAKE function; Refer to GNU make: Functions for String Substitution and Analysis.

piggy.o contains variable input_len and gzipped linux/vmlinux. input_len is at the beginning of piggy.o, and it is equal to the size of piggy.o excluding input_len itself. Refer to Using LD, the GNU linker: Section Data Expressions for "LONG(expression)" in piggy.o linker script.

To be exact, it is not linux/vmlinux itself (in ELF format) that is gzipped but its binary image, which is generated by objcopy command. Note that $(OBJCOPY) has been redefined by linux/arch/i386/Makefile in Section 2.3 to output raw binary using "-O binary" option.

When linking {bootsect, setup} or {bbootsect, bsetup}, $(LD) specifies "--oformat binary" option to output them in binary format. When making zImage (or bzImage), $(OBJCOPY) generates an intermediate binary output from compressed/vmlinux (or compressed/bvmlinux) too. It is vital that all components in zImage or bzImage are in raw binary format, so that the image can run by itself without asking a loader to load and relocate it.

Both vmlinux and bvmlinux prepend head.o and misc.o before piggy.o, but they are linked against different start addresses (0x1000 vs 0x100000).

2.6. linux/arch/i386/tools/build.c

linux/arch/i386/tools/build.c is a host utility to generate zImage or bzImage.

In linux/arch/i386/boot/Makefile:

tools/build bootsect setup compressed/vmlinux.out $(ROOT_DEV) > zImage

tools/build -b bbootsect bsetup compressed/bvmlinux.out $(ROOT_DEV) > bzImage
"-b" means is_big_kernel, used to check whether system image is too big.

tools/build outputs the following components to stdout, which is redirected to zImage or bzImage:

  1. bootsect or bbootsect: from linux/arch/i386/boot/bootsect.S, 512 bytes;

  2. setup or bsetup: from linux/arch/i386/boot/setup.S, 4 sectors or more, sector aligned;

  3. compressed/vmlinux.out or compressed/bvmlinux.out, including:

    1. head.o: from linux/arch/i386/boot/compressed/head.S;

    2. misc.o: from linux/arch/i386/boot/compressed/misc.c;

    3. piggy.o: from input_len and gzipped linux/vmlinux.

tools/build will change some contents of bootsect or bbootsect when outputting to stdout:

Table 5. Modification made by tools/build

Offset Byte Variable Comment
1F1 (497) 1 setup_sectors number of setup sectors, >=4
1F4 (500) 2 sys_size system size in 16-bytes, little-endian
1FC (508) 1 minor_root root dev minor
1FD (509) 1 major_root root dev major

In the following chapters, compressed/vmlinux will be referred as vmlinux and compressed/bvmlinux as bvmlinux, if not confusing.

2.7. Reference