split build and run scripts due to option conflicts

Remove -v from travis as it blew log length
This commit is contained in:
Ciro Santilli
2017-11-02 17:38:33 +00:00
parent d7569bec0c
commit 6c51ecaf7b
13 changed files with 220 additions and 225 deletions

View File

@@ -4,8 +4,9 @@ sudo: required
install: |
cd "$TRAVIS_BUILD_DIR"
./configure -t
bash -x ./configure -t
script: |
cd "$TRAVIS_BUILD_DIR"
bash -x ./run -e 'init=/poweroff.out' -j 16 -n -v
bash -x ./build -j 16
bash -x ./run -e 'init=/poweroff.out' -n

View File

@@ -6,6 +6,6 @@ Testing you should do before pushing: new modules:
New arch:
- `./run -a ARCH`
- `./build -a ARCH && ./run -a ARCH`
- `wget google.com` for Internet
- `./runqemu -a ARCH -d` and `./rungdb -a ARCH`
- `./run -a ARCH -d` and `./rungdb -a ARCH`

77
build Executable file
View File

@@ -0,0 +1,77 @@
#!/usr/bin/env bash
set -e
arch=x86_64
extra_targets=''
j="$(($(nproc) - 2))"
x11=false
v=0
while getopts 'a:e:j:nt:v' OPT; do
case "$OPT" in
a)
arch="$OPTARG"
;;
j)
j="$OPTARG"
;;
t)
extra_targets="$extra_args $OPTARG"
;;
x)
x11=true
;;
v)
v=1
;;
esac
done
case "$arch" in
x86_64)
defconfig=qemu_x86_64_defconfig
;;
arm)
# qemu_arm_vexpress_defconfig required a newer QEMU than 2.0.0 on a Ubuntu host.
# so let's stick to versatile for now.
defconfig=qemu_arm_versatile_defconfig
;;
aarch64)
defconfig=qemu_aarch64_virt_defconfig
;;
mips64)
defconfig=qemu_mips64r6_malta_defconfig
;;
esac
cd kernel_module
./make-host.sh -j "$j" clean
cd ../buildroot
for p in $(find '../buildroot_patches/' -maxdepth 1 -name '*.patch' -print); do
patch -N -r - -p 1 <"$p" || :
done
outdir="output.${arch}~"
make O="$outdir" BR2_EXTERNAL="$(pwd)/../kernel_module" "$defconfig"
# TODO Can't get rid of this for now.
# http://stackoverflow.com/questions/44078245/is-it-possible-to-use-config-fragments-with-buildroots-config
cat ../buildroot_config_fragment >> "${outdir}/.config"
if $x11; then
cat ../buildroot_config_fragment_x11 >> "${outdir}/.config"
fi
make O="$outdir" olddefconfig
# HOST_QEMU_OPTS is a hack that happens to work because the QEMU package luckly uses += at all times.
# It shouldn't be necessary in the first place: https://bugs.busybox.net/show_bug.cgi?id=9936
#
# Even if were an autotools package, there is no general way currently to pass extra configs to it:
# https://stackoverflow.com/questions/44341188/how-to-pass-extra-custom-configure-autotools-options-to-a-buildroot-package/44341225#44341225
time \
env \
-u LD_LIBRARY_PATH \
make \
O="$outdir" \
BR2_JLEVEL="$j" \
HOST_QEMU_OPTS="--enable-debug --enable-sdl --extra-cflags='-DDEBUG_PL061=1' --with-sdlabi=2.0 --enable-trace-backends=simple" \
V="$v" \
kernel_module-rebuild \
$extra_targets \
all \
;
cd ..
./runqemu "$@"

View File

@@ -8,7 +8,7 @@
Best attempt so far:
time ./runqemu -n -e 'init=/poweroff.out' -- -trace exec_tb,file=trace && \
time ./run -n -e 'init=/poweroff.out' -- -trace exec_tb,file=trace && \
time ./qemu/scripts/simpletrace.py qemu/trace-events trace >trace.txt && \
wc -l trace.txt && \
sed '/0x1000000/q' trace.txt >trace-boot.txt && \

View File

@@ -2,7 +2,7 @@
To GDB the Linux kernel, first run:
./runqemu -d
./run -d
If you want to break immediately at a symbol, e.g. `start_kernel` of the boot sequence, run on another shell:
@@ -46,7 +46,7 @@ So we cannot set the breakpoints before `insmod`.
However, the Linux kernel GDB scripts offer the `lx-symbols` command, which takes care of that beautifully for us:
./runqemu -d
./run -d
./rungdb
In QEMU:

View File

@@ -91,7 +91,7 @@ Custom init process:
- Shell 1:
./runqemu -d -e 'init=/sleep_forever.out' -n
./run -d -e 'init=/sleep_forever.out' -n
- Shell 2:
@@ -101,7 +101,7 @@ BusyBox custom init process:
- Shell 1:
./runqemu -d -e 'init=/bin/ls' -n
./run -d -e 'init=/bin/ls' -n
- Shell 2:
@@ -113,7 +113,7 @@ BusyBox default init process:
- Shell 1:
./runqemu -d -n
./run -d -n
- Shell 2:
@@ -127,7 +127,7 @@ Non-init process:
- Shell 1
./runqemu -d -n
./run -d -n
- Shell 2

View File

@@ -47,6 +47,7 @@ Reserve 12Gb of disk:
git clone --recursive https://github.com/cirosantilli/linux-kernel-module-cheat
cd linux-kernel-module-cheat
./configure
./build
./run
The first build will take a while ([GCC](https://stackoverflow.com/questions/10833672/buildroot-environment-with-host-toolchain), Linux kernel), e.g.:
@@ -80,23 +81,18 @@ Good bets inside guest are:
/modulename.sh
/modulename.out
## Save rebuild time
## Rebuild
After the first build, you can also run just:
./runqemu
to save a few seconds. `./run` wouldn't rebuild everything, but checking timestamps takes a few moments.
If you make changes to the kernel modules or most configurations, you can just use again:
If you make changes to the kernel modules or most configurations tracked on this repository, you can just use again:
./build
./run
and they will updated.
and the modified files will be rebuilt.
But if you change any package besides `kernel_module`, you must also request those packages to be reconfigured or rebuilt with extra targets, e.g.:
If you change any package besides `kernel_module`, you must also request those packages to be reconfigured or rebuilt with extra targets, e.g.:
./run -t linux-reconfigure -t host-qemu-reconfigure
./build -t linux-reconfigure -t host-qemu-reconfigure
Those aren't turned on by default because they take quite a few seconds.
@@ -104,20 +100,20 @@ Those aren't turned on by default because they take quite a few seconds.
The root filesystem is persistent across:
./runqemu
./run
date >f
sync
then:
./runqemu
./run
cat f
This is particularly useful to re-run shell commands from the history of a previous session with `Ctrl + R`.
When you do:
./run
./build
the disk image gets overwritten by a fresh filesystem and you lose all changes.
@@ -144,7 +140,7 @@ but I never managed to increase that buffer:
Show serial output of QEMU directly on the current terminal, without opening a QEMU window:
./runqemu -n
./run -n
To exit, just do a regular:
@@ -273,7 +269,7 @@ When you start interacting with QEMU hardware, it is useful to see what is going
This is of course trivial since QEMU is just an userland program on the host, but we make it a bit easier with:
./runqemu -q
./run -q
Then you could:

View File

@@ -20,11 +20,11 @@ Is the default BusyBox `/init` too bloated for you, minimalism freak?
No problem, just use the `init` kernel boot parameter:
./runqemu -e 'init=/sleep_forever.out'
./run -e 'init=/sleep_forever.out'
Remember that shell scripts can also be used for `init` <https://unix.stackexchange.com/questions/174062/init-as-a-shell-script/395375#395375>:
./runqemu -e 'init=/count.sh'
./run -e 'init=/count.sh'
Also remember that if your init returns, the kernel will panic, there are just two non-panic possibilities:
@@ -41,4 +41,4 @@ Add this line to `rootfs_post_build_script`:
To restore it, run:
./run -t initscripts-reconfigure
./build -t initscripts-reconfigure

View File

@@ -8,7 +8,7 @@ Cheaper than JTAG (free) and easier to setup (all you need is serial), but with
Usage:
./runqemu -k
./run -k
./rungdb -k
In GDB:

View File

@@ -6,15 +6,12 @@ The portability of the kernel and toolchains is amazing: change an option and mo
First build:
./build -a arm
./run -a arm
Run without build:
./runqemu -a arm
Debug:
./runqemu -a arm -d
./run -a arm -d
# On another terminal.
./rungdb -a arm
@@ -25,7 +22,7 @@ TODOs:
## aarch64
./run -a aarch64
./build -a aarch64
TODOs:
@@ -39,4 +36,4 @@ TODOs:
## mips64
./run -a mips64
./build -a mips64

173
run
View File

@@ -1,77 +1,120 @@
#!/usr/bin/env bash
set -e
arch='x86_64'
extra_targets=''
j="$(($(nproc) - 2))"
x11=false
v=0
while getopts a:e:j:nt:v OPT; do
# CLI handling.
arch=x86_64
debug=false
debug_qemu=''
kgdb=false
nographic=false
# norandmaps: Don't use address space randomization. Equivalent to echo 0 > /proc/sys/kernel/randomize_va_space.
# printk.time=y: log in format: "[time ] msg" for all printk messages.
extra_append='norandmaps printk.devkmsg=on printk.time=y'
extra_flags=''
while getopts a:de:knqt:x OPT; do
case "$OPT" in
'a')
a)
arch="$OPTARG"
;;
'j')
j="$OPTARG"
;;
't')
extra_targets="$extra_args $OPTARG"
;;
'x')
x11=true
;;
'v')
v=1
;;
;;
d)
debug=true
extra_flags="$extra_flags -S -s"
;;
e)
extra_append="$extra_append $OPTARG"
;;
k)
debug=true
extra_append="$extra_append kgdbwait"
# For those who want to try KDB.
#extra_append="$extra_append kgdbwait kgdboc=kbd"
extra_flags="$extra_flags -serial tcp::1234,server,nowait"
kgdb=true
;;
n)
extra_append="$extra_append console=ttyS0"
extra_flags="$extra_flags -nographic"
nographic=true
;;
q)
debug_qemu='gdb -q -ex start --args'
;;
esac
done
shift "$(($OPTIND - 1))"
extra_flags="$extra_flags $@"
buildroot_out_dir="./buildroot/output.${arch}~"
images_dir="$buildroot_out_dir/images"
qemu_common="\
$debug_qemu \
$buildroot_out_dir/host/usr/bin/qemu-system-${arch} \
-m 128M \
-monitor telnet::45454,server,nowait \
-netdev user,hostfwd=tcp::45455-:45455,id=net0 \
-smp 1 \
"
# The base QEMU commands are found under board/qemu/*/readme.tx
case "$arch" in
'x86_64')
defconfig='qemu_x86_64_defconfig'
x86_64)
if $kgdb; then
extra_append="$extra_append kgdboc=ttyS0,115200"
fi
cmd="$qemu_common \
-M pc \
-append 'root=/dev/vda nopat $extra_append' \
-device edu \
-device lkmc_pci_min \
-device virtio-net-pci,netdev=net0 \
-drive file=${images_dir}/rootfs.ext2,if=virtio,format=raw \
-kernel ${images_dir}/bzImage \
$extra_flags \
"
;;
'arm')
# qemu_arm_vexpress_defconfig required a newer QEMU than 2.0.0 on a Ubuntu host.
# so let's stick to versatile for now.
defconfig='qemu_arm_versatile_defconfig'
arm)
if $kgdb; then
extra_append="$extra_append kgdboc=ttyAMA0,115200"
fi
cmd="$qemu_common \
-M versatilepb \
-append 'root=/dev/sda $extra_append' \
-device rtl8139,netdev=net0 \
-drive file=${images_dir}/rootfs.ext2,if=scsi,format=raw \
-dtb ${images_dir}/versatile-pb.dtb \
-kernel ${images_dir}/zImage \
-serial stdio \
$extra_flags \
"
;;
'aarch64')
defconfig='qemu_aarch64_virt_defconfig'
aarch64)
if $kgdb; then
extra_append="$extra_append kgdboc=ttyAMA0,115200"
fi
cmd="$qemu_common \
-M virt \
-append 'root=/dev/sda $extra_append' \
-cpu cortex-a57 \
-device virtio-net-device,netdev=net0 \
-drive file=${images_dir}/rootfs.cpio,if=scsi,format=raw \
-kernel ${images_dir}/Image \
-nographic \
-serial stdio \
$extra_flags \
"
;;
'mips64')
defconfig='qemu_mips64r6_malta_defconfig'
mips64)
cmd="$qemu_common \
-M malta \
-append 'root=/dev/hda $extra_append' \
-cpu I6400 \
-device pcnet \
-drive file=${images_dir}/rootfs.ext2,format=raw \
-kernel ${images_dir}/vmlinux \
-nographic \
$extra_flags \
"
;;
esac
cd kernel_module
./make-host.sh -j "$j" clean
cd ../buildroot
for p in $(find '../buildroot_patches/' -maxdepth 1 -name '*.patch' -print); do
patch -N -r - -p 1 <"$p" || :
done
outdir="output.${arch}~"
make O="$outdir" BR2_EXTERNAL="$(pwd)/../kernel_module" "$defconfig"
# TODO Can't get rid of this for now.
# http://stackoverflow.com/questions/44078245/is-it-possible-to-use-config-fragments-with-buildroots-config
cat ../buildroot_config_fragment >> "${outdir}/.config"
if $x11; then
cat ../buildroot_config_fragment_x11 >> "${outdir}/.config"
fi
make O="$outdir" olddefconfig
# HOST_QEMU_OPTS is a hack that happens to work because the QEMU package luckly uses += at all times.
# It shouldn't be necessary in the first place: https://bugs.busybox.net/show_bug.cgi?id=9936
#
# Even if were an autotools package, there is no general way currently to pass extra configs to it:
# https://stackoverflow.com/questions/44341188/how-to-pass-extra-custom-configure-autotools-options-to-a-buildroot-package/44341225#44341225
time \
env \
-u LD_LIBRARY_PATH \
make \
O="$outdir" \
BR2_JLEVEL="$j" \
HOST_QEMU_OPTS="--enable-debug --enable-sdl --extra-cflags='-DDEBUG_PL061=1' --with-sdlabi=2.0 --enable-trace-backends=simple" \
V="$v" \
kernel_module-rebuild \
$extra_targets \
all \
;
cd ..
./runqemu "$@"
echo "$cmd"
eval "$cmd"

120
runqemu
View File

@@ -1,120 +0,0 @@
#!/usr/bin/env bash
set -e
# CLI handling.
arch='x86_64'
debug=false
debug_qemu=''
kgdb=false
nographic=false
# norandmaps: Don't use address space randomization. Equivalent to echo 0 > /proc/sys/kernel/randomize_va_space.
# printk.time=y: log in format: "[time ] msg" for all printk messages.
extra_append='norandmaps printk.devkmsg=on printk.time=y'
extra_flags=''
while getopts a:de:knqt:x OPT; do
case "$OPT" in
a)
arch="$OPTARG"
;;
d)
debug=true
extra_flags="$extra_flags -S -s"
;;
e)
extra_append="$extra_append $OPTARG"
;;
k)
debug=true
extra_append="$extra_append kgdbwait"
# For those who want to try KDB.
#extra_append="$extra_append kgdbwait kgdboc=kbd"
extra_flags="$extra_flags -serial tcp::1234,server,nowait"
kgdb=true
;;
n)
extra_append="$extra_append console=ttyS0"
extra_flags="$extra_flags -nographic"
nographic=true
;;
q)
debug_qemu='gdb -q -ex start --args'
;;
esac
done
shift "$(($OPTIND - 1))"
extra_flags="$extra_flags $@"
buildroot_out_dir="./buildroot/output.${arch}~"
images_dir="$buildroot_out_dir/images"
qemu_common="\
$debug_qemu \
$buildroot_out_dir/host/usr/bin/qemu-system-${arch} \
-m 128M \
-monitor telnet::45454,server,nowait \
-netdev user,hostfwd=tcp::45455-:45455,id=net0 \
-smp 1 \
"
# The base QEMU commands are found under board/qemu/*/readme.tx
case "$arch" in
'x86_64')
if $kgdb; then
extra_append="$extra_append kgdboc=ttyS0,115200"
fi
cmd="$qemu_common \
-M pc \
-append 'root=/dev/vda nopat $extra_append' \
-device edu \
-device lkmc_pci_min \
-device virtio-net-pci,netdev=net0 \
-drive file=${images_dir}/rootfs.ext2,if=virtio,format=raw \
-kernel ${images_dir}/bzImage \
$extra_flags \
"
;;
'arm')
if $kgdb; then
extra_append="$extra_append kgdboc=ttyAMA0,115200"
fi
cmd="$qemu_common \
-M versatilepb \
-append 'root=/dev/sda $extra_append' \
-device rtl8139,netdev=net0 \
-drive file=${images_dir}/rootfs.ext2,if=scsi,format=raw \
-dtb ${images_dir}/versatile-pb.dtb \
-kernel ${images_dir}/zImage \
-serial stdio \
$extra_flags \
"
;;
'aarch64')
if $kgdb; then
extra_append="$extra_append kgdboc=ttyAMA0,115200"
fi
cmd="$qemu_common \
-M virt \
-append 'root=/dev/sda $extra_append' \
-cpu cortex-a57 \
-device virtio-net-device,netdev=net0 \
-drive file=${images_dir}/rootfs.cpio,if=scsi,format=raw \
-kernel ${images_dir}/Image \
-nographic \
-serial stdio \
$extra_flags \
"
;;
'mips64')
cmd="$qemu_common \
-M malta \
-append 'root=/dev/hda $extra_append' \
-cpu I6400 \
-device pcnet \
-drive file=${images_dir}/rootfs.ext2,format=raw \
-kernel ${images_dir}/vmlinux \
-nographic \
$extra_flags \
"
;;
esac
echo "$cmd"
eval "$cmd"

5
x11.md
View File

@@ -4,9 +4,10 @@ Only tested successfully in `x86_64`.
Build:
./run -x
./build -x
./run
We don't build X11 by default because it takes a considerable amount of time (~20%), and is not expected to be used by most users.
We don't build X11 by default because it takes a considerable amount of time (~20%), and is not expected to be used by most users: you need to pass the `-x` flag to enable it.
Inside QEMU: