3rd-party/linux-kernel-module-cheat
1
0
Fork 0
mirror of https://github.com/cirosantilli/linux-kernel-module-cheat.git synced 2026-01-23 02:05:57 +01:00
Code Issues Releases Wiki Activity

split --prebuilt and --host into --gcc-which and --qemu-which

Browse Source 
Only one --host exists at ./build-modules, since that can select the host
kernel, which is independent from the toolchain.

Document that user mode simulation stopped working.
This commit is contained in:
Ciro Santilli 六四事件 法轮功
2019-02-16 00:00:00 +00:00
parent 01194dda5c
commit a8b6f758ba
10 changed files with 294 additions and 142 deletions
Download Patch File Download Diff File Expand all files Collapse all files

187
README.adoc
View File

@@ -533,13 +533,17 @@ cd linux-kernel-module-cheat
git checkout "$(git rev-list --tags --max-count=1)"
./release-download-latest
unzip lkmc-*.zip
./run --prebuilt
./run --qemu-which host
....
Or to run a baremetal example instead:
....
./run --arch aarch64 --baremetal baremetal/hello.c --prebuilt
./run \
--arch aarch64 \
--baremetal baremetal/hello.c \
--qemu-which host \
;
....
You have to checkout to the latest tag to ensure that the scripts match the release format: https://stackoverflow.com/questions/1404796/how-to-get-the-latest-tag-name-in-current-branch-in-git
@@ -558,7 +562,7 @@ To build the kernel modules as in <<your-first-kernel-module-hack>> do:
....
git submodule update --depth 1 --init --recursive "$(./getvar linux_source_dir)"
./build-linux --no-modules-install -- modules_prepare
./build-modules
./build-modules --gcc-which host
./run
....
@@ -567,7 +571,7 @@ TODO: for now the only way to test those modules out without <<qemu-buildroot-se
Command explanation:
* `modules_prepare` does the minimal build procedure required on the kernel for us to be able to compile the kernel modules, and is way faster than doing a full kernel build. A full kernel build would also work however.
* the `./build-modules` command automatically falls back to the Ubuntu packaged GCC since you don't have the Buildroot toolchain
* `--gcc-which host` selects your host Ubuntu packaged GCC, since you don't have the Buildroot toolchain
* `--no-modules-install` is required otherwise the `make modules_install` target we run by default fails, since the kernel wasn't built
To modify the Linux kernel, build and use it as usual:
@@ -585,7 +589,7 @@ For gem5, do:
git submodule update --init --depth 1 "$(./getvar linux_source_dir)"
sudo apt-get install qemu-utils
./build-gem5
./run --emulator gem5 --prebuilt
./run --emulator gem5 --qemu-which host
....
`qemu-utils` is required because we currently distribute `.qcow2` files which <<gem5-qcow2,gem5 can't handle>>, so we need `qemu-img` to extract them first.
@@ -653,21 +657,27 @@ It has however severe limitations:
Still interested?
....
./build-modules --host
./build-modules --gcc-which host --host
....
Compilation will likely fail for some modules because of kernel or toolchain differences that we can't control on the host.
The best solution is to compile just your modules with:
The best workaround is to compile just your modules with:
....
./build-modules --host -- hello hello2
./build-modules --gcc-which host --host -- hello hello2
....
which is equivalent to:
....
./build-modules --host -- kernel_modules/hello.c kernel_modules/hello2.c
./build-modules \
--gcc-which host \
--host \
-- \
kernel_modules/hello.c \
kernel_modules/hello2.c \
;
....
Or just remove the `.c` extension from the failing files and try again:
@@ -874,7 +884,7 @@ Much like <<baremetal-setup>>, this is another fun setup that does not require B
Introduction at: <<user-mode-simulation>>.
Getting started at: <<qemu-user-mode>>.
Getting started at: <<qemu-user-mode-getting-started>>.
[[gdb]]
== GDB step debug
@@ -2814,8 +2824,8 @@ Files that contain device trees have the `.dtb` extension when compiled, and `.d
You can convert between those formats with:
....
"$(./getvar host_dir)"/bin/dtc -I dtb -O dts -o a.dts a.dtb
"$(./getvar host_dir)"/bin/dtc -I dts -O dtb -o a.dtb a.dts
"$(./getvar buildroot_host_dir)"/bin/dtc -I dtb -O dts -o a.dts a.dtb
"$(./getvar buildroot_host_dir)"/bin/dtc -I dts -O dtb -o a.dtb a.dts
....
Buildroot builds the tool due to `BR2_PACKAGE_HOST_DTC=y`.
@@ -3021,7 +3031,7 @@ TODO: do the right thing and cross compile QEMU and gem5. gem5's Python parts mi
Both QEMU and gem5 have an user mode simulation mode in addition to full system simulation that we consider elsewhere in this project.
In QEMU, it is called just <<qemu-user-mode,"user mode">>, and in gem5 it is called <<gem5-syscall-emulation-mode,syscall emulation mode>>.
In QEMU, it is called just <<qemu-user-mode-getting-started,"user mode">>, and in gem5 it is called <<gem5-syscall-emulation-mode,syscall emulation mode>>.
In both, the basic idea is the same.
@@ -3043,16 +3053,27 @@ Disadvantages:
It may still work even if that is not the case, but could fail is a missing system call is reached.
+
The target Linux kernel of the executable is a GCC toolchain build-time configuration.
* cannot be used to test the Linux kernel, and results are less representative of a real system since we are faking more
** emulator implementers have to keep up with libc changes, some of which break even a C hello world due setup code executed before main.
+
See also: <<user-mode-simulation-with-glibc>>
* cannot be used to test the Linux kernel or any devices, and results are less representative of a real system since we are faking more
=== QEMU user mode
=== QEMU user mode getting started
First let's run a dynamically linked executable built with the Buildroot toolchain:
TODO: at 125d14805f769104f93c510bedaa685a52ec025d we <<libc-choice,moved Buildroot from uClibc to glibc>>, and this broke running user mode executables built with Buildroot in many setups, which would be ideal setup. Fixing them requires hacking up the emulators / glibc, but I'm lazy now. More details at: <<user-mode-simulation-with-glibc>>.
Running with Ubuntu 18.04 host packaged toolchains works however, as emulator maintainers have had more time to iron out the issues there, so let's go that route for now:
....
./build --arch aarch64 user-mode-qemu
sudo apt-get install gcc-aarch64-linux-gnu
./build-userland \
--arch aarch64 \
--gcc-which host \
--userland-build-id host \
;
./run \
--arch aarch64 \
--userland-build-id host \
--userland print_argv \
--userland-args 'asdf "qw er"' \
;
@@ -3067,15 +3088,98 @@ qw er
This runs link:userland/print_argv.c[].
All further sections assume that you are using the host toolchain unless otherwise noted:
* `--gcc-which host`: use the host toolchain.
+
We must pass this to `./run` as well because QEMU must know which dynamic libraries to use. See also: <<user-mode-static-executables>>.
* `--userland-build-id host`: put the host built into a <<build-variants>>
In order to build and run with the Buildroot toolchain to try and fix the emulators, do:
....
./build --arch aarch64 user-mode-qemu
./run \
--arch aarch64 \
--userland print_argv \
--userland-args 'asdf "qw er"' \
;
....
where `build` builds the whole toolchain for us.
`./run --userland` path resolution is analogous to <<baremetal-setup-getting-started,that of `./run --baremetal`>>.
`./build-userland` is further documented at: <<userland-directory>>.
`./build user-mode-qemu` first builds Buildroot, and then runs `./build-userland`, which is further documented at: <<userland-directory>>.
==== User mode simulation with glibc
===== FATAL: kernel too old
Happens on all gem5 setups, but not on QEMU.
glibc has a check for kernel version, likely obtained from the `uname` syscall, and if the kernel is not new enough, it quits.
Determining the right number to put there is of course highly non-trivial and would require an extensive userland testsuite, which most emulator don't have.
We don't have this failure for QEMU, only gem5. QEMU by default copies the host `uname`, but it also has the `-r` option to set it explicitly, try it out with:
....
./run --arch aarch64 --userland uname -- -r v4.17.0
....
Source: link:userland/uname.c[].
The QEMU source that does this is at: https://github.com/qemu/qemu/blob/v3.1.0/linux-user/syscall.c#L8931
In gem5, there are tons of missing syscalls, and that number currently just gets bumped up randomly from time to time when someone gets fed up:
* https://stackoverflow.com/questions/48959349/how-to-solve-fatal-kernel-too-old-when-running-gem5-in-syscall-emulation-se-m
* https://stackoverflow.com/questions/53085048/how-to-compile-and-run-an-executable-in-gem5-syscall-emulation-mode-with-se-py/53085049#53085049
* https://gem5-review.googlesource.com/c/public/gem5/+/15855
The ID is just hardcoded on the source:
===== stack smashing detected
Reproduction:
....
./run --userland hello
./run --userland hello --qemu-which host
....
Outcome:
....
*** stack smashing detected ***: <unknown> terminated
qemu: uncaught target signal 6 (Aborted) - core dumped
....
The following all work however:
....
./run --arch aarch64 --userland hello
./run --static --userland hello
....
A non-QEMU example of stack smashing is shown at: https://stackoverflow.com/questions/1345670/stack-smashing-detected/51897264#51897264
Related bug reports:
* https://bugs.launchpad.net/qemu/+bug/1701808
* https://bugs.launchpad.net/qemu/+bug/1776478
* https://github.com/multiarch/ubuntu-debootstrap/issues/10
Tested at: 2e32389ebf1bedd89c682aa7b8fe42c3c0cf96e5 + 1.
==== User mode static executables
Running dynamically linked executables in QEMU requires pointing it to the root filesystem with the `-L` option so that it can find the dynamic linker and shared libraries.
We pass `-L` by default, so everything just works:
We pass `-L` by default, so everything just works.
You can also try statically linked executables with:
However, in case something goes wrong, you can also try statically linked executables with:
....
./build-userland \
@@ -3090,23 +3194,7 @@ You can also try statically linked executables with:
;
....
Or you can run statically linked built by the host packaged toolchain with:
....
./build-userland \
--arch aarch64 \
--host \
--static \
;
./run \
--arch aarch64 \
--static \
--userland print_argv \
--userland-args 'asdf "qw er"' \
;
....
TODO expose dynamically linked executables built by the host toolchain. It also works, we just have to use e.g. `-L /usr/aarch64-linux-gnu`, so it's not really hard, I'm just lazy.
gem5 user mode currently only supports static executables: <<gem5-syscall-emulation-mode>>.
==== User mode GDB
@@ -4348,7 +4436,7 @@ although this can be useful when someone gives you a random image.
By default, link:build-linux[] generates a `.config` that is a mixture of:
* a base config extracted from Buildroot's minimal per machine `.config`, which has the minimal options needed to boot: <<about-buildroot-s-kernel-configs>>
* a base config extracted from Buildroot's minimal per machine `.config`, which has the minimal options needed to boot: <<about-buildroots-kernel-configs>>
* small overlays put top of that
To find out which kernel configs are being used exactly, simply run:
@@ -6588,6 +6676,7 @@ Yes!!! We read the correct value from the physical address.
We could not find however to write to memory from the QEMU monitor, boring.
[[dev-mem]]
====== /dev/mem
`/dev/mem` exposes access to physical addresses, and we use it through the convenient `devmem` BusyBox utility.
@@ -8090,7 +8179,7 @@ Snapshots are stored inside the `.qcow2` images themselves.
They can be observed with:
....
"$(./getvar host_dir)/bin/qemu-img" info "$(./getvar qcow2_file)"
"$(./getvar buildroot_host_dir)/bin/qemu-img" info "$(./getvar qcow2_file)"
....
which after `savevm my_snap_id` and `savevm asdf` contains an output of type:
@@ -10840,7 +10929,7 @@ For Buildroot problems, you should wither provide the config you have:
or try to reproduce with a minimal config, see: https://github.com/cirosantilli/buildroot/tree/in-tree-package-master
== libc choice
=== libc choice
Buildroot supports several libc implementations, including:
@@ -10864,6 +10953,8 @@ The full list of unsupported packages can be found by grepping the Buildroot sou
git -C "$(./getvar buildroot_source_dir)" grep 'depends on BR2_TOOLCHAIN_USES_GLIBC'
....
One "downside" of glibc is that it exercises much more kernel functionality on its more bloated pre-main init, which breaks user mode C hello worlds more often, see: <<user-mode-simulation-with-glibc>>. I quote "downside" because glibc is actually exposing emulator bugs which we should actually go and fix.
== Baremetal
Getting started at: <<baremetal-setup>>
@@ -11037,8 +11128,8 @@ For `arm`, some baremetal examples compile fine with:
....
sudo apt-get install gcc-arm-none-eabi qemu-system-arm
./build-baremetal --arch arm --prebuilt
./run --arch arm --baremetal interactive/prompt --prebuilt
./build-baremetal --arch arm --gcc-which host-baremetal
./run --arch arm --baremetal interactive/prompt --qemu-which host
....
However, there are as usual limitations to using prebuilts:
@@ -11228,7 +11319,7 @@ Source: link:baremetal/arch/aarch64/svc.c[]
The vector table format is described on <<armarm8>> Table D1-7 "Vector offsets from vector table base address".
A good representation of the format of the vector table can also be found at <<programmer-s-guide-for-armv8-a>> Table 10-2 "Vector table offsets from vector table base address".
A good representation of the format of the vector table can also be found at <<programmers-guide-for-armv8-a>> Table 10-2 "Vector table offsets from vector table base address".
The first part of the table contains:
@@ -12666,14 +12757,14 @@ make clean
or more cleanly out of tree:
....
./build-userland --host --userland-build-id host
./build-userland --gcc-which host --userland-build-id host
"$(./getvar --userland-build-id host userland_build_dir)/hello.out"
....
Extra make flags may be passed as:
....
./build-userland --host --userland-build-id host-static --make-args='-B CFLAGS_EXTRA=-static'
./build-userland --gcc-which host --userland-build-id host-static --make-args='-B CFLAGS_EXTRA=-static'
"$(./getvar --userland-build-id host-static userland_build_dir)/hello.out"
....
@@ -12946,9 +13037,9 @@ This magic output string is notably used by:
* the `common_assert_fail()` function, which is used by <<baremetal-tests>>
* link:rootfs_overlay/test_fail.sh[], which is used by <<test-userland-in-full-system>>
=== Non-automated tests
==== Non-automated tests
==== Test GDB Linux kernel
===== Test GDB Linux kernel
For the Linux kernel, do the following manual tests for now.
@@ -12971,7 +13062,7 @@ Then proceed to do the following tests:
* `/count.sh` and `break __x64_sys_write`
* `insmod /timer.ko` and `break lkmc_timer_callback`
==== Test the Internet
===== Test the Internet
You should also test that the Internet works:

9
build-baremetal
View File

@@ -7,6 +7,9 @@ from shell_helpers import LF
class Main(common.BuildCliFunction):
def __init__(self):
super().__init__(
defaults={
'gcc_which':'crosstool-ng',
},
description='''\
Build the baremetal examples with crosstool-NG.
''',
@@ -31,11 +34,7 @@ Build the baremetal examples with crosstool-NG.
'-mcpu={}'.format(self.env['mcpu']), LF,
'-nostartfiles', LF,
]
if self.env['prebuilt']:
gcc = 'arm-none-eabi-gcc'
else:
os.environ['PATH'] = self.env['crosstool_ng_bin_dir'] + os.environ['PATH']
gcc = self.get_toolchain_tool('gcc', allowed_toolchains=['crosstool-ng'])
gcc = self.get_toolchain_tool('gcc')
if self.env['emulator'] == 'gem5':
if self.env['machine'] == 'VExpress_GEM5_V1':
entry_address = 0x80000000

4
build-m5
View File

@@ -8,8 +8,8 @@ from shell_helpers import LF
class Main(common.BuildCliFunction):
def _get_make_cmd(self):
allowed_toolchains = ['buildroot']
cc = self.get_toolchain_tool('gcc', allowed_toolchains=allowed_toolchains)
ld = self.get_toolchain_tool('ld', allowed_toolchains=allowed_toolchains)
cc = self.get_toolchain_tool('gcc')
ld = self.get_toolchain_tool('ld')
if self.env['arch'] == 'x86_64':
arch = 'x86'
else:

10
build-modules
View File

@@ -24,8 +24,8 @@ See also: https://github.com/cirosantilli/linux-kernel-module-cheat#host
'--host',
default=False,
help='''\
Build the Linux kernel modules for the host instead of guest.
Use the host packaged cross toolchain.
Build the Linux kernel modules against the host kernel.
Place the modules on a separate magic directory from non --host builds.
''',
)
self.add_argument(
@@ -68,14 +68,12 @@ Use the host packaged cross toolchain.
else:
kernel_modules = map(lambda x: os.path.splitext(os.path.split(x)[1])[0], self.env['kernel_modules'])
object_files = map(lambda x: x + self.env['obj_ext'], kernel_modules)
tool = 'gcc'
if self.env['host']:
allowed_toolchains = ['host']
build_subdir = self.env['kernel_modules_build_host_subdir']
else:
allowed_toolchains = None
build_subdir = self.env['kernel_modules_build_subdir']
gcc = self.get_toolchain_tool(tool, allowed_toolchains=allowed_toolchains)
tool = 'gcc'
gcc = self.get_toolchain_tool(tool)
prefix = gcc[:-len(tool)]
ccache = shutil.which('ccache')
if ccache is not None:

24
build-userland
View File

@@ -49,12 +49,6 @@ has the OpenBLAS libraries and headers installed.
def build(self):
build_dir = self.get_build_dir()
os.makedirs(build_dir, exist_ok=True)
if self.env['host']:
allowed_toolchains = ['host']
else:
allowed_toolchains = ['buildroot']
cc = self.get_toolchain_tool('gcc', allowed_toolchains=allowed_toolchains)
cxx = self.get_toolchain_tool('g++', allowed_toolchains=allowed_toolchains)
make_args = shlex.split(self.env['make_args'])
if self.env['static']:
make_args.extend(['CCFLAGS_EXTRA=-static', LF])
@@ -66,15 +60,25 @@ has the OpenBLAS libraries and headers installed.
'ARCH={}'.format(self.env['arch']), LF,
'CCFLAGS_SCRIPT={} {}'.format('-I', self.env['userland_source_dir']), LF,
'COMMON_DIR={}'.format(self.env['root_dir']), LF,
'CC={}'.format(cc), LF,
'CXX={}'.format(cxx), LF,
'CC={}'.format(self.get_toolchain_tool('gcc')), LF,
'CXX={}'.format(self.get_toolchain_tool('g++')), LF,
'PKG_CONFIG={}'.format(self.env['buildroot_pkg_config']), LF,
'STAGING_DIR={}'.format(self.env['buildroot_staging_dir']), LF,
'OUT_DIR={}'.format(build_dir), LF,
] +
self.sh.add_newlines(['HAS_{}=y'.format(package.upper()) for package in self.env['has_package']]) +
self.sh.add_newlines([
'HAS_{}=y'.format(package.upper())
for package in
self.env['has_package']
]) +
make_args +
self.sh.add_newlines([os.path.join(build_dir, os.path.splitext(os.path.split(target)[1])[0]) + self.env['userland_build_ext'] for target in self.env['targets']])
self.sh.add_newlines([
os.path.join(
build_dir,
os.path.splitext(os.path.split(target)[1])[0]
) + self.env['userland_build_ext']
for target in self.env['targets']
])
),
cwd=self.env['userland_source_dir'],
extra_paths=[self.env['ccache_dir']],

156
common.py
View File

@@ -141,7 +141,9 @@ class LkmcCliFunction(cli_function.CliFunction):
arches_string.append('{} ({})'.format(arch_long, arch_short))
arches_string = ', '.join(arches_string)
self.add_argument(
'-A', '--all-archs', default=False,
'-A',
'--all-archs',
default=False,
help='''\
Run action for all supported --archs archs. Ignore --archs.
'''.format(arches_string)
@@ -180,14 +182,34 @@ mkdir are generally omitted since those are obvious
'''
)
self.add_argument(
'--print-time', default=True,
'--gcc-which',
choices=[
'buildroot',
'crosstool-ng',
'host',
'host-baremetal'
],
default='buildroot',
help='''\
Which toolchain binaries to use:
- buildroot: the ones we built with ./build-buildroot. For userland, links to glibc.
- crosstool-ng: the ones we built with ./build-crosstool-ng. For baremetal, links to newlib.
- host: the host distro pre-packaged userland ones. For userland, links to glibc.
- host-baremetal: the host distro pre-packaged bare one. For baremetal, links to newlib.
'''
)
self.add_argument(
'--print-time',
default=True,
help='''\
Print how long it took to run the command at the end.
Implied by --quiet.
'''
)
self.add_argument(
'-q', '--quiet', default=False,
'-q',
'--quiet',
default=False,
help='''\
Don't print anything to stdout, except if it is part of an interactive terminal.
TODO: implement fully, some stuff is escaping it currently.
@@ -201,7 +223,9 @@ Stop running at the first failed test.
'''
)
self.add_argument(
'-v', '--verbose', default=False,
'-v',
'--verbose',
default=False,
help='Show full compilation commands when they are not shown by default.'
)
@@ -218,14 +242,16 @@ Ignore --gem5-build-id and --gem5-build-type.
'''
)
self.add_argument(
'-M', '--gem5-build-id',
'-M',
'--gem5-build-id',
help='''\
gem5 build ID. Allows you to keep multiple separate gem5 builds.
Default: {}
'''.format(consts['default_build_id'])
)
self.add_argument(
'--gem5-build-type', default='opt',
'--gem5-build-type',
default='opt',
help='gem5 build type, most often used for "debug" builds.'
)
self.add_argument(
@@ -235,7 +261,8 @@ Use the given directory as the gem5 source tree. Ignore `--gem5-worktree`.
'''
)
self.add_argument(
'-N', '--gem5-worktree',
'-N',
'--gem5-worktree',
help='''\
Create and use a git worktree of the gem5 submodule.
See: https://github.com/cirosantilli/linux-kernel-module-cheat#gem5-worktree
@@ -250,7 +277,9 @@ Use the given directory as the Linux build directory. Ignore --linux-build-id.
'''
)
self.add_argument(
'-L', '--linux-build-id', default=consts['default_build_id'],
'-L',
'--linux-build-id',
default=consts['default_build_id'],
help='''\
Linux build ID. Allows you to keep multiple separate Linux builds.
'''
@@ -280,7 +309,8 @@ See: https://github.com/cirosantilli/linux-kernel-module-cheat#initrd
# Baremetal.
self.add_argument(
'-b', '--baremetal',
'-b',
'--baremetal',
help='''\
Use the given baremetal executable instead of the Linux kernel.
@@ -298,13 +328,16 @@ inside baremetal/ and then try to use corresponding executable.
help='Buildroot build ID. Allows you to keep multiple separate gem5 builds.'
)
self.add_argument(
'--buildroot-linux', default=False,
'--buildroot-linux',
default=False,
help='Boot with the Buildroot Linux kernel instead of our custom built one. Mostly for sanity checks.'
)
# Android.
self.add_argument(
'--rootfs-type', default='buildroot', choices=('buildroot', 'android'),
'--rootfs-type',
default='buildroot',
choices=('buildroot', 'android'),
help='Which rootfs to use.'
)
self.add_argument(
@@ -322,11 +355,13 @@ of SSD.
# crosstool-ng
self.add_argument(
'--crosstool-ng-build-id', default=consts['default_build_id'],
'--crosstool-ng-build-id',
default=consts['default_build_id'],
help='Crosstool-NG build ID. Allows you to keep multiple separate crosstool-NG builds.'
)
self.add_argument(
'--docker', default=False,
'--docker',
default=False,
help='''\
Use the docker download Ubuntu root filesystem instead of the default Buildroot one.
'''
@@ -334,9 +369,21 @@ Use the docker download Ubuntu root filesystem instead of the default Buildroot
# QEMU.
self.add_argument(
'-Q', '--qemu-build-id', default=consts['default_build_id'],
'-Q',
'--qemu-build-id',
default=consts['default_build_id'],
help='QEMU build ID. Allows you to keep multiple separate QEMU builds.'
)
self.add_argument(
'--qemu-which',
choices=['lkmc', 'host'],
default='lkmc',
help='''\
Which qemu binaries to use: qemu-system-, qemu-, qemu-img, etc.:
- lkmc: the ones we built with ./build-qemu
- host: the host distro pre-packaged provided ones
'''
)
self.add_argument(
'--machine',
help='''\
@@ -513,9 +560,9 @@ Valid emulators: {}
env['buildroot_build_build_dir'] = join(env['buildroot_build_dir'], 'build')
env['buildroot_linux_build_dir'] = join(env['buildroot_build_build_dir'], 'linux-custom')
env['buildroot_vmlinux'] = join(env['buildroot_linux_build_dir'], 'vmlinux')
env['host_dir'] = join(env['buildroot_build_dir'], 'host')
env['host_bin_dir'] = join(env['host_dir'], 'usr', 'bin')
env['buildroot_pkg_config'] = join(env['host_bin_dir'], 'pkg-config')
env['buildroot_host_dir'] = join(env['buildroot_build_dir'], 'host')
env['buildroot_host_bin_dir'] = join(env['buildroot_host_dir'], 'usr', 'bin')
env['buildroot_pkg_config'] = join(env['buildroot_host_bin_dir'], 'pkg-config')
env['buildroot_images_dir'] = join(env['buildroot_build_dir'], 'images')
env['buildroot_rootfs_raw_file'] = join(env['buildroot_images_dir'], 'rootfs.ext2')
env['buildroot_qcow2_file'] = env['buildroot_rootfs_raw_file'] + '.qcow2'
@@ -737,6 +784,37 @@ Valid emulators: {}
lunch aosp_{}-eng
'''.format(self.env['android_arch'])
# Toolchain choice.
if not env['_args_given']['gcc_which']:
if env['baremetal']:
env['gcc_which'] = 'crosstool-ng'
if env['gcc_which'] == 'buildroot':
env['toolchain_prefix'] = os.path.join(
env['buildroot_host_bin_dir'],
env['buildroot_toolchain_prefix']
)
env['userland_library_dir'] = env['target_dir']
elif env['gcc_which'] == 'crosstool-ng':
env['toolchain_prefix'] = os.path.join(
env['crosstool_ng_bin_dir'],
env['crosstool_ng_toolchain_prefix']
)
elif env['gcc_which'] == 'host':
env['toolchain_prefix'] = env['ubuntu_toolchain_prefix']
if env['arch'] == 'x86_64':
env['userland_library_dir'] = '/'
elif env['arch'] == 'arm':
env['userland_library_dir'] = '/usr/arm-linux-gnueabihf'
elif env['arch'] == 'aarch64':
env['userland_library_dir'] = '/usr/aarch64-linux-gnu/'
elif env['gcc_which'] == 'host-baremetal':
if env['arch'] == 'arm':
env['toolchain_prefix'] = 'arm-none-eabi'
else:
raise Exception('There is no host baremetal chain for arch: ' + env['arch'])
else:
raise Exception('Unknown toolchain: ' + env['gcc_which'])
def add_argument(self, *args, **kwargs):
'''
Also handle:
@@ -805,44 +883,8 @@ lunch aosp_{}-eng
ret.append(cols[1])
return ret
def get_toolchain_prefix(self, tool, allowed_toolchains=None):
buildroot_full_prefix = os.path.join(self.env['host_bin_dir'], self.env['buildroot_toolchain_prefix'])
buildroot_exists = os.path.exists('{}-{}'.format(buildroot_full_prefix, tool))
crosstool_ng_full_prefix = os.path.join(self.env['crosstool_ng_bin_dir'], self.env['crosstool_ng_toolchain_prefix'])
crosstool_ng_exists = os.path.exists('{}-{}'.format(crosstool_ng_full_prefix, tool))
host_tool = '{}-{}'.format(self.env['ubuntu_toolchain_prefix'], tool)
host_path = shutil.which(host_tool)
if host_path is not None:
host_exists = True
host_full_prefix = host_path[:-(len(tool)+1)]
else:
host_exists = False
host_full_prefix = None
known_toolchains = {
'crosstool-ng': (crosstool_ng_exists, crosstool_ng_full_prefix),
'buildroot': (buildroot_exists, buildroot_full_prefix),
'host': (host_exists, host_full_prefix),
}
if allowed_toolchains is None:
if self.env['baremetal'] is None:
allowed_toolchains = ['buildroot', 'crosstool-ng', 'host']
else:
allowed_toolchains = ['crosstool-ng', 'buildroot', 'host']
tried = []
for toolchain in allowed_toolchains:
exists, prefix = known_toolchains[toolchain]
tried.append('{}-{}'.format(prefix, tool))
if exists:
return prefix
error_message = 'Tool not found. Tried:\n' + '\n'.join(tried)
if self.env['dry_run']:
self.log_error(error_message)
return ''
else:
raise Exception(error_message)
def get_toolchain_tool(self, tool, allowed_toolchains=None):
return '{}-{}'.format(self.get_toolchain_prefix(tool, allowed_toolchains), tool)
def get_toolchain_tool(self, tool):
return '{}-{}'.format(self.env['toolchain_prefix'], tool)
def github_make_request(
self,
@@ -1010,8 +1052,8 @@ lunch aosp_{}-eng
if self.env['print_time'] and not self.env['quiet']:
print('time {}'.format(self.seconds_to_hms(ellapsed_seconds)))
def raw_to_qcow2(self, prebuilt=False, reverse=False):
if prebuilt or not os.path.exists(self.env['qemu_img_executable']):
def raw_to_qcow2(self, qemu_which=False, reverse=False):
if qemu_which == 'host' or not os.path.exists(self.env['qemu_img_executable']):
disable_trace = []
qemu_img_executable = self.env['qemu_img_basename']
else:

18
run
View File

@@ -99,7 +99,9 @@ gem.op5 --debug-flags=Exec fs.py --cpu-type=HPI --caches
'''
)
self.add_argument(
'--gem5-script', default='fs', choices=['fs', 'biglittle'],
'--gem5-script',
default='fs',
choices=['fs', 'biglittle'],
help='Which gem5 script to use'
)
self.add_argument(
@@ -299,7 +301,7 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
if not os.path.exists(self.env['rootfs_raw_file']):
if not os.path.exists(self.env['qcow2_file']):
raise_rootfs_not_found()
self.raw_to_qcow2(prebuilt=self.env['prebuilt'], reverse=True)
self.raw_to_qcow2(qemu_which=self.env['qemu_which'], reverse=True)
else:
if not os.path.exists(self.env['gem5_fake_iso']):
os.makedirs(os.path.dirname(self.env['gem5_fake_iso']), exist_ok=True)
@@ -420,7 +422,7 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
cmd.extend(
[
os.path.join(self.env['qemu_build_dir'], '{}-linux-user'.format(self.env['arch']), 'qemu-{}'.format(self.env['arch'])), LF,
'-L', self.env['target_dir'], LF
'-L', self.env['userland_library_dir'], LF
] +
qemu_user_and_system_options +
debug_args
@@ -430,12 +432,12 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
raise_image_not_found()
extra_emulator_args.extend(extra_qemu_args)
self.make_run_dirs()
if self.env['prebuilt'] or not os.path.exists(self.env['qemu_executable']):
if self.env['qemu_which'] == 'host' or not os.path.exists(self.env['qemu_executable']):
qemu_executable = self.env['qemu_executable_basename']
qemu_executable_prebuilt = True
qemu_executable_host = True
else:
qemu_executable = self.env['qemu_executable']
qemu_executable_prebuilt = False
qemu_executable_host = False
qemu_executable = shutil.which(qemu_executable)
if qemu_executable is None:
raise Exception('QEMU executable not found, did you forget to build or install it?\n' \
@@ -500,7 +502,7 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
)
if self.env['dtb'] is not None:
cmd.extend(['-dtb', self.env['dtb'], LF])
if not qemu_executable_prebuilt:
if not qemu_executable_host:
cmd.extend(qemu_user_and_system_options)
if self.env['initrd']:
extra_emulator_args.extend(['-initrd', self.env['buildroot_cpio'], LF])
@@ -523,7 +525,7 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
if not os.path.exists(self.env['qcow2_file']):
if not os.path.exists(self.env['rootfs_raw_file']):
raise_rootfs_not_found()
self.raw_to_qcow2(prebuilt=self.env['prebuilt'])
self.raw_to_qcow2(qemu_which=self.env['qemu_which'])
extra_emulator_args.extend([
'-drive',
'file={},format=qcow2,if={}{}{}'.format(self.env['disk_image'], driveif, snapshot, rrid),

6
run-gdb
View File

@@ -152,12 +152,8 @@ See: https://github.com/cirosantilli/linux-kernel-module-cheat#gdb-builtin-cpu-s
test_script_path = os.path.splitext(self.env['source_path'])[0] + '.py'
else:
image = self.env['vmlinux']
if self.env['baremetal']:
allowed_toolchains = ['crosstool-ng', 'buildroot', 'host']
else:
allowed_toolchains = ['buildroot', 'crosstool-ng', 'host']
cmd = (
[self.get_toolchain_tool('gdb', allowed_toolchains=allowed_toolchains), LF] +
[self.get_toolchain_tool('gdb'), LF] +
before
)
if linux_full_system:

2
run-toolchain
View File

@@ -18,7 +18,7 @@ For example, to get some information about the arm vmlinux:
Get the list of available tools with:
....
ls "$(./getvar -a arm host_bin_dir)"
ls "$(./getvar -a arm buildroot_host_bin_dir)"
....
'''
})

20
userland/uname.c Normal file
View File

@@ -0,0 +1,20 @@
/* https://github.com/cirosantilli/linux-kernel-module-cheat#fatal-kernel-too-old */
#define _XOPEN_SOURCE 700
#include <stdio.h>
#include <stdlib.h>
#include <sys/utsname.h>
int main(void) {
struct utsname info;
if (uname(&info) == -1) {
perror("uname");
exit(EXIT_FAILURE);
}
printf("sysname = %s\n", info.sysname );
printf("nodename = %s\n", info.nodename);
printf("release = %s\n", info.release );
printf("version = %s\n", info.version );
printf("machine = %s\n", info.machine );
return EXIT_SUCCESS;
}