3rd-party/linux-kernel-module-cheat
1
0
Fork 0
mirror of https://github.com/cirosantilli/linux-kernel-module-cheat.git synced 2026-01-23 02:05:57 +01:00
Code Issues Releases Wiki Activity

Make this repo good enough to move in cpp-cheat, x86-assembly-cheat and arm-assembly-cheat in

Browse Source 
This commit is a large squash, the full development branch is available at:
https://github.com/cirosantilli/linux-kernel-module-cheat/tree/asm

This notably means a refactor of the userland build and testing, to support:

- improved assembly infrastructure unified across arm and x86
- native in-tree build and test helpers
- parallel building and testing, which implies thread_pool.py
- selection of what to build and test from the CLI
- path_properties.py to indicate how to build and run different examples
- in full system, move all userland stuff into /lkmc
- prefix everything that we defined across files with LKMC
- --gdb uber convenient helper
- remove import imp which was deprecated

Full commit messages from the branch follow:

1:

userland: add assembly support

Move arm assembly cheat here, and start some work on x86 cheat as well.

2:

document userland asm syscall interfaces

3:

userland assembly: structure readme

4:

x86 fail works

5:

asm: more links

6:

userland: add ported to all archs

7:

move all our stuff into /lkmc in guest

Motivation: userland is getting several new subdirectories, it would be
too insane to just dump all of that in the guest root filesystem.

To alleviate the cd pain, .profile puts user inside /lkmc by default.

8:

start the big userland migration

9:

migrate all

10:

bak

11:

build-userland-in-tree is now a Python command

./build calls it, we did this to allow --download-dependencies to work
perfectly.

12:

rename include to lkmc

13:

mtops.h is perfect now

14:

userland: make build perfect

15:

preparing test_user_mode, need to generalize stuff as usual

16:

asm: prefix every linux specific with linux/

17:

userland: maybe it really works

18:

userland: fix kernel version to work on older ubuntu

Expose --kernel-version to allow customization.

Update LTP info.

19:

userland: build really truly working now

userland test: start work, in a working state, but no features

20:

test-user-mode: make perfect like build-userland

Multithreading and target selection.

21:

userland: get a bit closer to perfection

22:

thread_pool: support passing thread IDs

Then use that to fix gem5 error log read race.

23:

userland: native testing

24:

userland: path properties getting nice!

25:

userland: move posix/environ from cpp-cheat

26:

gem5: --debug-flags without =, looks nicer whenever it can be done

27:

run: rename --wait-gdb in --gdb-wait, --gdb prefix might become a thing

28:

run: create --tmux-program gdb to open gem5 GDB

29:

run: create the uber convenient --gdb option

30:

userland: move getchar from cpp-cheat

31:

prebuilt: kernel boot aarch64 does not work on Ubuntu 16.04

32:

userland: x86_64 linux hello world make PIE

33:

userland: try to make userland executable selection saner

Only allow existing files to be built, stop extension expansion madness.

cli_function: get_cli print booleans properly, was printing without --no-
for negations.

34:

userland: only link to lkmc.o if needed

35:

path_properties: make data very compact with only tuples and dicts

Spend 2 hours of my life thinking about low value tree walks ;-)

36:

userland: move more userland/arch/ logic into property tree

37:

userland: make libs work

Working for build, but now test-user-mode-in-tree is not using --in-tree,
TODO fix later on.

38:

userland: make libs really work

39:

userland: document path_properties

40:

userland: classify linux

41:

waste your life

42:

common: fix absolute path runs

--gdb: allow running from arbitrary directory

43:

baremetal: arm allow using floating point instructions

44:

baremetal: stat preparing to make perfect like userland/

45:

run: fix image check logic accounting for userland

Was failing if I try to run userland (with abspath) when out/
directory is not present.

46:

cli-function: raise if the config file is given and does not exist

47:

common: define missing 'ld' variable, this broke m5 build

48:

rum: --qemu-which host now works for user mode as well as system

Don't fall back on host QEMU automatically, too much insanity.

49:

userland: refix silly mistakes

50:

userland: use path_properties flags for all builds, including lkmc. and userland/arch/main.c

Without this in particular, --gdb fails on assembly because main.c
was not being built with -ggdb3.

51:

userland: start refactor to show failing values on failure!

aarch64 basically done, but missing:

- other archs
- maybe convert main.c into C++ to use templates?
- full review of ASSERT_EQ calling convention issues not seen by tests
  by chance
- documentation

52:

readme: releases are more stable...

53:

submodules: sort gitmodules

54:

test-baremetal: same interface as test-user-mode

In particular, runs tests in parallel, and allows selecting given tests

55:

baremetal: allow arbitrary exit status with the magic string

test-baremetal: fix missing setting x0 return value

Examples were just returning on ret without setting x0, which led to
failures... those were not noticed because of how broken the testing system
was ;-)

56:

baremetal: ah, actually nope, it didn't work :-(

Workaround for now. Works on asserts, but not on exit 1.

Some other day, maybe.

https://github.com/cirosantilli/linux-kernel-module-cheat/issues/59

57:

panic on panic: improve behaviour description

58:

baremetal: get exit status working with on_exit :-)

59:

baremetal: implement C assert

60:

test-baremetal: remove commented out exit status workaround

61:

test-user-mode: handle exit status for signals. Fix #61.

62:

aarch64: fix ASSERT_EQ_REG tests on gem5

Was doing an 8-byte aligned store, which gem5 dislikes.

But the ARMARM says bad things may happen there, notably a signal:
"D1.8.2 SP alignment checking" so gem5 is not really too wrong,
QEMU just happens to work by chance.

63:

userland assembly: build empty.S and fail.S to toplevel and run fail.S with path_properties exit_status

They were just duplicating stuff needlessly while we don't support non-native in-tree builds,
which leads to executable conflicts for C file anyways.

64:

gem5: use a single build tree for all build types

gem5 already has different object names for each build type it seems, so
let's just make sure that works and save some disk space.

65:

userland x86_64: ASSERT_EQ show actual and expected values

66:

assert_fail.c: add to readme index

67:

userland x86_64: implement ASSERT_MEMCMP

68:

userland x86_64: allow ASSERT_EQ to take just about anything

69:

gas data sizes

70:

gas_data_sizes.S: make PIE for all ISAs

71:

x86: paddq

72:

x86 paddq: test entire family

73:

Get rid of imp, started giving deprecation warning every time in Python 3.7 in Ubuntu 19.04.

Please python stop torturing me with refactors.

Make ./run -u blow up if executable not found, otherwise I go crazy.

Get ./test-gdb back to life after the ./run relative path refactor, forgot to test this.

74:

fix run-toolchain, qemu-monitor, trace-boot, trace2line, bisect-linux-boot-gem5. Fixes part of #63

I'm sad no one reported qemu-monitor break, that one is kind of important.

count.out arguments broke it as an init program, since the kernel adds trash
parameters to every init.

Is anyone using this repo, I wonder? Keep pushing, keep pushing.
One day it gets good enough, and the whole world will see.

75:

x86 assembly: addpd

76:

Fix import_path circular dependency by splitting it out.

Use import thread_pool instead from, from is evil.

Fix poweroff.out path for ./trace-boot.

77:

run: rename cryptic tmu to tmux-split, ./run is good now so I never use it anymore explicitly

78:

assembly SIMD add: make uniform for all ISAs, mark as entry point to learning SIMD

79:

start moving arm-assembly-cheat readme in here

80:

arm assembly: move some more in

81:

move more arm in

82:

userland: attempt to fix all assembly example links to README

83:

assembly: improve organization of simd add

84:

ld2 move in

85:

Make userland / assembly getting started more uniform / visible

Forward --gcc-which to ./run --tmux.

Use gdb-multiarch for --gcc-which host.

86:

userland: disable PIE explicitly on command line for all executables

87:

userland: make userland content a better landing page

88:

build: check git version from --version and degrade gracefully

89:

build: make --dry-run work again on all

90:

import_path: importlib explicit for Ubuntu 16.04

91:

make all submodules point to my forks

git servers are insane, submodule implementation is crap, what can you do

92:

build: log warning on git too old for --update

93:

build-linux: do olddefconfig even if no fragments

In particular, gem5 kernel 4.15 needs it

94:

userland content: improve a bit landing page for cpp-cheat
This commit is contained in:
Ciro Santilli 六四事件 法轮功
2019-05-16 00:00:00 +00:00
parent 7a5ca339a3
commit d1003f1cb2
388 changed files with 9972 additions and 3432 deletions
Download Patch File Download Diff File Expand all files Collapse all files

427
run
View File

@@ -18,32 +18,31 @@ Run some content on an emulator.
'''
)
self.add_argument(
'--background', default=False,
help='''\
Send QEMU output to a file instead of the terminal so it does not require a
terminal attached to run on the background. Interactive input cannot be given.
TODO: use a port instead. If only there was a way to redirect a serial to multiple
places, both to a port and a file? We use the file currently to be able to have
any output at all.
https://superuser.com/questions/1373226/how-to-redirect-qemu-serial-output-to-both-a-file-and-the-terminal-or-a-port
'''
)
self.add_argument(
'-c', '--cpus', default=1, type=int,
'-c',
'--cpus',
default=1,
type=int,
help='Number of guest CPUs to emulate. Default: %(default)s'
)
self.add_argument(
'--ctrl-c-host', default=False,
'--ctrl-c-host',
default=False,
help='''\
Ctrl +C kills the QEMU simulator instead of being passed to the guest.
'''
)
self.add_argument(
'-D', '--debug-vm', default=False,
help='Run GDB on the emulator itself.'
'-D',
'--debug-vm',
default=False,
help='''\
Run GDB on the emulator itself.
For --emulator native, this debugs the target program.
'''
)
self.add_argument(
'--debug-vm-args', default='',
'--debug-vm-args',
default='',
help='Pass arguments to GDB. Implies --debug-vm.'
)
self.add_argument(
@@ -54,10 +53,90 @@ which is what you usually want.
'''
)
self.add_argument(
'-E', '--eval',
'-E',
'--eval',
help='''\
Replace the normal init with a minimal init that just evals the given string.
Replace the normal init with a minimal init that just evals the given sh string.
See: https://github.com/cirosantilli/linux-kernel-module-cheat#replace-init
chdir into lkmc_home before running the command:
https://github.com/cirosantilli/linux-kernel-module-cheat#lkmc_home
'''
)
self.add_argument(
'-F',
'--eval-after',
help='''\
Similar to --eval, but the string gets evaled at the last init script,
after the normal init finished.
See: https://github.com/cirosantilli/linux-kernel-module-cheat#init-busybox
'''
)
self.add_argument(
'-G',
'--gem5-exe-args',
default='',
help='''\
Pass extra options to the gem5 executable.
Do not confuse with the arguments passed to config scripts,
like `fs.py`. Example:
./run --emulator gem5 --gem5-exe-args '--debug-flags=Exec --debug' -- --cpu-type=HPI --caches
will run:
gem.op5 --debug-flags=Exec fs.py --cpu-type=HPI --caches
'''
)
self.add_argument(
'--gdb',
default=False,
help='''\
Shortcut for the most common GDB options that you want most of the time. Implies:
* --gdb-wait
* --tmux-args <main> where <main> is:
** start_kernel in full system
** main in user mode
* --tmux-program gdb
'''
)
self.add_argument(
'--gdb-wait',
default=False,
help='''\
Wait for GDB to connect before starting execution
See: https://github.com/cirosantilli/linux-kernel-module-cheat#gdb
'''
)
self.add_argument(
'--gem5-script',
default='fs',
choices=['fs', 'biglittle'],
help='Which gem5 script to use'
)
self.add_argument(
'--gem5-readfile',
default='',
help='Set the contents of m5 readfile to this string.'
)
self.add_argument(
'--gem5-restore',
type=int,
help='''\
Restore the nth most recently taken gem5 checkpoint according to directory
timestamps.
'''
)
self.add_argument(
'--graphic',
default=False,
help='''\
Run in graphic mode.
See: http://github.com/cirosantilli/linux-kernel-module-cheat#graphics
'''
)
self.add_argument(
'--kdb',
default=False,
help='''\
Setup KDB kernel CLI options.
See: http://github.com/cirosantilli/linux-kernel-module-cheat#kdb
'''
)
self.add_argument(
@@ -66,15 +145,7 @@ See: https://github.com/cirosantilli/linux-kernel-module-cheat#replace-init
Pass an extra Linux kernel command line options, and place them before
the dash separator `-`. Only options that come before the `-`, i.e.
"standard" options, should be passed with this option.
Example: `./run --arch arm --kernel-cli 'init=/poweroff.out'`
'''
)
self.add_argument(
'-F', '--eval-after',
help='''\
Pass a base64 encoded command line parameter that gets evalled at the end of
the normal init.
See: https://github.com/cirosantilli/linux-kernel-module-cheat#init-busybox
Example: `./run --arch arm --kernel-cli 'init=/lkmc/poweroff.out'`
'''
)
self.add_argument(
@@ -88,45 +159,41 @@ Example: `./run --kernel-cli-after-dash 'lkmc_eval="wget google.com" lkmc_lala=y
'''
)
self.add_argument(
'-G', '--gem5-exe-args', default='',
'--kernel-version',
default='5.0',
help='''\
Pass extra options to the gem5 executable.
Do not confuse with the arguments passed to config scripts,
like `fs.py`. Example:
./run --emulator gem5 --gem5-exe-args '--debug-flags=Exec --debug' -- --cpu-type=HPI --caches
will run:
gem.op5 --debug-flags=Exec fs.py --cpu-type=HPI --caches
Pass a base64 encoded command line parameter that gets evalled at the end of
the normal init.
See: https://github.com/cirosantilli/linux-kernel-module-cheat#init-busybox
chdir into lkmc_home before running the command:
https://github.com/cirosantilli/linux-kernel-module-cheat#lkmc_home
Specify the Linux kernel version to be reported by syscall emulation.
Defaults to the same kernel version as our default Buildroot build.
Currently only works for QEMU.
See: http://github.com/cirosantilli/linux-kernel-module-cheat#fatal-kernel-too-old
'''
)
self.add_argument(
'--gem5-script',
default='fs',
choices=['fs', 'biglittle'],
help='Which gem5 script to use'
)
self.add_argument(
'--gem5-readfile', default='',
help='Set the contents of m5 readfile to this string.'
)
self.add_argument(
'-K', '--kvm', default=False,
help='Use KVM. Only works if guest arch == host arch'
)
self.add_argument(
'--kgdb', default=False,
)
self.add_argument(
'--kdb', default=False,
)
self.add_argument(
'--gem5-restore', type=int,
'--kgdb',
default=False,
help='''\
Restore the nth most recently taken gem5 checkpoint according to directory
timestamps.
Setup KGDB kernel CLI options.
See: http://github.com/cirosantilli/linux-kernel-module-cheat#kgdb
'''
)
self.add_argument(
'-m', '--memory', default='256M',
'-K',
'--kvm',
default=False,
help='''\
Use KVM. Only works if guest arch == host arch.
See: http://github.com/cirosantilli/linux-kernel-module-cheat#kvm
'''
)
self.add_argument(
'-m',
'--memory',
default='256M',
help='''\
Set the memory size of the guest. E.g.: `-m 512M`. We try to keep the default
at the minimal ammount amount that boots all archs. Anything lower could lead
@@ -142,21 +209,43 @@ Setup a kernel init parameter that makes the emulator quit immediately after boo
'''
)
self.add_argument(
'-R', '--replay', default=False,
'-R',
'--replay',
default=False,
help='Replay a QEMU run record deterministically'
)
self.add_argument(
'-r', '--record', default=False,
'-r',
'--record',
default=False,
help='Record a QEMU run record for later replay with `-R`'
)
self.add_argument(
'-T', '--trace',
'--show-stdout',
default=True,
help='''Show emulator stdout and stderr on the host terminal.'''
)
self.add_argument(
'--terminal',
default=False,
help='''\
Output directly to the terminal, don't pipe to tee as the default.
With this, we don't not save the output to a file as is done by default,
but we are able to do things that require not having a pipe suh as you to
using debuggers. This option issSet automatically by --debug-vm, but you still need
it to debug gem5 Python scripts with pdb.
'''
)
self.add_argument(
'-T',
'--trace',
help='''\
Set trace events to be enabled. If not given, gem5 tracing is completely
disabled, while QEMU tracing is enabled but uses default traces that are very
rare and don't affect performance, because `./configure
--enable-trace-backends=simple` seems to enable some traces by default, e.g.
`pr_manager_run`, and I don't know how to get rid of them.
See: http://github.com/cirosantilli/linux-kernel-module-cheat#tracing
'''
)
self.add_argument(
@@ -174,16 +263,9 @@ Trace instructions run to stdout. Shortcut for --trace --trace-stdout.
'''
)
self.add_argument(
'--terminal', default=False,
help='''\
Output to the terminal, don't pipe to tee as the default.
Does not save the output to a file, but allows you to use debuggers.
Set automatically by --debug-vm, but you still need this option to debug
gem5 Python scripts with pdb.
'''
)
self.add_argument(
'-t', '--tmux', default=False,
'-t',
'--tmux',
default=False,
help='''\
Create a tmux split the window. You must already be inside of a `tmux` session
to use this option:
@@ -201,27 +283,35 @@ Parameters to pass to the program running on the tmux split. Implies --tmux.
'''
)
self.add_argument(
'-w', '--wait-gdb', default=False,
help='Wait for GDB to connect before starting execution'
'--tmux-program',
choices=('gdb', 'shell'),
help='''\
Which program to run in tmux. Implies --tmux. Defaults:
* 'gdb' in qemu
* 'shell' in gem5. 'shell' is only supported in gem5 currently.
'''
)
self.add_argument(
'-x', '--graphic', default=False,
help='Run in graphic mode. Mnemonic: X11'
)
self.add_argument(
'-V', '--vnc', default=False,
'--vnc',
default=False,
help='''\
Run QEMU with VNC instead of the default SDL. Connect to it with:
`vinagre localhost:5900`.
'''
)
self.add_argument(
'extra_emulator_args', nargs='*', default=[],
help='Extra options to append at the end of the emulator command line'
'extra_emulator_args',
nargs='*',
default=[],
help='''\
Extra options to append at the end of the emulator command line.
'''
)
def timed_main(self):
show_stdout = True
if self.env['emulator'] == 'native' and self.env['userland'] is None:
raise Exception('native emulator only supported in user mode')
show_stdout = self.env['show_stdout']
# Common qemu / gem5 logic.
# nokaslr:
# * https://unix.stackexchange.com/questions/397939/turning-off-kaslr-to-debug-linux-kernel-using-qemu-and-gdb
@@ -232,16 +322,31 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
kernel_cli += ' {}'.format(self.env['kernel_cli'])
if self.env['quit_after_boot']:
kernel_cli += ' {}'.format(self.env['quit_init'])
kernel_cli_after_dash = ''
kernel_cli_after_dash = ' lkmc_home={}'.format(self.env['guest_lkmc_home'])
extra_emulator_args = []
extra_qemu_args = []
if self.env['tmux_args'] is not None:
if not self.env['_args_given']['tmux_program']:
if self.env['emulator'] == 'qemu':
self.env['tmux_program'] = 'gdb'
elif self.env['emulator'] == 'gem5':
self.env['tmux_program'] = 'shell'
if self.env['gdb']:
if not self.env['_args_given']['gdb_wait']:
self.env['gdb_wait'] = True
if not self.env['_args_given']['tmux_args']:
if self.env['userland'] is None and self.env['baremetal'] is None:
self.env['tmux_args'] = 'start_kernel'
else:
self.env['tmux_args'] = 'main'
if not self.env['_args_given']['tmux_program']:
self.env['tmux_program'] = 'gdb'
if self.env['tmux_args'] is not None or self.env['_args_given']['tmux_program']:
self.env['tmux'] = True
if self.env['debug_vm'] or self.env['debug_vm_args']:
debug_vm = ['gdb', LF, '-q', LF] + self.sh.shlex_split(self.env['debug_vm_args']) + ['--args', LF]
else:
debug_vm = []
if self.env['wait_gdb']:
if self.env['gdb_wait']:
extra_qemu_args.extend(['-S', LF])
if self.env['eval_after'] is not None:
kernel_cli_after_dash += ' lkmc_eval_base64="{}"'.format(self.sh.base64_encode(self.env['eval_after']))
@@ -252,7 +357,7 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
else:
vnc = []
if self.env['eval'] is not None:
kernel_cli += ' {}=/eval_base64.sh'.format(self.env['initarg'])
kernel_cli += ' {}=/lkmc/eval_base64.sh'.format(self.env['initarg'])
kernel_cli_after_dash += ' lkmc_eval="{}"'.format(self.sh.base64_encode(self.env['eval']))
if not self.env['graphic']:
extra_qemu_args.extend(['-nographic', LF])
@@ -301,36 +406,40 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
if not self.env['dry_run']:
raise Exception('Root filesystem not found. Did you build it? ' \
'Tried to use: ' + self.env['disk_image'])
def raise_image_not_found():
def raise_image_not_found(image):
if not self.env['dry_run']:
raise Exception('Executable image not found. Did you build it? ' \
'Tried to use: ' + self.env['image'])
if self.env['image'] is None:
raise Exception('Baremetal ELF file not found. Tried:\n' + '\n'.join(paths))
'Tried to use: ' + image)
if not os.path.exists(self.env['image']):
raise_image_not_found(self.env['image'])
cmd = debug_vm.copy()
if self.env['emulator'] == 'gem5':
if self.env['quiet']:
show_stdout = False
if self.env['baremetal'] is None:
if not self.env['baremetal'] is None:
if not os.path.exists(self.env['gem5_fake_iso']):
os.makedirs(os.path.dirname(self.env['gem5_fake_iso']), exist_ok=True)
self.sh.write_string_to_file(self.env['gem5_fake_iso'], 'a' * 512)
elif self.env['userland'] is None:
if not os.path.exists(self.env['rootfs_raw_file']):
if not os.path.exists(self.env['qcow2_file']):
raise_rootfs_not_found()
self.raw_to_qcow2(qemu_which=self.env['qemu_which'], reverse=True)
else:
if not os.path.exists(self.env['gem5_fake_iso']):
os.makedirs(os.path.dirname(self.env['gem5_fake_iso']), exist_ok=True)
self.sh.write_string_to_file(self.env['gem5_fake_iso'], 'a' * 512)
if not os.path.exists(self.env['image']):
# This is to run gem5 from a prebuilt download.
if (not self.env['baremetal'] is None) or (not os.path.exists(self.env['linux_image'])):
raise_image_not_found()
self.sh.run_cmd([os.path.join(self.env['extract_vmlinux'], self.env['linux_image'])])
if (
self.env['baremetal'] is None and
self.env['userland'] is None
):
if not os.path.exists(self.env['linux_image']):
raise_image_not_found(self.env['image'])
self.sh.run_cmd([os.path.join(self.env['extract_vmlinux'], self.env['linux_image'])])
os.makedirs(os.path.dirname(self.env['gem5_readfile']), exist_ok=True)
self.sh.write_string_to_file(self.env['gem5_readfile'], self.env['gem5_readfile'])
memory = '{}B'.format(self.env['memory'])
gem5_exe_args = self.sh.shlex_split(self.env['gem5_exe_args'])
if do_trace:
gem5_exe_args.extend(['--debug-flags={}'.format(trace_type), LF])
gem5_exe_args.extend(['--debug-flags', trace_type, LF])
extra_env['M5_PATH'] = self.env['gem5_system_dir']
# https://stackoverflow.com/questions/52312070/how-to-modify-a-file-under-src-python-and-run-it-without-rebuilding-in-gem5/52312071#52312071
extra_env['M5_OVERRIDE_PY_SOURCE'] = 'true'
@@ -350,7 +459,7 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
if self.env['userland'] is not None:
cmd.extend([
self.env['gem5_se_file'], LF,
'--cmd', self.resolve_userland_executable(self.env['userland']), LF,
'--cmd', self.env['image'], LF,
])
if self.env['userland_args'] is not None:
cmd.extend(['--options', self.env['userland_args'], LF])
@@ -421,7 +530,7 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
])
if self.env['dtb']:
cmd.extend(['--dtb', os.path.join(self.env['gem5_system_dir'], 'arm', 'dt', 'armv8_gem5_v1_big_little_2_2.dtb'), NL])
if self.env['wait_gdb']:
if self.env['gdb_wait']:
# https://stackoverflow.com/questions/49296092/how-to-make-gem5-wait-for-gdb-to-connect-to-reliably-break-at-start-kernel-of-th
cmd.extend(['--param', 'system.cpu[0].wait_for_remote_gdb = True', LF])
elif self.env['emulator'] == 'qemu':
@@ -429,38 +538,28 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
'-trace', 'enable={},file={}'.format(trace_type, self.env['qemu_trace_file']), LF,
]
if self.env['userland'] is not None:
if self.env['wait_gdb']:
if self.env['gdb_wait']:
debug_args = ['-g', str(self.env['gdb_port']), LF]
else:
debug_args = []
cmd.extend(
[
os.path.join(self.env['qemu_build_dir'], '{}-linux-user'.format(self.env['arch']), 'qemu-{}'.format(self.env['arch'])), LF,
'-L', self.env['userland_library_dir'], LF
self.env['qemu_executable'], LF,
'-L', self.env['userland_library_dir'], LF,
'-r', self.env['kernel_version'], LF,
'-seed', '0', LF,
] +
qemu_user_and_system_options +
debug_args
)
else:
if not os.path.exists(self.env['image']):
raise_image_not_found()
extra_emulator_args.extend(extra_qemu_args)
self.make_run_dirs()
if self.env['qemu_which'] == 'host' or not os.path.exists(self.env['qemu_executable']):
qemu_executable = self.env['qemu_executable_basename']
qemu_executable_host = True
else:
qemu_executable = self.env['qemu_executable']
qemu_executable_host = False
qemu_executable = shutil.which(qemu_executable)
if qemu_executable is None:
raise Exception('QEMU executable not found, did you forget to build or install it?\n' \
'Tried to use: ' + qemu_executable)
if self.env['debug_vm']:
serial_monitor = []
else:
if self.env['background']:
serial_monitor = ['-serial', 'file:{}'.format(self.env['qemu_background_serial_file']), LF]
serial_monitor = ['-serial', 'file:{}'.format(self.env['guest_terminal_file']), LF]
if self.env['quiet']:
show_stdout = False
else:
@@ -496,7 +595,7 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
machine2 = []
cmd.extend(
[
qemu_executable, LF,
self.env['qemu_executable'], LF,
'-machine', self.env['machine'], LF,
] +
machine2 +
@@ -506,7 +605,11 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
'-kernel', self.env['image'], LF,
'-m', self.env['memory'], LF,
'-monitor', 'telnet::{},server,nowait'.format(self.env['qemu_monitor_port']), LF,
'-netdev', 'user,hostfwd=tcp::{}-:{},hostfwd=tcp::{}-:22,id=net0'.format(self.env['qemu_hostfwd_generic_port'], self.env['qemu_hostfwd_generic_port'], self.env['qemu_hostfwd_ssh_port']), LF,
'-netdev', 'user,hostfwd=tcp::{}-:{},hostfwd=tcp::{}-:22,id=net0'.format(
self.env['qemu_hostfwd_generic_port'],
self.env['qemu_hostfwd_generic_port'],
self.env['qemu_hostfwd_ssh_port']
), LF,
'-no-reboot', LF,
'-smp', str(self.env['cpus']), LF,
] +
@@ -516,7 +619,7 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
)
if self.env['dtb'] is not None:
cmd.extend(['-dtb', self.env['dtb'], LF])
if not qemu_executable_host:
if not self.env['qemu_which'] == 'host':
cmd.extend(qemu_user_and_system_options)
if self.env['initrd']:
extra_emulator_args.extend(['-initrd', self.env['buildroot_cpio'], LF])
@@ -542,7 +645,12 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
self.raw_to_qcow2(qemu_which=self.env['qemu_which'])
extra_emulator_args.extend([
'-drive',
'file={},format=qcow2,if={}{}{}'.format(self.env['disk_image'], driveif, snapshot, rrid),
'file={},format=qcow2,if={}{}{}'.format(
self.env['disk_image'],
driveif,
snapshot,
rrid
),
LF,
])
if rr:
@@ -553,7 +661,10 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
if rr:
extra_emulator_args.extend([
'-object', 'filter-replay,id=replay,netdev=net0',
'-icount', 'shift=7,rr={},rrfile={}'.format('record' if self.env['record'] else 'replay', self.env['qemu_rrfile']),
'-icount', 'shift=7,rr={},rrfile={}'.format(
'record' if self.env['record'] else 'replay',
self.env['qemu_rrfile']
),
])
virtio_gpu_pci = []
else:
@@ -580,34 +691,44 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
cmd.extend(append)
if self.env['tmux']:
tmux_args = '--run-id {}'.format(self.env['run_id'])
if self.env['emulator'] == 'gem5':
tmux_cmd = './gem5-shell'
else:
tmux_cmd = './run-gdb'
if self.env['tmux_program'] == 'shell':
if self.env['emulator'] == 'gem5':
tmux_cmd = './gem5-shell'
else:
raise Exception('--tmux-program is only supported in gem5 currently.')
elif self.env['tmux_program'] == 'gdb':
tmux_cmd = os.path.join(self.env['root_dir'], 'run-gdb')
# TODO find a nicer way to forward all those args automatically.
# Part of me wants to: https://github.com/jonathanslenders/pymux
# but it cannot be used as a library properly it seems, and it is
# slower than tmux.
tmux_args += " --arch {} --linux-build-id '{}' --run-id '{}'".format(
tmux_args += " --arch {} --emulator '{}' --gcc-which '{}' --linux-build-id '{}' --run-id '{}' --userland-build-id '{}'".format(
self.env['arch'],
self.env['emulator'],
self.env['gcc_which'],
self.env['linux_build_id'],
self.env['run_id'],
self.env['userland_build_id'],
)
if self.env['baremetal']:
tmux_args += " --baremetal '{}'".format(self.env['baremetal'])
if self.env['userland']:
tmux_args += " --userland '{}'".format(self.env['userland'])
if self.env['in_tree']:
tmux_args += ' --in-tree'
if self.env['tmux_args'] is not None:
tmux_args += ' {}'.format(self.env['tmux_args'])
subprocess.Popen([
os.path.join(self.env['root_dir'], 'tmu'),
tmux_cmd = [
os.path.join(self.env['root_dir'], 'tmux-split'),
"sleep 2;{} {}".format(tmux_cmd, tmux_args)
])
]
self.log_info(tmux_cmd)
subprocess.Popen(tmux_cmd)
cmd.extend(extra_emulator_args)
cmd.extend(self.env['extra_emulator_args'])
if self.env['emulator'] == 'qemu' and self.env['userland']:
if self.env['userland'] and self.env['emulator'] in ('qemu', 'native'):
# The program and arguments must come at the every end of the CLI.
cmd.extend([self.resolve_userland_executable(self.env['userland']), LF])
cmd.extend([self.env['image'], LF])
if self.env['userland_args'] is not None:
cmd.extend(self.sh.shlex_split(self.env['userland_args']))
if debug_vm or self.env['terminal']:
@@ -623,35 +744,37 @@ Run QEMU with VNC instead of the default SDL. Connect to it with:
show_stdout=show_stdout,
)
if exit_status == 0:
# Check if guest panicked.
if self.env['emulator'] == 'gem5':
# We have to do some parsing here because gem5 exits with status 0 even when panic happens.
# Grepping for '^panic: ' does not work because some errors don't show that message.
panic_msg = b'--- BEGIN LIBC BACKTRACE ---$'
else:
panic_msg = b'Kernel panic - not syncing'
panic_re = re.compile(panic_msg)
error_string_found = False
exit_status = 0
if out_file is not None and not self.env['dry_run']:
with open(self.env['termout_file'], 'br') as logfile:
line = None
for line in logfile:
if panic_re.search(line):
exit_status = 1
if line is not None:
last_line = line.rstrip()
match = re.search(b'Simulated exit code not 0! Exit code is (\d+)', last_line)
if match:
exit_status = int(match.group(1))
if self.env['emulator'] == 'gem5':
with open(self.env['termout_file'], 'br') as logfile:
# We have to do some parsing here because gem5 exits with status 0 even when panic happens.
# Grepping for '^panic: ' does not work because some errors don't show that message...
gem5_panic_re = re.compile(b'--- BEGIN LIBC BACKTRACE ---$')
line = None
for line in logfile:
if gem5_panic_re.search(line):
exit_status = 1
if self.env['userland']:
if line is not None:
last_line = line.rstrip()
match = re.search(b'Simulated exit code not 0! Exit code is (\d+)', last_line)
if match:
exit_status = int(match.group(1))
if not self.env['userland']:
if os.path.exists(self.env['guest_terminal_file']):
with open(self.env['guest_terminal_file'], 'br') as logfile:
linux_panic_re = re.compile(b'Kernel panic - not syncing')
serial_magic_exit_status_regexp = re.compile(self.env['serial_magic_exit_status_regexp_string'])
for line in logfile.readlines():
if line.rstrip() == self.env['magic_fail_string']:
line = line.rstrip()
if not self.env['baremetal'] and linux_panic_re.search(line):
exit_status = 1
break
if exit_status != 0:
match = serial_magic_exit_status_regexp.match(line)
if match:
exit_status = int(match.group(1))
if exit_status != 0 and self.env['show_stdout']:
self.log_error('simulation error detected by parsing logs')
return exit_status