98 Commits

Author SHA1 Message Date
Ciro Santilli 六四事件 法轮功
da900a579c LKMC v3.0
This is a squash commit, the unsquashed development went through many
unstable phases which would break bisects. The unsquashed branch is:
https://github.com/cirosantilli/linux-kernel-module-cheat/tree/v3.0-unsquash

The main improvement of this release was to greatly generalize the testing system.

The key addition was cli_function.py, which allows scripts such as ./run to
be transparently called either from Python or from the command line.

New tests scripts were created using this improved framework: test-baremetal
and test-user-mode.

We were lazy to port some of less important tests to the new setup, TODO's were
added, and we need comes they will be fixed. Getting started is however sacred
as usual and should work.

Other changes include:

-   gem5: update to 7fa4c946386e7207ad5859e8ade0bbfc14000d91

-   run: --tmux-args implies --tmux

-   run: add --userland-args to make userland arguments across QEMU and gem5

    Get rid of --userland-before as a consequence.

-   bring initrd and initramfs back to life

-   build-userland: create --static to make build a bit easier

-   gem5: --gem5-worktree also set --gem5-build-id

-   remove --gem5, use --emulator gem5 everywhere

    Allow passing --emulator multiple times for transparent tests selection
    just like --arch.

-   test-userland: allow selecting just a few tests

-   linux: update to v4.20

-   buildroot: update to 2018.08

    The main motivation for this was to fix the build for Ubuntu 18.10, which
    has glibc 2.28, which broke the 2018.05 build at the m4-host package with:

        #error "Please port gnulib fseeko.c to your platform!

-   getvar --type input

-   failed xen attempt, refactor timer, failed svc attempt, aarch64 use gicv3

-   build-doc: exit 1 on error, add to release testing

-   build: add --apt option to make things easier on other distros

-   build-linux: --no-modules-install
2019-01-22 00:00:00 +00:00
Ciro Santilli 六四事件 法轮功
f73eaaffd2 prebuilt: automatically pick up qemu from PATH if not built like qemu-img 2018-11-22 00:00:02 +00:00
Ciro Santilli 六四事件 法轮功
985913e1e6 prepare for kvm testing 2018-11-19 22:35:16 +00:00
Ciro Santilli 六四事件 法轮功
074fbbdf4c build: fix arm gcc package names for arm hosts 2018-11-17 00:00:01 +00:00
Ciro Santilli 六四事件 法轮功
c2ac65a807 docker: split minimum setup to separate script 2018-11-17 00:00:00 +00:00
Ciro Santilli 六四事件 法轮功
238f1ad1c0 common: add --qemu option to override configs 2018-11-13 00:00:02 +00:00
Ciro Santilli 六四事件 法轮功
e54635c3ab make gdb test automation even awesomer 2018-11-13 00:00:01 +00:00
Ciro Santilli 六四事件 法轮功
a06872241b the docker setup is perfect 2018-11-11 00:00:02 +00:00
Ciro Santilli 六四事件 法轮功
403d4a9d06 docker: migrate to python 2018-11-11 00:00:01 +00:00
Ciro Santilli 六四事件 法轮功
32420eec33 download-dependencies: merge into ./build --download-dependencies
Reuses the module system dependencies present there.

run: make --dry-run work even when there is no out directory yet

docker: make the wrapping more intuitive
2018-11-10 00:00:02 +00:00
Ciro Santilli 六四事件 法轮功
5d649377fc build: make baremetal parts more flexible and powerful
Document test-gdb on readme
2018-11-09 00:00:01 +00:00
Ciro Santilli 六四事件 法轮功
e0dbe2416d gdb: create some automated tests with pytest
gem5 baremetal: use m5exit m5op in exit() so as to not force users to
apply a patch for almost all examples
2018-11-09 09:33:44 +00:00
Ciro Santilli 六四事件 法轮功
8fb9db3931 manually encode newlines on all printed commands
This way we group key value arguments: e.g.:

    make \
    -j 8 \
    all

instead of:

    make \
    -j \
    8 \
    all

and reach CLI nirvana, while also subtly breaking several commands due to
lack of testing.
2018-11-04 00:00:01 +00:00
Ciro Santilli 六四事件 法轮功
196dd616ff better build setups for testing and release 2018-11-03 00:00:01 +00:00
Ciro Santilli 六四事件 法轮功
d80c9ccee2 build: add --extra-args motivated by --clean 2018-10-28 08:26:35 +00:00
Ciro Santilli 六四事件 法轮功
8f1c662e3d build kind of works 2018-10-26 00:00:00 +00:00
Ciro Santilli 六四事件 法轮功
004c100453 copy-overlay. ./build broken btw :-) 2018-10-25 00:00:02 +00:00
Ciro Santilli 六四事件 法轮功
2a77df690c common: create a Component class to factor out builds
Not yet finished factoring, but half way there, do for all build-
2018-10-18 00:00:00 +00:00
Ciro Santilli 六四事件 法轮功
cb3d8929ba dry run for all common 2018-10-18 00:00:00 +00:00
Ciro Santilli 六四事件 法轮功
23a9d767ba build-buildroot twice, and split build-m5 2018-10-18 00:00:00 +00:00
Ciro Santilli 六四事件 法轮功
04812521b2 getting started: hide the initial build under ./build
make build awesomer and more generic, convert to python

rename ./configure to ./download-dependencies, since it wasn't configuring
anything
2018-10-14 00:00:00 +00:00
Ciro Santilli
56738a1c70 lkmc v2-rc
Unsquashed version at v2-rc-unsquashed, but that cannot be merged as it
breaks bisects at several points. All bugs will not bisect to this
humongous change.

It all started with a conversion of the Bash scripts to Python, mainly
because I couldn't stand not being able to properly use --options for
run which has a million options.

Then since that required a full testing, I decided to do all the
refactorings that I had in mind at once, and so I did and it became
v2-rc.

This is the largest patch I have ever done! OMG a few weeks of extra time.
I'm never writing a Bash script for anything that starts getting big again.

Some of the features are:

* separate build-qemu and build-gem5 commands
* common: convert scripts to python. Add --option for everything
* rename build to build-buildroot now that we are splitting all the build
  commands, Linux kernel to follow
* move all git submodules to submodules/ and all buildroot packages to
  packages/
* refactor the out/ structure. Keep projects on toplevel, because guest
  projects separate archs and host ones don't, making a toplevel arch wrong
* do-release: rename to just release
  https://stackoverflow.com/questions/16174992/cant-get-argparse-to-read-quoted-string-with-dashes-in-it
* run: add --terminal and explain gem5 pdb
* just track the lvimrc
* store CLI kernel config fragment inside buildlroot to avoid conflicts
* gem5: document m5 initparam
* readme: make a bunch of things awesomer
* readme: fix broken refs
* parsec-benchmark: update to 75d55ac446a43c47efb1044844a108c6c330184c
  Could not fetch otherwise.
* gem5: M5_OVERRIDE_PY_SOURCE
2018-09-14 07:42:31 +01:00
Ciro Santilli
63ec42f72f common: factor out common opts to all scripts 2018-08-14 00:27:55 +01:00
Ciro Santilli
0f5c5a9912 readme: move build and run docs in 2018-08-13 23:14:39 +01:00
Ciro Santilli
539b176e1d gem5: test out panic_on_panic
patches: create patche/manual patch dir, put all patch directories under
patches/ subdir
2018-08-11 20:52:41 +01:00
Ciro Santilli
f5a997b2b6 do-release: sketch + a bunch of fixes from bench-all
qemu: monitor command from guest
2018-08-09 12:56:17 +01:00
Ciro Santilli
b1bd45684f config: replace cli with direct common_ options 2018-08-09 07:12:20 +01:00
Ciro Santilli
1cb056a995 common: simplify set_common_vars, rename to common_setup
Don't pass all arguments explicitly, just use existing vars.

Prefix all common_setup inputs and outputs with common_ to avoid name conflicts.
2018-08-09 06:59:52 +01:00
Ciro Santilli
2e42a776c5 qemu build variant 2018-08-06 02:48:43 +01:00
Ciro Santilli
d330f8d250 Use absolute paths for . common
Otherwise it can pick up other common in PATH first...
2018-07-22 10:42:46 +01:00
Ciro Santilli
1721f184d6 gem5: track build and run variants separately with -M and -N
Otherwise, checking out branches is too insane, as it does not
update the worktrees, even though the gem5/gem5 module was updated.

gem5: expose build types, document debug builds.

simultaneous runs: store stdout and stderr on a file to allow running
all from a single terminal on the background cleanly.
2018-06-27 16:31:24 +01:00
Ciro Santilli
57dbe81f33 lkmc_many_files: create package to test a rootfs with many files
Automatically pick up packages from under packages/ into BR2_EXTERNAL.

Move many subdirectory READMEs into the toplevel and link to toplevel from those subreadmes instead.
2018-06-17 11:36:56 +01:00
Ciro Santilli
37f64de29f gem5: prefix woktree branch with wt/ 2018-06-11 18:46:20 +01:00
Ciro Santilli
c375d58e44 build: only set Linux kernel cli config if options changed
Otherwise, the Linux kernel gets rebuilt every time, which adds a few
seconds to every rebuild.
2018-06-10 09:29:23 +01:00
Ciro Santilli
b562d8dfe7 gem5: support variants 2018-06-10 08:57:39 +01:00
Ciro Santilli
38fd6153d9 gem5 graphic mode: checkout just the linux kernel 2018-06-07 06:55:32 +01:00
Ciro Santilli
61c5162fa6 linux: introduce build variants selectable at runtime 2018-06-07 05:26:04 +01:00
Ciro Santilli
cea0bd7f00 build: move br2 config files to a dedicated directory 2018-06-01 09:56:45 +01:00
Ciro Santilli
07b9ae39d7 kmscube: document failed attempt
build: remove duplicated lines from some merge
2018-06-01 09:50:00 +01:00
Ciro Santilli
32ccda88c8 prebuilt: allow running prebuilt images 2018-05-28 19:36:07 +01:00
Ciro Santilli
207306fa31 gpio: move documentation to README, document that it broke on -M virt 2018-05-22 08:09:06 +01:00
Ciro Santilli
fee5183468 build: fix build without -K undefined d 2018-05-19 19:43:42 +01:00
Ciro Santilli
de9223c03d gem5: aarch64 graphic mode works
build: allow passing extra kernel config fragment with -c.
Old -c was renamed to -C, and old -C to -f.
2018-05-19 09:41:09 +01:00
Ciro Santilli
2b2b433ade build: fix -K 2018-05-15 15:03:39 +01:00
Ciro Santilli
c735790099 build: -c to enable a single linux kernel config
Disable CONFIG_FORTIFY_SOURCE=y by default as it might incur overhead,
enable CONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y as it should not.
2018-05-05 11:50:22 +01:00
Ciro Santilli
8d30cf31ce build: fix -K not working with absolute paths 2018-04-29 19:33:50 +01:00
Ciro Santilli
f10ef3a467 bench-all: add build benchmarks and make all benchmarks options
run: fix ./run -gu broken behaviour. Document ./tmu window switch failure.

readme: move travis failed attempt to readme.
2018-04-22 21:09:17 +01:00
Ciro Santilli
2bddcc2891 build: unify QEMU and gem5 kernel configs! The last one!
Split optional and required kernel configs into separate files.

readme: describe how our kernel configs are generated from Buildroot

readme: document QEMU `-no-reboot` and speculate gem5 `panic_on_panic`
in preparation for some more automated bisection with ktest.
2018-04-21 11:50:02 +01:00
Ciro Santilli
af210a7671 Move arm to virt armv7, unify arm QEMU and gem5 builds
gem5 requires armv7, and we learnt that the versatiledb we were using
was pre-v7.

We could have moved to -M vexpress-*, but in the end decided to go for
-M virt due to its simpliciy, and uniformity with aarch64.

platform_device: does not work anymore and was removed, since it was tied
to versatilepb.

We left a mention on the README and removed all in tree source. The QEMU
patch is still left as it was.

As a consequence, the linux tree had no other patches, and we now use
vanilla linux by default, which is a great thing for reproducibility.

Another consequence is the /poweroff.out works for arm -M virt,
and we removed all mentions of the problem.
2018-04-19 23:38:17 +01:00
Ciro Santilli
9805d333ea tc: create helper to access toolchain tools 2018-04-19 08:24:10 +01:00