mirror of
https://github.com/cirosantilli/linux-kernel-module-cheat.git
synced 2026-01-23 02:05:57 +01:00
getting started: hide the initial build under ./build
make build awesomer and more generic, convert to python rename ./configure to ./download-dependencies, since it wasn't configuring anything
This commit is contained in:
@@ -4,14 +4,14 @@ sudo: required
|
||||
|
||||
install: |
|
||||
cd "$TRAVIS_BUILD_DIR"
|
||||
bash -x ./configure --travis
|
||||
bash -x ./download-dependencies --travis
|
||||
|
||||
script: |
|
||||
cd "$TRAVIS_BUILD_DIR"
|
||||
# --nproc: I'm unable to install nproc on Travis.
|
||||
# TODO why? Is part of coreutils in Ubuntu 16.04:
|
||||
# http://manpages.ubuntu.com/manpages/trusty/man1/nproc.1.html
|
||||
# which ./configure is installing.
|
||||
# which ./download-dependencies is installing.
|
||||
#
|
||||
# awk: without it, too much stdout (4Mb max)
|
||||
# If we ignore stdout: Travis kills job because it spent
|
||||
|
||||
74
README.adoc
74
README.adoc
@@ -72,12 +72,8 @@ Reserve 12Gb of disk and run:
|
||||
....
|
||||
git clone https://github.com/cirosantilli/linux-kernel-module-cheat
|
||||
cd linux-kernel-module-cheat
|
||||
./configure --qemu && \
|
||||
./build-qemu && \
|
||||
./build-linux && \
|
||||
./build-modules && \
|
||||
./build-userland && \
|
||||
./build-buildroot && \
|
||||
./download-dependencies
|
||||
./build
|
||||
./run
|
||||
....
|
||||
|
||||
@@ -95,7 +91,7 @@ If you don't want to wait, you could also try the following faster but much more
|
||||
|
||||
but you will soon find that they are simply not enough if you anywhere near serious about systems programming.
|
||||
|
||||
If `./configure` fails with:
|
||||
If `./download-dependencies` fails with:
|
||||
|
||||
....
|
||||
E: You must put some 'source' URIs in your sources.list
|
||||
@@ -103,15 +99,7 @@ E: You must put some 'source' URIs in your sources.list
|
||||
|
||||
see this: https://askubuntu.com/questions/496549/error-you-must-put-some-source-uris-in-your-sources-list/857433#857433 I don't know how to automate this step. Why, Ubuntu, why.
|
||||
|
||||
It does not work if you just download the `.zip` from GitHub because we use link:.gitmodules[Git submodules], you must clone this repo. `./configure` then fetches only the required submodules for you.
|
||||
|
||||
The order of build commands matters:
|
||||
|
||||
* `./build-linux` must come before `./build-modules` because the kernel modules depend on the Linux kernel build. We could lessen this need by calling `make modules_prepare` on the kernel tree, which does not require a full build, but this is not currently done
|
||||
* `./build-modules` and `./build-userland` must come before `./build-buildroot` because generate files that will be placed in the root filesystem. If you don't call them before, the generated files will not be in the root filesystem.
|
||||
* `build-qemu` must come before `./build-buildroot` because it builds the `qemu-img` tool that we use to convert the raw disk image into link:https://en.wikipedia.org/wiki/Qcow[qcow2] format that QEMU boots from in our setup
|
||||
|
||||
If you mess up the order, just build things again in the right order and you will be fine.
|
||||
It does not work if you just download the `.zip` from GitHub because we use link:.gitmodules[Git submodules], you must clone this repo. `./download-dependencies` then fetches only the required submodules for you.
|
||||
|
||||
After `./run`, QEMU opens up and you can start playing with the kernel modules inside the simulated system:
|
||||
|
||||
@@ -243,6 +231,16 @@ Then rebuild the Linux kernel, quit QEMU and reboot the modified kernel:
|
||||
|
||||
and, surely enough, your message has appeared at the beginning of the boot.
|
||||
|
||||
We could have used just `./build` as in the initial build, but doing just `./build-linux` will save us a bit of time.
|
||||
|
||||
The link:build[`./build`] script is just a lightweight wrapper, but when you start modifying components such as the Linux kernel, it is better to run individual steps directly.
|
||||
|
||||
You can see that `./build` does `./build-linux` by running:
|
||||
|
||||
....
|
||||
./build --dry-run
|
||||
....
|
||||
|
||||
So you are now officially a Linux kernel hacker, way to go!
|
||||
|
||||
===== Your first kernel module hack
|
||||
@@ -285,7 +283,7 @@ The safe way, is to fist quit QEMU, then rebuild the modules, root filesystem, a
|
||||
|
||||
`./build-buildroot` generates the root filesystem with the modules that we compiled at `./build-modules`.
|
||||
|
||||
`--eval-busybox` is optional: you could just type `insmod /hello.ko` in the terminal, but this makes it run automatically at the end of boot.
|
||||
`--eval-busybox` is optional: you could just type `insmod /hello.ko` in the terminal, but this makes it run automatically at the end of boot, and then drops you into a shell.
|
||||
|
||||
If the guest and host are the same arch, typically x86_64, you can speed up boot further with <<kvm>>:
|
||||
|
||||
@@ -371,8 +369,6 @@ All of this makes QEMU the natural choice of default system simulator.
|
||||
|
||||
=== gem5 Buildroot setup
|
||||
|
||||
TODO document kernel modules don't work on 9p
|
||||
|
||||
==== About the gem5 Buildroot setup
|
||||
|
||||
This setup is like the <<qemu-buildroot-setup>>, but it uses link:http://gem5.org/[gem5] instead of QEMU as a system simulator.
|
||||
@@ -393,16 +389,26 @@ See <<gem5-vs-qemu>> for a more thorough comparison.
|
||||
|
||||
==== gem5 Buildroot setup getting started
|
||||
|
||||
For the most part, if you just add the `--gem5` option or `*-gem5` suffix to all commands and everything should magically work:
|
||||
For the most part, if you just add the `--gem5` option or `*-gem5` suffix to all commands and everything should magically work.
|
||||
|
||||
If you haven't built Buildroot yet for <<qemu-buildroot-setup>>, you can build from the beginning with:
|
||||
|
||||
....
|
||||
./configure --gem5
|
||||
./build-gem5
|
||||
./download-dependencies --gem5
|
||||
./build --gem5 --no-qemu
|
||||
./build-buildroot --gem5
|
||||
./run --gem5
|
||||
....
|
||||
|
||||
If you have already built <<qemu-buildroot-setup>> previously, don't be afraid, gem5 and QEMU use almost the same root filesystem and kernel, so `./build-buildroot --gem` will be fast. It is currently only needed for the <<m5>> tool.
|
||||
`--no-qemu` is optional, but it makes the
|
||||
....
|
||||
./download-dependencies --gem5
|
||||
./build-gem5 --gem5
|
||||
./build-buildroot --gem5
|
||||
./run --gem5
|
||||
....
|
||||
|
||||
If you have already built previously, don't be afraid: gem5 and QEMU use almost the same root filesystem and kernel, so `./build-buildroot --gem` will be fast. It is currently only needed for the <<m5>> tool.
|
||||
|
||||
To get a terminal, either open a new shell and run:
|
||||
|
||||
@@ -802,7 +808,7 @@ Our C bare-metal compiler is built with link:https://github.com/crosstool-ng/cro
|
||||
QEMU:
|
||||
|
||||
....
|
||||
./configure --baremetal --qemu
|
||||
./download-dependencies --baremetal --qemu
|
||||
./build-qemu --arch arm
|
||||
./build-crosstool-ng --arch arm
|
||||
./build-baremetal --arch arm
|
||||
@@ -857,7 +863,7 @@ To use gem5 instead of QEMU do:
|
||||
|
||||
....
|
||||
patch -d "$(./getvar gem5_src_dir)" -p 1 < patches/manual/gem5-semihost.patch
|
||||
./configure --baremetal --gem5
|
||||
./download-dependencies --baremetal --gem5
|
||||
./build-gem5 --arch arm
|
||||
./build-crosstool-ng --arch arm
|
||||
./build-baremetal --arch arm --gem5
|
||||
@@ -8007,7 +8013,7 @@ less "$(./getvar -a "$arch" run_dir)/trace.txt"
|
||||
|
||||
This functionality relies on the following setup:
|
||||
|
||||
* `./configure --enable-trace-backends=simple`. This logs in a binary format to the trace file.
|
||||
* `./download-dependencies --enable-trace-backends=simple`. This logs in a binary format to the trace file.
|
||||
+
|
||||
It makes 3x execution faster than the default trace backend which logs human readable data to stdout.
|
||||
+
|
||||
@@ -8747,7 +8753,7 @@ There are two ways to run PARSEC with this repo:
|
||||
====== PARSEC benchmark without parsecmgmt
|
||||
|
||||
....
|
||||
./configure --gem5 --parsec-benchmark
|
||||
./download-dependencies --gem5 --parsec-benchmark
|
||||
./build-buildroot --arch arm --buildroot-config 'BR2_PACKAGE_PARSEC_BENCHMARK=y' --gem5
|
||||
./run --arch arm --gem5
|
||||
....
|
||||
@@ -10189,7 +10195,7 @@ CT_GDB_CROSS_SIM=y
|
||||
which by grepping crosstool-NG we can see does on GDB:
|
||||
|
||||
....
|
||||
./configure --enable-sim
|
||||
./download-dependencies --enable-sim
|
||||
....
|
||||
|
||||
Those are not set by default on `gdb-multiarch` in Ubuntu 16.04.
|
||||
@@ -10384,7 +10390,7 @@ We tried to automate it on Travis with link:.travis.yml[] but it hits the curren
|
||||
Benchmark all:
|
||||
|
||||
....
|
||||
./build-all
|
||||
./build --all-linux-components
|
||||
./bench-boot
|
||||
cat "$(./getvar bench_boot)"
|
||||
....
|
||||
@@ -10464,7 +10470,7 @@ Kernel panic - not syncing: Attempted to kill the idle task!
|
||||
|
||||
==== Benchmark builds
|
||||
|
||||
The build times are calculated after doing `./configure` and link:https://buildroot.org/downloads/manual/manual.html#_offline_builds[`make source`], which downloads the sources, and basically benchmarks the <<benchmark-internets,Internet>>.
|
||||
The build times are calculated after doing `./download-dependencies` and link:https://buildroot.org/downloads/manual/manual.html#_offline_builds[`make source`], which downloads the sources, and basically benchmarks the <<benchmark-internets,Internet>>.
|
||||
|
||||
Sample build time at 2c12b21b304178a81c9912817b782ead0286d282: 28 minutes, 15 with full ccache hits. Breakdown: 19% GCC, 13% Linux kernel, 7% uclibc, 6% host-python, 5% host-qemu, 5% host-gdb, 2% host-binutils
|
||||
|
||||
@@ -11127,7 +11133,7 @@ It takes too much time to be feasible for every patch, but it should be done for
|
||||
==== Automated tests
|
||||
|
||||
....
|
||||
./build-all
|
||||
./build --all-linux-components
|
||||
./test --size 3
|
||||
echo $?
|
||||
....
|
||||
@@ -11136,7 +11142,7 @@ should output 0.
|
||||
|
||||
Sources:
|
||||
|
||||
* link:build-all[]
|
||||
* link:build[]
|
||||
* link:test[]
|
||||
|
||||
Test just the kernel modules:
|
||||
@@ -11307,10 +11313,10 @@ TODO also run tests and only release if they pass.
|
||||
|
||||
==== release-zip
|
||||
|
||||
Create a zip containing all files required for <<prebuilt>>
|
||||
Create a zip containing all files required for <<prebuilt>>:
|
||||
|
||||
....
|
||||
./build-all -G
|
||||
./build --all-linux-components
|
||||
./release-zip
|
||||
....
|
||||
|
||||
|
||||
@@ -116,7 +116,7 @@ fi
|
||||
|
||||
if "$bench_linux_boot"; then
|
||||
cd "${root_dir}"
|
||||
"${root_dir}/build-all"
|
||||
"${root_dir}/build" --all
|
||||
"${root_dir}/bench-boot" --size 3
|
||||
cp "$(${root_dir}/getvar bench_boot)" "$new_dir"
|
||||
fi
|
||||
|
||||
172
build
Executable file
172
build
Executable file
@@ -0,0 +1,172 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import argparse
|
||||
import collections
|
||||
import os
|
||||
|
||||
import common
|
||||
|
||||
class Component():
|
||||
def __init__(self, default_selected, build_callback):
|
||||
self.default_selected = default_selected
|
||||
self.build_callback = build_callback
|
||||
def build(self, arch, dry_run):
|
||||
self.build_callback(arch, dry_run)
|
||||
|
||||
class BaremetalComponent(Component):
|
||||
def __init__(self, default_selected):
|
||||
self.default_selected = default_selected
|
||||
def build(self, arch, dry_run):
|
||||
common.run_cmd(['build-crosstool-ng', '--arch', arch], dry_run=dry_run)
|
||||
common.run_cmd(['build-baremetal', '--arch', arch], dry_run=dry_run)
|
||||
common.run_cmd(['build-baremetal', '--arch', arch, '--gem5'], dry_run=dry_run)
|
||||
common.run_cmd(['build-baremetal', '--arch', arch, '--gem5', '--machine', 'RealViewPBX'], dry_run=dry_run)
|
||||
|
||||
def add_bool_arg(parser, name, default=False):
|
||||
group = parser.add_mutually_exclusive_group(required=False)
|
||||
group.add_argument('--' + name, default=False, action='store_true')
|
||||
group.add_argument('--no-' + name, default=False, action='store_true')
|
||||
|
||||
def run_cmd(cmd, dry_run):
|
||||
cmd_abs = cmd.copy()
|
||||
cmd_abs[0] = os.path.join(common.root_dir, cmd[0])
|
||||
common.run_cmd(cmd_abs, dry_run=dry_run)
|
||||
|
||||
# Topological sorted on build dependencies.
|
||||
name_to_component_map = collections.OrderedDict([
|
||||
('baremetal', BaremetalComponent(False)),
|
||||
('gem5', Component(
|
||||
False,
|
||||
lambda arch, dry_run: run_cmd(['build-gem5', '--arch', arch], dry_run)
|
||||
)),
|
||||
('qemu', Component(
|
||||
True,
|
||||
lambda arch, dry_run: run_cmd(['build-qemu', '--arch', arch], dry_run)
|
||||
)),
|
||||
('linux', Component(
|
||||
True,
|
||||
lambda arch, dry_run: run_cmd(['build-linux', '--arch', arch], dry_run=dry_run)
|
||||
)),
|
||||
('modules', Component(
|
||||
True,
|
||||
lambda arch, dry_run: run_cmd(['build-modules', '--arch', arch], dry_run=dry_run)
|
||||
)),
|
||||
('userland', Component(
|
||||
True,
|
||||
lambda arch, dry_run: run_cmd(['build-userland', '--arch', arch], dry_run=dry_run),
|
||||
)),
|
||||
('buildroot', Component(
|
||||
True,
|
||||
lambda arch, dry_run: run_cmd(['build-buildroot', '--arch', arch, '--gem5'], dry_run=dry_run),
|
||||
)),
|
||||
])
|
||||
component_names = name_to_component_map.keys()
|
||||
linux_component_names = {
|
||||
'gem5',
|
||||
'qemu',
|
||||
'linux',
|
||||
'modules',
|
||||
'userland',
|
||||
'buildroot',
|
||||
}
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description= '''
|
||||
Shallow helper to build everything, or a subset of everything conveniently.
|
||||
|
||||
While developing something however, you will likely want to just run the
|
||||
required sub-build commands manually to speed things up and better understand
|
||||
what is going on.
|
||||
|
||||
Without any args, build only what is necessary for
|
||||
https://github.com/cirosantilli/linux-kernel-module-cheat#qemu-buildroot-setup
|
||||
|
||||
....
|
||||
./%(prog)s
|
||||
....
|
||||
|
||||
This includes:
|
||||
|
||||
* QEMU
|
||||
* Linux kernel
|
||||
* kernel modules and userland tools
|
||||
* Buildroot
|
||||
|
||||
just for x86_64.
|
||||
|
||||
To build EVERYTHING:
|
||||
|
||||
This will build QEMU, gem5, Buildroot, Linux, etc.
|
||||
for x86_64, arm and aarch64.
|
||||
|
||||
With --archs, build everything for just the given archs:
|
||||
|
||||
....
|
||||
./%(prog)s --archs 'arm aarch64'
|
||||
....
|
||||
|
||||
Other options make this script build only the given components. E.g., to build
|
||||
just Linux and QEMU for all archs, but not gem5, Buildroot, etc.:
|
||||
|
||||
....
|
||||
./%(prog)s --linux --qemu
|
||||
....
|
||||
|
||||
this is useful to while developing those components to prepare to quickly
|
||||
''',
|
||||
formatter_class=argparse.RawTextHelpFormatter,
|
||||
)
|
||||
parser.add_argument('--all', default=False, action='store_true', help='''\
|
||||
Build absolutely everything.
|
||||
''')
|
||||
parser.add_argument('--all-components', default=False, action='store_true', help='''\
|
||||
Build all components within the selected archs.
|
||||
''')
|
||||
parser.add_argument('--all-linux-components', default=False, action='store_true', help='''\
|
||||
Build all Linux-releated components within the selected archs.
|
||||
Excludes for example baremetal examples.
|
||||
''')
|
||||
group = parser.add_mutually_exclusive_group(required=False)
|
||||
group.add_argument('--all-archs', default=False, action='store_true', help='''\
|
||||
Build the selected components for all archs.
|
||||
''')
|
||||
group.add_argument('--arch', choices=common.arch_choices, default=[], action='append', help='''\
|
||||
Build the selected components for this arch. Select multiple arches by
|
||||
passing this option multiple times. Default: [{}]
|
||||
'''.format(common.default_arch))
|
||||
for component in component_names:
|
||||
add_bool_arg(parser, component)
|
||||
parser.add_argument('--dry-run', default=False, action='store_true', help='''\
|
||||
Print the commands that would be run, but don't run them.
|
||||
''')
|
||||
args = parser.parse_args()
|
||||
|
||||
# Decide archs.
|
||||
if args.arch == []:
|
||||
if args.all or args.all_archs:
|
||||
archs = common.all_archs.copy()
|
||||
else:
|
||||
archs = set([common.default_arch])
|
||||
else:
|
||||
archs = set()
|
||||
for arch in args.arch:
|
||||
if arch in common.arch_short_to_long_dict:
|
||||
arch = common.arch_short_to_long_dict[arch]
|
||||
archs.add(arch)
|
||||
|
||||
# Decide components.
|
||||
selected_component_names = []
|
||||
for name in component_names:
|
||||
component = name_to_component_map[name]
|
||||
if (
|
||||
args.all or
|
||||
args.all_components or
|
||||
(args.all_linux_components and name in linux_component_names) or
|
||||
getattr(args, name) or
|
||||
component.default_selected
|
||||
):
|
||||
selected_component_names.append(name)
|
||||
|
||||
for arch in archs:
|
||||
for name in selected_component_names:
|
||||
name_to_component_map[name].build(arch, args.dry_run)
|
||||
111
build-all
111
build-all
@@ -1,111 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# Shallow helper to build everything, or a subset of everything conveniently.
|
||||
#
|
||||
# While developing something however, you will likely want to just run the
|
||||
# required sub-build commands manually to speed things up and understand what
|
||||
# is going on.
|
||||
#
|
||||
# Without any args, build everything for all archs:
|
||||
#
|
||||
# ./build-all
|
||||
#
|
||||
# This will build QEMU, gem5, Buildroot, Linux, etc.
|
||||
# for x86_64, arm and aarch64.
|
||||
#
|
||||
# With --archs, build everything for just the given archs:
|
||||
#
|
||||
# ./build-all --archs 'arm aarch64'
|
||||
#
|
||||
# Other options make this script build only the given components. E.g., to build
|
||||
# just Linux and QEMU for all archs, but not gem5, Buildroot, etc.:
|
||||
#
|
||||
# ./build-all --linux --qemu
|
||||
#
|
||||
# this is useful to while developing those components to prepare to quickly
|
||||
|
||||
set -eu
|
||||
archs='x86_64 arm aarch64'
|
||||
baremetal=false
|
||||
buildroot=false
|
||||
gem5=false
|
||||
linux=false
|
||||
modules=false
|
||||
qemu=false
|
||||
userland=false
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--archs)
|
||||
archs="$2"
|
||||
shift 2
|
||||
;;
|
||||
--baremetal)
|
||||
baremetal=true
|
||||
shift
|
||||
;;
|
||||
--buildroot)
|
||||
buildroot=true
|
||||
shift
|
||||
;;
|
||||
--gem5)
|
||||
gem5=true
|
||||
shift
|
||||
;;
|
||||
--linux)
|
||||
linux=true
|
||||
shift
|
||||
;;
|
||||
--modules)
|
||||
modules=true
|
||||
shift
|
||||
;;
|
||||
--qemu)
|
||||
qemu=true
|
||||
shift
|
||||
;;
|
||||
--userland)
|
||||
userland=true
|
||||
shift
|
||||
;;
|
||||
esac
|
||||
done
|
||||
if ! (
|
||||
"$baremetal" || \
|
||||
"$buildroot" || \
|
||||
"$gem5" || \
|
||||
"$linux" || \
|
||||
"$qemu"
|
||||
)
|
||||
then
|
||||
baremetal=true
|
||||
buildroot=true
|
||||
gem5=true
|
||||
linux=true
|
||||
qemu=true
|
||||
fi
|
||||
for arch in $archs; do
|
||||
if "$qemu"; then
|
||||
./build-qemu --arch "$arch"
|
||||
fi
|
||||
if "$linux"; then
|
||||
./build-linux --arch "$arch"
|
||||
fi
|
||||
if "$modules"; then
|
||||
./build-modules
|
||||
fi
|
||||
if "$userland"; then
|
||||
./build-userland
|
||||
fi
|
||||
if "$baremetal" && [ ! "$arch" = x86_64 ]; then
|
||||
./build-crosstool-ng --arch "$arch"
|
||||
./build-baremetal --arch "$arch"
|
||||
./build-baremetal --arch "$arch" --gem5
|
||||
./build-baremetal --arch "$arch" --gem5 --machine RealViewPBX
|
||||
fi
|
||||
if "$buildroot"; then
|
||||
./build-buildroot --arch "$arch" --gem5 --kernel-modules "$@"
|
||||
fi
|
||||
if "$gem5"; then
|
||||
./build-gem5 --arch "$arch"
|
||||
fi
|
||||
done
|
||||
@@ -32,7 +32,6 @@ defaults = {
|
||||
def get_argparse():
|
||||
parser = common.get_argparse(argparse_args={'description':'Run Linux on an emulator'})
|
||||
common.add_build_arguments(parser)
|
||||
kernel_module_group = parser.add_mutually_exclusive_group()
|
||||
parser.add_argument(
|
||||
'-B', '--buildroot-config', default=defaults['buildroot_config'], action='append',
|
||||
help='''Add a single Buildroot config to the current build.
|
||||
@@ -93,7 +92,7 @@ Ignore all default kernel configurations and use this file instead.
|
||||
Still uses options explicitly passed with `-C` and `-c` on top of it.
|
||||
'''
|
||||
)
|
||||
kernel_module_group.add_argument(
|
||||
parser.add_argument(
|
||||
'-k', '--kernel-modules', default=defaults['kernel_modules'], action='store_true',
|
||||
help='''Reconfigure and rebuild the kernel modules package.
|
||||
Force --build-linux to true, since building the modules requires building Linux.
|
||||
|
||||
50
common.py
50
common.py
@@ -2,6 +2,7 @@
|
||||
|
||||
import argparse
|
||||
import base64
|
||||
import collections
|
||||
import copy
|
||||
import datetime
|
||||
import distutils.file_util
|
||||
@@ -44,12 +45,17 @@ qemu_src_dir = os.path.join(submodules_dir, 'qemu')
|
||||
parsec_benchmark_src_dir = os.path.join(submodules_dir, 'parsec-benchmark')
|
||||
ccache_dir = os.path.join('/usr', 'lib', 'ccache')
|
||||
default_build_id = 'default'
|
||||
arch_map = {
|
||||
'a': 'arm',
|
||||
'A': 'aarch64',
|
||||
'x': 'x86_64',
|
||||
}
|
||||
arches = [arch_map[k] for k in arch_map]
|
||||
arch_short_to_long_dict = collections.OrderedDict([
|
||||
('x', 'x86_64'),
|
||||
('a', 'arm'),
|
||||
('A', 'aarch64'),
|
||||
])
|
||||
all_archs = [arch_short_to_long_dict[k] for k in arch_short_to_long_dict]
|
||||
arch_choices = []
|
||||
for key in this.arch_short_to_long_dict:
|
||||
arch_choices.append(key)
|
||||
arch_choices.append(this.arch_short_to_long_dict[key])
|
||||
default_arch = 'x86_64'
|
||||
gem5_cpt_prefix = '^cpt\.'
|
||||
sha = subprocess.check_output(['git', '-C', root_dir, 'log', '-1', '--format=%H']).decode().rstrip()
|
||||
release_dir = os.path.join(this.out_dir, 'release')
|
||||
@@ -112,16 +118,12 @@ def get_argparse(default_args=None, argparse_args=None):
|
||||
default_args = {}
|
||||
if argparse_args is None:
|
||||
argparse_args = {}
|
||||
arch_choices = []
|
||||
for key in this.arch_map:
|
||||
arch_choices.append(key)
|
||||
arch_choices.append(this.arch_map[key])
|
||||
parser = argparse.ArgumentParser(
|
||||
formatter_class=argparse.RawTextHelpFormatter,
|
||||
**argparse_args
|
||||
)
|
||||
parser.add_argument(
|
||||
'-a', '--arch', choices=arch_choices, default='x86_64',
|
||||
'-a', '--arch', choices=this.arch_choices, default=this.default_arch,
|
||||
help='CPU architecture. Default: %(default)s'
|
||||
)
|
||||
parser.add_argument(
|
||||
@@ -166,8 +168,8 @@ See the documentation for other values known to work.
|
||||
'''
|
||||
)
|
||||
parser.add_argument(
|
||||
'-M', '--gem5-build-id', default=default_build_id,
|
||||
help='gem5 build ID. Allows you to keep multiple separate gem5 builds. Default: %(default)s'
|
||||
'-M', '--gem5-build-id',
|
||||
help='gem5 build ID. Allows you to keep multiple separate gem5 builds. Default: %(default)s'.format(default_build_id)
|
||||
)
|
||||
parser.add_argument(
|
||||
'-N', '--gem5-worktree',
|
||||
@@ -341,7 +343,7 @@ def make_build_dirs():
|
||||
|
||||
def make_run_dirs():
|
||||
'''
|
||||
Make directories rquired for the run.
|
||||
Make directories required for the run.
|
||||
The user could nuke those anytime between runs to try and clean things up.
|
||||
'''
|
||||
global this
|
||||
@@ -435,6 +437,7 @@ def run_cmd(
|
||||
extra_env=None,
|
||||
extra_paths=None,
|
||||
delete_env=None,
|
||||
dry_run=False,
|
||||
**kwargs
|
||||
):
|
||||
'''
|
||||
@@ -456,6 +459,9 @@ def run_cmd(
|
||||
|
||||
:param extra_env: extra environment variables to add when running the command
|
||||
:type extra_env: Dict[str,str]
|
||||
|
||||
:param dry_run: don't run the commands, just potentially print them. Debug aid.
|
||||
:type dry_run: Bool
|
||||
'''
|
||||
if out_file is not None:
|
||||
stdout = subprocess.PIPE
|
||||
@@ -501,6 +507,7 @@ def run_cmd(
|
||||
#sigpipe_old = signal.getsignal(signal.SIGPIPE)
|
||||
#signal.signal(signal.SIGPIPE, signal.SIG_DFL)
|
||||
|
||||
if not dry_run:
|
||||
# https://stackoverflow.com/questions/15535240/python-popen-write-to-stdout-and-log-file-simultaneously/52090802#52090802
|
||||
with subprocess.Popen(cmd, stdout=stdout, stderr=stderr, env=env, **kwargs) as proc:
|
||||
if out_file is not None:
|
||||
@@ -522,6 +529,8 @@ def run_cmd(
|
||||
signal.signal(signal.SIGINT, sigint_old)
|
||||
#signal.signal(signal.SIGPIPE, sigpipe_old)
|
||||
return proc.returncode
|
||||
else:
|
||||
return 0
|
||||
|
||||
def setup(parser):
|
||||
'''
|
||||
@@ -530,11 +539,14 @@ def setup(parser):
|
||||
'''
|
||||
global this
|
||||
args = parser.parse_args()
|
||||
if args.arch in this.arch_map:
|
||||
args.arch = this.arch_map[args.arch]
|
||||
# Because argparse sucks:
|
||||
# https://stackoverflow.com/questions/30487767/check-if-argparse-optional-argument-is-set-or-not
|
||||
if args.gem5_worktree is not None and args.gem5_build_id == default_build_id:
|
||||
if args.arch in this.arch_short_to_long_dict:
|
||||
args.arch = this.arch_short_to_long_dict[args.arch]
|
||||
if args.gem5_build_id is None:
|
||||
args.gem5_build_id = default_build_id
|
||||
gem5_build_id_given = False
|
||||
else:
|
||||
gem5_build_id_given = True
|
||||
if args.gem5_worktree is not None and not gem5_build_id_given:
|
||||
args.gem5_build_id = args.gem5_worktree
|
||||
this.machine = args.machine
|
||||
if args.arch == 'arm':
|
||||
|
||||
@@ -1,4 +1,12 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# Download build dependencies for the build, notably:
|
||||
#
|
||||
# - required git submodules
|
||||
# - host packages
|
||||
#
|
||||
# Only needs to be run once before each type of build per-system.
|
||||
|
||||
set -eux
|
||||
all=false
|
||||
apt_get=true
|
||||
2
release
2
release
@@ -24,7 +24,7 @@ start_time = time.time()
|
||||
# know what the smallest root filesystem size is and use it either...
|
||||
# https://stackoverflow.com/questions/47320800/how-to-clean-only-target-in-buildroot
|
||||
subprocess.check_call([os.path.join(common.root_dir, 'configure'), '--all'])
|
||||
subprocess.check_call([os.path.join(common.root_dir, 'build-all')])
|
||||
subprocess.check_call([os.path.join(common.root_dir, 'build'), '--all-linux-components'])
|
||||
release_zip.main()
|
||||
subprocess.check_call(['git', 'push'])
|
||||
release_upload.main()
|
||||
|
||||
@@ -15,7 +15,7 @@ def main():
|
||||
if os.path.exists(common.release_zip_file):
|
||||
os.unlink(common.release_zip_file)
|
||||
zipf = zipfile.ZipFile(common.release_zip_file, 'w', zipfile.ZIP_DEFLATED)
|
||||
for arch in common.arches:
|
||||
for arch in common.all_archs:
|
||||
common.setup(common.get_argparse(default_args={'arch': arch}))
|
||||
zipf.write(common.qcow2_file, arcname=os.path.relpath(common.qcow2_file, common.root_dir))
|
||||
zipf.write(common.linux_image, arcname=os.path.relpath(common.linux_image, common.root_dir))
|
||||
|
||||
Reference in New Issue
Block a user