diff --git a/README.adoc b/README.adoc index 423dae8..f296f42 100644 --- a/README.adoc +++ b/README.adoc @@ -13418,6 +13418,147 @@ and that is where the main event loop, `doSimLoop`, gets called and starts kicki Tested at gem5 b4879ae5b0b6644e6836b0881e4da05c64a6550d. +===== gem5 `m5.objects` module + +All `SimObjects` seem to be automatically added to the `m5.objects` namespace, and this is done in a very convoluted way, let's try to understand a bit: + +.... +src/python/m5/objects/__init__.py +.... + +contains: + +.... +modules = __loader__.modules + +for module in modules.keys(): + if module.startswith('m5.objects.'): + exec("from %s import *" % module) +.... + +And from <> we see that this appears to loop over every object string of type `m5.objects.modulename`. + +This `__init__` gets called from `src/python/importer.py` at the `exec`: + +.... +class CodeImporter(object): + def load_module(self, fullname): + override = os.environ.get('M5_OVERRIDE_PY_SOURCE', 'false').lower() + if override in ('true', 'yes') and os.path.exists(abspath): + src = open(abspath, 'r').read() + code = compile(src, abspath, 'exec') + + if os.path.basename(srcfile) == '__init__.py': + mod.__path__ = fullname.split('.') + mod.__package__ = fullname + else: + mod.__package__ = fullname.rpartition('.')[0] + mod.__file__ = srcfile + + exec(code, mod.__dict__) + +import sys +importer = CodeImporter() +add_module = importer.add_module +sys.meta_path.append(importer) +.... + +Here as a bonus here we also see how <> works. + +In `src/SConscript` we see that `SimObject` is just a `PySource` with module equals to `m5.objects`: + +.... +class SimObject(PySource): + def __init__(self, source, tags=None, add_tags=None): + '''Specify the source file and any tags (automatically in + the m5.objects package)''' + super(SimObject, self).__init__('m5.objects', source, tags, add_tags) +.... + +The `add_module` method seems to be doing the magic and is called from `src/sim/init.cc`: + +.... +bool +EmbeddedPython::addModule() const +{ + PyObject *code = getCode(); + PyObject *result = PyObject_CallMethod(importerModule, PyCC("add_module"), +.... + +which is called from: + +.... +int +EmbeddedPython::initAll() +{ + // Load the importer module + PyObject *code = importer->getCode(); + importerModule = PyImport_ExecCodeModule(PyCC("importer"), code); + if (!importerModule) { + PyErr_Print(); + return 1; + } + + // Load the rest of the embedded python files into the embedded + // python importer + list::iterator i = getList().begin(); + list::iterator end = getList().end(); + for (; i != end; ++i) + if (!(*i)->addModule()) +.... + +and `getList` comes from: + +.... +EmbeddedPython::EmbeddedPython(const char *filename, const char *abspath, + const char *modpath, const unsigned char *code, int zlen, int len) + : filename(filename), abspath(abspath), modpath(modpath), code(code), + zlen(zlen), len(len) +{ + // if we've added the importer keep track of it because we need it + // to bootstrap. + if (string(modpath) == string("importer")) + importer = this; + else + getList().push_back(this); +} + +list & +EmbeddedPython::getList() +{ + static list the_list; + return the_list; +} +.... + +and the constructor in turn gets called from per `SimObject` autogenerated files such as e.g. `dev/storage/Ide.py.cc` for `src/dev/storage/Ide.py`: + +.... +EmbeddedPython embedded_m5_objects_Ide( + "m5/objects/Ide.py", + "/home/ciro/bak/git/linux-kernel-module-cheat/data/gem5/master4/src/dev/storage/Ide.py", + "m5.objects.Ide", + data_m5_objects_Ide, + 947, + 2099); + +} // anonymous namespace +.... + +which get autogenerated at `src/SConscript`: + +.... +def embedPyFile(target, source, env): + +for source in PySource.all: + base_py_env.Command(source.cpp, [ py_marshal, source.tnode ], + MakeAction(embedPyFile, Transform("EMBED PY"))) +.... + +where the `PySource.all` thing as you might expect is a static list of all `PySource` source files as they get updated in the constructor. + +Tested in gem5 d9cb548d83fa81858599807f54b52e5be35a6b03. + ==== gem5 event queue gem5 is an event based simulator, and as such the event queue is of of the crucial elements in the system. @@ -16248,6 +16389,17 @@ Bibliography: https://www.mail-archive.com/gem5-users@gem5.org/msg16989.html ==== gem5 build system +[[m5-override-py-source]] +===== M5_OVERRIDE_PY_SOURCE + +https://stackoverflow.com/questions/52312070/how-to-modify-a-file-under-src-python-and-run-it-without-rebuilding-in-gem5 + +Running gem5 with the `M5_OVERRIDE_PY_SOURCE=true` environment variable allows you to modify a file under src/python and run it without rebuilding in gem5? + +We set this environment variable by default in our link:run[] script. + +How `M5_OVERRID_PY_SOURCE` works is shown at: <>. + ===== gem5 build broken on recent compiler version gem5 moves a bit slowly, and if your host compiler is very new, the gem5 build might be broken for it, e.g. this was the case for Ubuntu 19.10 with GCC 9 and gem5 62d75e7105fe172eb906d4f80f360ff8591d4178 from Dec 2019. diff --git a/run b/run index d891136..1676287 100755 --- a/run +++ b/run @@ -518,7 +518,7 @@ Extra options to append at the end of the emulator command line. gem5_exe_args = self.sh.shlex_split(self.env['gem5_exe_args']) if do_trace: gem5_exe_args.extend(['--debug-flags', trace_type, LF]) - # https://stackoverflow.com/questions/52312070/how-to-modify-a-file-under-src-python-and-run-it-without-rebuilding-in-gem5/52312071#52312071 + # https://cirosantilli.com/linux-kernel-module-cheat#m5-override-py-source extra_env['M5_OVERRIDE_PY_SOURCE'] = 'true' if self.env['trace_stdout']: debug_file = 'cout'