# Copyright (c) 2011-2019, Ulf Magnusson
# SPDX-License-Identifier: ISC
"""
Overview
========
Kconfiglib is a Python 2/3 library for scripting and extracting information
from Kconfig (https://www.kernel.org/doc/Documentation/kbuild/kconfig-language.txt)
configuration systems.
See the homepage at https://github.com/ulfalizer/Kconfiglib for a longer
overview.
Using Kconfiglib on the Linux kernel with the Makefile targets
==============================================================
For the Linux kernel, a handy interface is provided by the
scripts/kconfig/Makefile patch, which can be applied with either 'git am' or
the 'patch' utility:
$ wget -qO- https://raw.githubusercontent.com/ulfalizer/Kconfiglib/master/makefile.patch | git am
$ wget -qO- https://raw.githubusercontent.com/ulfalizer/Kconfiglib/master/makefile.patch | patch -p1
Warning: Not passing -p1 to patch will cause the wrong file to be patched.
Please tell me if the patch does not apply. It should be trivial to apply
manually, as it's just a block of text that needs to be inserted near the other
*conf: targets in scripts/kconfig/Makefile.
Look further down for a motivation for the Makefile patch and for instructions
on how you can use Kconfiglib without it.
If you do not wish to install Kconfiglib via pip, the Makefile patch is set up
so that you can also just clone Kconfiglib into the kernel root:
$ git clone git://github.com/ulfalizer/Kconfiglib.git
$ git am Kconfiglib/makefile.patch (or 'patch -p1 < Kconfiglib/makefile.patch')
Warning: The directory name Kconfiglib/ is significant in this case, because
it's added to PYTHONPATH by the new targets in makefile.patch.
The targets added by the Makefile patch are described in the following
sections.
make kmenuconfig
----------------
This target runs the curses menuconfig interface with Python 3 (Python 2 is
currently not supported for the menuconfig).
make [ARCH=<arch>] iscriptconfig
--------------------------------
This target gives an interactive Python prompt where a Kconfig instance has
been preloaded and is available in 'kconf'. To change the Python interpreter
used, pass PYTHONCMD=<executable> to make. The default is "python".
To get a feel for the API, try evaluating and printing the symbols in
kconf.defined_syms, and explore the MenuNode menu tree starting at
kconf.top_node by following 'next' and 'list' pointers.
The item contained in a menu node is found in MenuNode.item (note that this can
be one of the constants kconfiglib.MENU and kconfiglib.COMMENT), and all
symbols and choices have a 'nodes' attribute containing their menu nodes
(usually only one). Printing a menu node will print its item, in Kconfig
format.
If you want to look up a symbol by name, use the kconf.syms dictionary.
make scriptconfig SCRIPT=<script> [SCRIPT_ARG=<arg>]
----------------------------------------------------
This target runs the Python script given by the SCRIPT parameter on the
configuration. sys.argv[1] holds the name of the top-level Kconfig file
(currently always "Kconfig" in practice), and sys.argv[2] holds the SCRIPT_ARG
argument, if given.
See the examples/ subdirectory for example scripts.
make dumpvarsconfig
-------------------
This target prints a list of all environment variables referenced from the
Kconfig files, together with their values. See the
Kconfiglib/examples/dumpvars.py script.
Only environment variables that are referenced via the Kconfig preprocessor
$(FOO) syntax are included. The preprocessor was added in Linux 4.18.
Using Kconfiglib without the Makefile targets
=============================================
The make targets are only needed to pick up environment variables exported from
the Kbuild makefiles and referenced inside Kconfig files, via e.g.
'source "arch/$(SRCARCH)/Kconfig" and commands run via '$(shell,...)'.
These variables are referenced as of writing (Linux 4.18), together with sample
values:
srctree (.)
ARCH (x86)
SRCARCH (x86)
KERNELVERSION (4.18.0)
CC (gcc)
HOSTCC (gcc)
HOSTCXX (g++)
CC_VERSION_TEXT (gcc (Ubuntu 7.3.0-16ubuntu3) 7.3.0)
Older kernels only reference ARCH, SRCARCH, and KERNELVERSION.
If your kernel is recent enough (4.18+), you can get a list of referenced
environment variables via 'make dumpvarsconfig' (see above). Note that this
command is added by the Makefile patch.
To run Kconfiglib without the Makefile patch, set the environment variables
manually:
$ srctree=. ARCH=x86 SRCARCH=x86 KERNELVERSION=`make kernelversion` ... python(3)
>>> import kconfiglib
>>> kconf = kconfiglib.Kconfig() # filename defaults to "Kconfig"
Search the top-level Makefile for "Additional ARCH settings" to see other
possibilities for ARCH and SRCARCH.
Intro to symbol values
======================
Kconfiglib has the same assignment semantics as the C implementation.
Any symbol can be assigned a value by the user (via Kconfig.load_config() or
Symbol.set_value()), but this user value is only respected if the symbol is
visible, which corresponds to it (currently) being visible in the menuconfig
interface.
For symbols with prompts, the visibility of the symbol is determined by the
condition on the prompt. Symbols without prompts are never visible, so setting
a user value on them is pointless. A warning will be printed by default if
Symbol.set_value() is called on a promptless symbol. Assignments to promptless
symbols are normal within a .config file, so no similar warning will be printed
by load_config().
Dependencies from parents and 'if'/'depends on' are propagated to properties,
including prompts, so these two configurations are logically equivalent:
(1)
menu "menu"
depends on A
if B
config FOO
tristate "foo" if D
default y
depends on C
endif
endmenu
(2)
menu "menu"
depends on A
config FOO
tristate "foo" if A && B && C && D
default y if A && B && C
endmenu
In this example, A && B && C && D (the prompt condition) needs to be non-n for
FOO to be visible (assignable). If its value is m, the symbol can only be
assigned the value m: The visibility sets an upper bound on the value that can
be assigned by the user, and any higher user value will be truncated down.
'default' properties are independent of the visibility, though a 'default' will
often get the same condition as the prompt due to dependency propagation.
'default' properties are used if the symbol is not visible or has no user
value.
Symbols with no user value (or that have a user value but are not visible) and
no (active) 'default' default to n for bool/tristate symbols, and to the empty
string for other symbol types.
'select' works similarly to symbol visibility, but sets a lower bound on the
value of the symbol. The lower bound is determined by the value of the
select*ing* symbol. 'select' does not respect visibility, so non-visible
symbols can be forced to a particular (minimum) value by a select as well.
For non-bool/tristate symbols, it only matters whether the visibility is n or
non-n: m visibility acts the same as y visibility.
Conditions on 'default' and 'select' work in mostly intuitive ways. If the
condition is n, the 'default' or 'select' is disabled. If it is m, the
'default' or 'select' value (the value of the selecting symbol) is truncated
down to m.
When writing a configuration with Kconfig.write_config(), only symbols that are
visible, have an (active) default, or are selected will get written out (note
that this includes all symbols that would accept user values). Kconfiglib
matches the .config format produced by the C implementations down to the
character. This eases testing.
For a visible bool/tristate symbol FOO with value n, this line is written to
.config:
# CONFIG_FOO is not set
The point is to remember the user n selection (which might differ from the
default value the symbol would get), while at the same sticking to the rule
that undefined corresponds to n (.config uses Makefile format, making the line
above a comment). When the .config file is read back in, this line will be
treated the same as the following assignment:
CONFIG_FOO=n
In Kconfiglib, the set of (currently) assignable values for a bool/tristate
symbol appear in Symbol.assignable. For other symbol types, just check if
sym.visibility is non-0 (non-n) to see whether the user value will have an
effect.
Intro to the menu tree
======================
The menu structure, as seen in e.g. menuconfig, is represented by a tree of
MenuNode objects. The top node of the configuration corresponds to an implicit
top-level menu, the title of which is shown at the top in the standard
menuconfig interface. (The title is also available in Kconfig.mainmenu_text in
Kconfiglib.)
The top node is found in Kconfig.top_node. From there, you can visit child menu
nodes by following the 'list' pointer, and any following menu nodes by
following the 'next' pointer. Usually, a non-None 'list' pointer indicates a
menu or Choice, but menu nodes for symbols can sometimes have a non-None 'list'
pointer too due to submenus created implicitly from dependencies.
MenuNode.item is either a Symbol or a Choice object, or one of the constants
MENU and COMMENT. The prompt of the menu node can be found in MenuNode.prompt,
which also holds the title for menus and comments. For Symbol and Choice,
MenuNode.help holds the help text (if any, otherwise None).
Most symbols will only have a single menu node. A symbol defined in multiple
locations will have one menu node for each location. The list of menu nodes for
a Symbol or Choice can be found in the Symbol/Choice.nodes attribute.
Note that prompts and help texts for symbols and choices are stored in their
menu node(s) rather than in the Symbol or Choice objects themselves. This makes
it possible to define a symbol in multiple locations with a different prompt or
help text in each location. To get the help text or prompt for a symbol with a
single menu node, do sym.nodes[0].help and sym.nodes[0].prompt, respectively.
The prompt is a (text, condition) tuple, where condition determines the
visibility (see 'Intro to expressions' below).
This organization mirrors the C implementation. MenuNode is called
'struct menu' there, but I thought "menu" was a confusing name.
It is possible to give a Choice a name and define it in multiple locations,
hence why Choice.nodes is also a list.
As a convenience, the properties added at a particular definition location are
available on the MenuNode itself, in e.g. MenuNode.defaults. This is helpful
when generating documentation, so that symbols/choices defined in multiple
locations can be shown with the correct properties at each location.
Intro to expressions
====================
Expressions can be evaluated with the expr_value() function and printed with
the expr_str() function (these are used internally as well). Evaluating an
expression always yields a tristate value, where n, m, and y are represented as
0, 1, and 2, respectively.
The following table should help you figure out how expressions are represented.
A, B, C, ... are symbols (Symbol instances), NOT is the kconfiglib.NOT
constant, etc.
Expression Representation
---------- --------------
A A
"A" A (constant symbol)
!A (NOT, A)
A && B (AND, A, B)
A && B && C (AND, A, (AND, B, C))
A || B (OR, A, B)
A || (B && C && D) (OR, A, (AND, B, (AND, C, D)))
A = B (EQUAL, A, B)
A != "foo" (UNEQUAL, A, foo (constant symbol))
A && B = C && D (AND, A, (AND, (EQUAL, B, C), D))
n Kconfig.n (constant symbol)
m Kconfig.m (constant symbol)
y Kconfig.y (constant symbol)
"y" Kconfig.y (constant symbol)
Strings like "foo" in 'default "foo"' or 'depends on SYM = "foo"' are
represented as constant symbols, so the only values that appear in expressions
are symbols***. This mirrors the C implementation.
***For choice symbols, the parent Choice will appear in expressions as well,
but it's usually invisible as the value interfaces of Symbol and Choice are
identical. This mirrors the C implementation and makes different choice modes
"just work".
Manual evaluation examples:
- The value of A && B is min(A.tri_value, B.tri_value)
- The value of A || B is max(A.tri_value, B.tri_value)
- The value of !A is 2 - A.tri_value
- The value of A = B is 2 (y) if A.str_value == B.str_value, and 0 (n)
otherwise. Note that str_value is used here instead of tri_value.
For constant (as well as undefined) symbols, str_value matches the name of
the symbol. This mirrors the C implementation and explains why
'depends on SYM = "foo"' above works as expected.
n/m/y are automatically converted to the corresponding constant symbols
"n"/"m"/"y" (Kconfig.n/m/y) during parsing.
Kconfig.const_syms is a dictionary like Kconfig.syms but for constant symbols.
If a condition is missing (e.g., <cond> when the 'if <cond>' is removed from
'default A if <cond>'), it is actually Kconfig.y. The standard __str__()
functions just avoid printing 'if y' conditions to give cleaner output.
Kconfig extensions
==================
Kconfiglib includes a couple of Kconfig extensions:
'source' with relative path
---------------------------
The 'rsource' statement sources Kconfig files with a path relative to directory
of the Kconfig file containing the 'rsource' statement, instead of relative to
the project root.
Consider following directory tree:
Project
+--Kconfig
|
+--src
+--Kconfig
|
+--SubSystem1
+--Kconfig
|
+--ModuleA
+--Kconfig
In this example, assume that src/SubSystem1/Kconfig wants to source
src/SubSystem1/ModuleA/Kconfig.
With 'source', this statement would be used:
source "src/SubSystem1/ModuleA/Kconfig"
With 'rsource', this turns into
rsource "ModuleA/Kconfig"
If an absolute path is given to 'rsource', it acts the same as 'source'.
'rsource' can be used to create "position-independent" Kconfig trees that can
be moved around freely.
Globbing 'source'
-----------------
'source' and 'rsource' accept glob patterns, sourcing all matching Kconfig
files. They require at least one matching file, throwing a KconfigError
otherwise.
For example, the following statement might source sub1/foofoofoo and
sub2/foobarfoo:
source "sub[12]/foo*foo"
The glob patterns accepted are the same as for the standard glob.glob()
function.
Two additional statements are provided for cases where it's acceptable for a
pattern to match no files: 'osource' and 'orsource' (the o is for "optional").
For example, the following statements will be no-ops if neither "foo" nor any
files matching "bar*" exist:
osource "foo"
osource "bar*"
'orsource' does a relative optional source.
'source' and 'osource' are analogous to 'include' and '-include' in Make.
Generalized def_* keywords
--------------------------
def_int, def_hex, and def_string are available in addition to def_bool and
def_tristate, allowing int, hex, and string symbols to be given a type and a
default at the same time.
Extra optional warnings
-----------------------
Some optional warnings can be controlled via environment variables:
- KCONFIG_WARN_UNDEF: If set to 'y', warnings will be generated for all
references to undefined symbols within Kconfig files. The only gotcha is
that all hex literals must be prefixed with "0x" or "0X", to make it
possible to distinguish them from symbol references.
Some projects (e.g. the Linux kernel) use multiple Kconfig trees with many
shared Kconfig files, leading to some safe undefined symbol references.
KCONFIG_WARN_UNDEF is useful in projects that only have a single Kconfig
tree though.
KCONFIG_STRICT is an older alias for this environment variable, supported
for backwards compatibility.
- KCONFIG_WARN_UNDEF_ASSIGN: If set to 'y', warnings will be generated for
all assignments to undefined symbols within .config files. By default, no
such warnings are generated.
This warning can also be enabled/disabled via
Kconfig.enable/disable_undef_warnings().
Preprocessor user functions defined in Python
---------------------------------------------
Preprocessor functions can be defined in Python, which makes it simple to
integrate information from existing Python tools into Kconfig (e.g. to have
Kconfig symbols depend on hardware information stored in some other format).
Putting a Python module named kconfigfunctions(.py) anywhere in sys.path will
cause it to be imported by Kconfiglib (in Kconfig.__init__()). Note that
sys.path can be customized via PYTHONPATH, and includes the directory of the
module being run by default, as well as installation directories.
If the KCONFIG_FUNCTIONS environment variable is set, it gives a different
module name to use instead of 'kconfigfunctions'.
The imported module is expected to define a global dictionary named 'functions'
that maps function names to Python functions, as follows:
def my_fn(kconf, name, arg_1, arg_2, ...):
# kconf:
# Kconfig instance
#
# name:
# Name of the user-defined function ("my-fn"). Think argv[0].
#
# arg_1, arg_2, ...:
# Arguments passed to the function from Kconfig (strings)
#
# Returns a string to be substituted as the result of calling the
# function
...
def my_other_fn(kconf, name, arg_1, arg_2, ...):
...
functions = {
"my-fn": (my_fn, <min.args>, <max.args>/None),
"my-other-fn": (my_other_fn, <min.args>, <max.args>/None),
...
}
...
<min.args> and <max.args> are the minimum and maximum number of arguments
expected by the function (excluding the implicit 'name' argument). If
<max.args> is None, there is no upper limit to the number of arguments. Passing
an invalid number of arguments will generate a KconfigError exception.
Once defined, user functions can be called from Kconfig in the same way as
other preprocessor functions:
config FOO
...
depends on $(my-fn,arg1,arg2)
If my_fn() returns "n", this will result in
config FOO
...
depends on n
Warning
*******
User-defined preprocessor functions are called as they're encountered at parse
time, before all Kconfig files have been processed, and before the menu tree
has been finalized. There are no guarantees that accessing Kconfig symbols or
the menu tree via the 'kconf' parameter will work, and it could potentially
lead to a crash. The 'kconf' parameter is provided for future extension (and
because the predefined functions take it anyway).
Preferably, user-defined functions should be stateless.
Feedback
========
Send bug reports, suggestions, and questions to ulfalizer a.t Google's email
service, or open a ticket on the GitHub page.
"""
import errno
import glob
import importlib
import os
import platform
import re
import subprocess
import sys
# File layout:
#
# Public classes
# Public functions
# Internal functions
# Public global constants
# Internal global constants
# Line length: 79 columns
#
# Public classes
#
class Kconfig(object):
"""
Represents a Kconfig configuration, e.g. for x86 or ARM. This is the set of
symbols, choices, and menu nodes appearing in the configuration. Creating
any number of Kconfig objects (including for different architectures) is
safe. Kconfiglib doesn't keep any global state.
The following attributes are available. They should be treated as
read-only, and some are implemented through @property magic.
syms:
A dictionary with all symbols in the configuration, indexed by name. Also
includes all symbols that are referenced in expressions but never
defined, except for constant (quoted) symbols.
Undefined symbols can be recognized by Symbol.nodes being empty -- see
the 'Intro to the menu tree' section in the module docstring.
const_syms:
A dictionary like 'syms' for constant (quoted) symbols
named_choices:
A dictionary like 'syms' for named choices (choice FOO)
defined_syms:
A list with all defined symbols, in the same order as they appear in the
Kconfig files. Symbols defined in multiple locations appear multiple
times.
Note: You probably want to use 'unique_defined_syms' instead. This
attribute is mostly maintained for backwards compatibility.
unique_defined_syms:
A list like 'defined_syms', but with duplicates removed. Just the first
instance is kept for symbols defined in multiple locations. Kconfig order
is preserved otherwise.
Using this attribute instead of 'defined_syms' can save work, and
automatically gives reasonable behavior when writing configuration output
(symbols defined in multiple locations only generate output once, while
still preserving Kconfig order for readability).
choices:
A list with all choices, in the same order as they appear in the Kconfig
files.
Note: You probably want to use 'unique_choices' instead. This attribute
is mostly maintained for backwards compatibility.
unique_choices:
Analogous to 'unique_defined_syms', for choices. Named choices can have
multiple definition locations.
menus:
A list with all menus, in the same order as they appear in the Kconfig
files
comments:
A list with all comments, in the same order as they appear in the Kconfig
files
kconfig_filenames:
A list with the filenames of all Kconfig files included in the
configuration, relative to $srctree (or relative to the current directory
if $srctree isn't set), except absolute paths (e.g.
'source "/foo/Kconfig") are kept as-is.
The files are listed in the order they are source'd, starting with the
top-level Kconfig file. If a file is source'd multiple times, it will
appear multiple times. Use set() to get unique filenames.
Note: Using this for incremental builds is redundant. Kconfig.sync_deps()
already indirectly catches any file modifications that change the
configuration output.
env_vars:
A set() with the names of all environment variables referenced in the
Kconfig files.
Only environment variables referenced with the preprocessor $(FOO) syntax
will be registered. The older $FOO syntax is only supported for backwards
compatibility.
Also note that $(FOO) won't be registered unless the environment variable
$FOO is actually set. If it isn't, $(FOO) is an expansion of an unset
preprocessor variable (which gives the empty string).
Another gotcha is that environment variables referenced in the values of
recursively expanded preprocessor variables (those defined with =) will
only be registered if the variable is actually used (expanded) somewhere.
The note from the 'kconfig_filenames' documentation applies here too.
n/m/y:
The predefined constant symbols n/m/y. Also available in const_syms.
modules:
The Symbol instance for the modules symbol. Currently hardcoded to
MODULES, which is backwards compatible. Kconfiglib will warn if
'option modules' is set on some other symbol. Tell me if you need proper
'option modules' support.
'modules' is never None. If the MODULES symbol is not explicitly defined,
its tri_value will be 0 (n), as expected.
A simple way to enable modules is to do 'kconf.modules.set_value(2)'
(provided the MODULES symbol is defined and visible). Modules are
disabled by default in the kernel Kconfig files as of writing, though
nearly all defconfig files enable them (with 'CONFIG_MODULES=y').
defconfig_list:
The Symbol instance for the 'option defconfig_list' symbol, or None if no
defconfig_list symbol exists. The defconfig filename derived from this
symbol can be found in Kconfig.defconfig_filename.
defconfig_filename:
The filename given by the defconfig_list symbol. This is taken from the
first 'default' with a satisfied condition where the specified file
exists (can be opened for reading). If a defconfig file foo/defconfig is
not found and $srctree was set when the Kconfig was created,
$srctree/foo/defconfig is looked up as well.
'defconfig_filename' is None if either no defconfig_list symbol exists,
or if the defconfig_list symbol has no 'default' with a satisfied
condition that specifies a file that exists.
Gotcha: scripts/kconfig/Makefile might pass --defconfig=<defconfig> to
scripts/kconfig/conf when running e.g. 'make defconfig'. This option
overrides the defconfig_list symbol, meaning defconfig_filename might not
always match what 'make defconfig' would use.
top_node:
The menu node (see the MenuNode class) of the implicit top-level menu.
Acts as the root of the menu tree.
mainmenu_text:
The prompt (title) of the top menu (top_node). Defaults to "Main menu".
Can be changed with the 'mainmenu' statement (see kconfig-language.txt).
variables:
A dictionary with all preprocessor variables, indexed by name. See the
Variable class.
warnings:
A list of strings containing all warnings that have been generated. This
allows flexibility in how warnings are printed and processed.
See the 'warn_to_stderr' parameter to Kconfig.__init__() and the
Kconfig.enable/disable_stderr_warnings() functions as well. Note that
warnings still get added to Kconfig.warnings when 'warn_to_stderr' is
True.
Just as for warnings printed to stderr, only optional warnings that are
enabled will get added to Kconfig.warnings. See the various
Kconfig.enable/disable_*_warnings() functions.
missing_syms:
A list with (name, value) tuples for all assignments to undefined symbols
within the most recently loaded .config file(s). 'name' is the symbol
name without the 'CONFIG_' prefix. 'value' is a string that gives the
right-hand side of the assignment verbatim.
See Kconfig.load_config() as well.
srctree:
The value of the $srctree environment variable when the configuration was
loaded, or the empty string if $srctree wasn't set. This gives nice
behavior with os.path.join(), which treats "" as the current directory,
without adding "./".
Kconfig files are looked up relative to $srctree (unless absolute paths
are used), and .config files are looked up relative to $srctree if they
are not found in the current directory. This is used to support
out-of-tree builds. The C tools use this environment variable in the same
way.
Changing $srctree after creating the Kconfig instance has no effect. Only
the value when the configuration is loaded matters. This avoids surprises
if multiple configurations are loaded with different values for $srctree.
config_prefix:
The value of the $CONFIG_ environment variable when the configuration was
loaded. This is the prefix used (and expected) on symbol names in .config
files and C headers. Defaults to "CONFIG_". Used in the same way in the C
tools.
Like for srctree, only the value of $CONFIG_ when the configuration is
loaded matters.
"""
__slots__ = (
"_encoding",
"_functions",
"_set_match",
"_unset_match",
"_warn_for_no_prompt",
"_warn_for_override",
"_warn_for_redun_assign",
"_warn_for_undef_assign",
"_warn_to_stderr",
"_warnings_enabled",
"choices",
"comments",
"config_prefix",
"const_syms",
"defconfig_list",
"defined_syms",
"env_vars",
"kconfig_filenames",
"m",
"mainmenu_text",
"menus",
"missing_syms",
"modules",
"n",
"named_choices",
"srctree",
"syms",
"top_node",
"unique_choices",
"unique_defined_syms",
"variables",
"warnings",
"y",
# Parsing-related
"_parsing_kconfigs",
"_readline",
"_filename",
"_linenr",
"_include_path",
"_filestack",
"_line",
"_tokens",
"_tokens_i",
"_reuse_tokens",
)
#
# Public interface
#
def __init__(self, filename="Kconfig", warn=True, warn_to_stderr=True,
encoding="utf-8"):
"""
Creates a new Kconfig object by parsing Kconfig files.
Note that Kconfig files are not the same as .config files (which store
configuration symbol values).
See the module docstring for some environment variables that influence
default warning settings (KCONFIG_WARN_UNDEF and
KCONFIG_WARN_UNDEF_ASSIGN).
Raises KconfigError on syntax errors, and (possibly a subclass of)
IOError on IO errors ('errno', 'strerror', and 'filename' are
available). Note that IOError can be caught as OSError on Python 3.
filename (default: "Kconfig"):
The Kconfig file to load. For the Linux kernel, you'll want "Kconfig"
from the top-level directory, as environment variables will make sure
the right Kconfig is included from there (arch/$SRCARCH/Kconfig as of
writing).
If $srctree is set, 'filename' will be looked up relative to it.
$srctree is also used to look up source'd files within Kconfig files.
See the class documentation.
If you are using Kconfiglib via 'make scriptconfig', the filename of
the base base Kconfig file will be in sys.argv[1]. It's currently
always "Kconfig" in practice.
warn (default: True):
True if warnings related to this configuration should be generated.
This can be changed later with Kconfig.enable/disable_warnings(). It
is provided as a constructor argument since warnings might be
generated during parsing.
See the other Kconfig.enable_*_warnings() functions as well, which
enable or suppress certain warnings when warnings are enabled.
All generated warnings are added to the Kconfig.warnings list. See
the class documentation.
warn_to_stderr (default: True):
True if warnings should be printed to stderr in addition to being
added to Kconfig.warnings.
This can be changed later with
Kconfig.enable/disable_stderr_warnings().
encoding (default: "utf-8"):
The encoding to use when reading and writing files. If None, the
encoding specified in the current locale will be used.
The "utf-8" default avoids exceptions on systems that are configured
to use the C locale, which implies an ASCII encoding.
This parameter has no effect on Python 2, due to implementation
issues (regular strings turning into Unicode strings, which are
distinct in Python 2). Python 2 doesn't decode regular strings
anyway.
Related PEP: https://www.python.org/dev/peps/pep-0538/
"""
self.srctree = os.environ.get("srctree", "")
self.config_prefix = os.environ.get("CONFIG_", "CONFIG_")
# Regular expressions for parsing .config files
self._set_match = _re_match(self.config_prefix + r"([^=]+)=(.*)")
self._unset_match = \
_re_match(r"# {}([^ ]+) is not set".format(self.config_prefix))
self.warnings = []
self._warnings_enabled = warn
self._warn_to_stderr = warn_to_stderr
self._warn_for_undef_assign = \
os.environ.get("KCONFIG_WARN_UNDEF_ASSIGN") == "y"
self._warn_for_redun_assign = self._warn_for_override = True
self._encoding = encoding
self.syms = {}
self.const_syms = {}
self.defined_syms = []
self.missing_syms = []
self.named_choices = {}
self.choices = []
self.menus = []
self.comments = []
for nmy in "n", "m", "y":
sym = Symbol()
sym.kconfig = self
sym.name = nmy
sym.is_constant = True
sym.orig_type = TRISTATE
sym._cached_tri_val = STR_TO_TRI[nmy]
self.const_syms[nmy] = sym
self.n = self.const_syms["n"]
self.m = self.const_syms["m"]
self.y = self.const_syms["y"]
# Make n/m/y well-formed symbols
for nmy in "n", "m", "y":
sym = self.const_syms[nmy]
sym.rev_dep = sym.weak_rev_dep = sym.direct_dep = self.n
# Maps preprocessor variables names to Variable instances
self.variables = {}
# Predefined preprocessor functions, with min/max number of arguments
self._functions = {
"info": (_info_fn, 1, 1),
"error-if": (_error_if_fn, 2, 2),
"filename": (_filename_fn, 0, 0),
"lineno": (_lineno_fn, 0, 0),
"shell": (_shell_fn, 1, 1),
"warning-if": (_warning_if_fn, 2, 2),
}
# Add any user-defined preprocessor functions
try:
self._functions.update(
importlib.import_module(
os.environ.get("KCONFIG_FUNCTIONS", "kconfigfunctions")
).functions)
except ImportError:
pass
# This is used to determine whether previously unseen symbols should be
# registered. They shouldn't be if we parse expressions after parsing,
# as part of Kconfig.eval_string().
self._parsing_kconfigs = True
self.modules = self._lookup_sym("MODULES")
self.defconfig_list = None
self.top_node = MenuNode()
self.top_node.kconfig = self
self.top_node.item = MENU
self.top_node.is_menuconfig = True
self.top_node.visibility = self.y
self.top_node.prompt = ("Main menu", self.y)
self.top_node.parent = None
self.top_node.dep = self.y
self.top_node.filename = filename
self.top_node.linenr = 1
self.top_node.include_path = ()
# Parse the Kconfig files
# Not used internally. Provided as a convenience.
self.kconfig_filenames = [filename]
self.env_vars = set()
# Used to avoid retokenizing lines when we discover that they're not
# part of the construct currently being parsed. This is kinda like an
# unget operation.
self._reuse_tokens = False
# Keeps track of the location in the parent Kconfig files. Kconfig
# files usually source other Kconfig files. See _enter_file().
self._filestack = []
self._include_path = ()
# The current parsing location
self._filename = filename
self._linenr = 0
# Open the top-level Kconfig file. Store the readline() method directly
# as a small optimization.
self._readline = \
self._open(os.path.join(self.srctree, filename), "r").readline
try:
# Parse everything
self._parse_block(None, self.top_node, self.top_node)
except UnicodeDecodeError as e:
_decoding_error(e, self._filename)
# Close the top-level Kconfig file. __self__ fetches the 'file' object
# for the method.
self._readline.__self__.close()
self.top_node.list = self.top_node.next
self.top_node.next = None
self._parsing_kconfigs = False
self.unique_defined_syms = _ordered_unique(self.defined_syms)
self.unique_choices = _ordered_unique(self.choices)
# Do various post-processing of the menu tree
self._finalize_tree(self.top_node, self.y)
# Do sanity checks. Some of these depend on everything being finalized.
self._check_sym_sanity()
self._check_choice_sanity()
# KCONFIG_STRICT is an older alias for KCONFIG_WARN_UNDEF, supported
# for backwards compatibility
if os.environ.get("KCONFIG_WARN_UNDEF") == "y" or \
os.environ.get("KCONFIG_STRICT") == "y":
self._check_undef_syms()
# Build Symbol._dependents for all symbols and choices
self._build_dep()
# Check for dependency loops
check_dep_loop_sym = _check_dep_loop_sym # Micro-optimization
for sym in self.unique_defined_syms:
check_dep_loop_sym(sym, False)
# Add extra dependencies from choices to choice symbols that get
# awkward during dependency loop detection
self._add_choice_deps()
self._warn_for_no_prompt = True
self.mainmenu_text = self.top_node.prompt[0]
@property
def defconfig_filename(self):
"""
See the class documentation.
"""
if self.defconfig_list:
for filename, cond in self.defconfig_list.defaults:
if expr_value(cond):
try:
with self._open_config(filename.str_value) as f:
return f.name
except IOError:
continue
return None
def load_config(self, filename=None, replace=True, verbose=True):
"""
Loads symbol values from a file in the .config format. Equivalent to
calling Symbol.set_value() to set each of the values.
"# CONFIG_FOO is not set" within a .config file sets the user value of
FOO to n. The C tools work the same way.
For each symbol, the Symbol.user_value attribute holds the value the
symbol was assigned in the .config file (if any). The user value might
differ from Symbol.str/tri_value if there are unsatisfied dependencies.
Calling this function also updates the Kconfig.missing_syms attribute
with a list of all assignments to undefined symbols within the
configuration file. Kconfig.missing_syms is cleared if 'replace' is
True, and appended to otherwise. See the documentation for
Kconfig.missing_syms as well.
Raises (possibly a subclass of) IOError on IO errors ('errno',
'strerror', and 'filename' are available). Note that IOError can be
caught as OSError on Python 3.
filename (default: None):
Path to load configuration from (a string). Respects $srctree if set
(see the class documentation).
If 'filename' is None (the default), the configuration file to load
(if any) is calculated automatically, giving the behavior you'd
usually want:
1. If the KCONFIG_CONFIG environment variable is set, it gives the
path to the configuration file to load. Otherwise, ".config" is
used. See standard_config_filename().
2. If the path from (1.) doesn't exist, the configuration file
given by kconf.defconfig_filename is loaded instead, which is
derived from the 'option defconfig_list' symbol.
3. If (1.) and (2.) fail to find a configuration file to load, no
configuration file is loaded, and symbols retain their current
values (e.g., their default values). This is not an error.
See the return value as well.
replace (default: True):
If True, all existing user values will be cleared before loading the
.config. Pass False to merge configurations.
verbose (default: True):
If True and filename is None (automatically infer configuration
file), a message will be printed to stdout telling which file got
loaded (or that no file got loaded). This is meant to reduce
boilerplate in tools.
Returns True if an existing configuration was loaded (that didn't come
from the 'option defconfig_list' symbol), and False otherwise. This is
mostly useful in conjunction with filename=None, as True will always be
returned otherwise.
"""
loaded_existing = True
if filename is None:
filename = standard_config_filename()
if os.path.exists(filename):
if verbose:
print("Using existing configuration '{}' as base"
.format(filename))
else:
filename = self.defconfig_filename
if filename is None:
if verbose:
print("Using default symbol values as base")
return False
if verbose:
print("Using default configuration found in '{}' as "
"base".format(filename))
loaded_existing = False
# Disable the warning about assigning to symbols without prompts. This
# is normal and expected within a .config file.
self._warn_for_no_prompt = False
# This stub only exists to make sure _warn_for_no_prompt gets reenabled
try:
self._load_config(filename, replace)
except UnicodeDecodeError as e:
_decoding_error(e, filename)
finally:
self._warn_for_no_prompt = True
return loaded_existing
def _load_config(self, filename, replace):
with self._open_config(filename) as f:
if replace:
self.missing_syms = []
# If we're replacing the configuration, keep track of which
# symbols and choices got set so that we can unset the rest
# later. This avoids invalidating everything and is faster.
# Another benefit is that invalidation must be rock solid for
# it to work, making it a good test.
for sym in self.unique_defined_syms:
sym._was_set = False
for choice in self.unique_choices:
choice._was_set = False
# Small optimizations
set_match = self._set_match
unset_match = self._unset_match
syms = self.syms
for linenr, line in enumerate(f, 1):
# The C tools ignore trailing whitespace
line = line.rstrip()
match = set_match(line)
if match:
name, val = match.groups()
if name not in syms:
self._undef_assign(name, val, filename, linenr)
continue
sym = syms[name]
if not sym.nodes:
self._undef_assign(name, val, filename, linenr)
continue
if sym.orig_type in _BOOL_TRISTATE:
# The C implementation only checks the first character
# to the right of '=', for whatever reason
if not ((sym.orig_type is BOOL and
val.startswith(("y", "n"))) or
(sym.orig_type is TRISTATE and
val.startswith(("y", "m", "n")))):
self._warn("'{}' is not a valid value for the {} "
"symbol {}. Assignment ignored."
.format(val, TYPE_TO_STR[sym.orig_type],
_name_and_loc(sym)),
filename, linenr)
continue
val = val[0]
if sym.choice and val != "n":
# During .config loading, we infer the mode of the
# choice from the kind of values that are assigned
# to the choice symbols
prev_mode = sym.choice.user_value
if prev_mode is not None and \
TRI_TO_STR[prev_mode] != val:
self._warn("both m and y assigned to symbols "
"within the same choice",
filename, linenr)
# Set the choice's mode
sym.choice.set_value(val)
elif sym.orig_type is STRING:
match = _conf_string_match(val)
if not match:
self._warn("malformed string literal in "
"assignment to {}. Assignment ignored."
.format(_name_and_loc(sym)),
filename, linenr)
continue
val = unescape(match.group(1))
else:
match = unset_match(line)
if not match:
# Print a warning for lines that match neither
# set_match() nor unset_match() and that are not blank
# lines or comments. 'line' has already been
# rstrip()'d, so blank lines show up as "" here.
if line and not line.lstrip().startswith("#"):
self._warn("ignoring malformed line '{}'"
.format(line),
filename, linenr)
continue
name = match.group(1)
if name not in syms:
self._undef_assign(name, "n", filename, linenr)
continue
sym = syms[name]
if not sym.nodes:
self._undef_assign(name, "n", filename, linenr)
continue
if sym.orig_type not in _BOOL_TRISTATE:
continue
val = "n"
# Done parsing the assignment. Set the value.
if sym._was_set:
# Use strings for bool/tristate user values in the warning
if sym.orig_type in _BOOL_TRISTATE:
display_user_val = TRI_TO_STR[sym.user_value]
else:
display_user_val = sym.user_value
msg = '{} set more than once. Old value: "{}", new value: "{}".'.format(
_name_and_loc(sym), display_user_val, val
)
if display_user_val == val:
self._warn_redun_assign(msg, filename, linenr)
else:
self._warn_override(msg, filename, linenr)
sym.set_value(val)
if replace:
# If we're replacing the configuration, unset the symbols that
# didn't get set
for sym in self.unique_defined_syms:
if not sym._was_set:
sym.unset_value()
for choice in self.unique_choices:
if not choice._was_set:
choice.unset_value()
def _undef_assign(self, name, val, filename, linenr):
# Called for assignments to undefined symbols during .config loading
self.missing_syms.append((name, val))
if self._warn_for_undef_assign:
self._warn(
"attempt to assign the value '{}' to the undefined symbol {}"
.format(val, name), filename, linenr)
def write_autoconf(self, filename,
header="/* Generated by Kconfiglib (https://github.com/ulfalizer/Kconfiglib) */\n"):
r"""
Writes out symbol values as a C header file, matching the format used
by include/generated/autoconf.h in the kernel.
The ordering of the #defines matches the one generated by
write_config(). The order in the C implementation depends on the hash
table implementation as of writing, and so won't match.
filename:
Self-explanatory.
header (default: "/* Generated by Kconfiglib (https://github.com/ulfalizer/Kconfiglib) */\n"):
Text that will be inserted verbatim at the beginning of the file. You
would usually want it enclosed in '/* */' to make it a C comment,
and include a final terminating newline.
"""
with self._open(filename, "w") as f:
f.write(header)
for sym in self.unique_defined_syms:
# Note: _write_to_conf is determined when the value is
# calculated. This is a hidden function call due to
# property magic.
val = sym.str_value
if sym._write_to_conf:
if sym.orig_type in _BOOL_TRISTATE:
if val != "n":
f.write("#define {}{}{} 1\n"
.format(self.config_prefix, sym.name,
"_MODULE" if val == "m" else ""))
elif sym.orig_type is STRING:
f.write('#define {}{} "{}"\n'
.format(self.config_prefix, sym.name,
escape(val)))
else: # sym.orig_type in _INT_HEX:
if sym.orig_type is HEX and \
not val.startswith(("0x", "0X")):
val = "0x" + val
f.write("#define {}{} {}\n"
.format(self.config_prefix, sym.name, val))
def write_config(self, filename=None,
header="# Generated by Kconfiglib (https://github.com/ulfalizer/Kconfiglib)\n",
save_old=True, verbose=True):
r"""
Writes out symbol values in the .config format. The format matches the
C implementation, including ordering.
Symbols appear in the same order in generated .config files as they do
in the Kconfig files. For symbols defined in multiple locations, a
single assignment is written out corresponding to the first location
where the symbol is defined.
See the 'Intro to symbol values' section in the module docstring to
understand which symbols get written out.
filename (default: None):
Filename to save configuration to (a string).
If None (the default), the filename in the the environment variable
KCONFIG_CONFIG is used if set, and ".config" otherwise. See
standard_config_filename().
header (default: "# Generated by Kconfiglib (https://github.com/ulfalizer/Kconfiglib)\n"):
Text that will be inserted verbatim at the beginning of the file. You
would usually want each line to start with '#' to make it a comment,
and include a final terminating newline.
save_old (default: True):
If True and <filename> already exists, a copy of it will be saved to
.<filename>.old in the same directory before the new configuration is
written. The leading dot is added only if the filename doesn't
already start with a dot.
Errors are silently ignored if .<filename>.old cannot be written
(e.g. due to being a directory).
verbose (default: True):
If True and filename is None (automatically infer configuration
file), a message will be printed to stdout telling which file got
written. This is meant to reduce boilerplate in tools.
"""
if filename is None:
filename = standard_config_filename()
else:
verbose = False
if save_old:
_save_old(filename)
with self._open(filename, "w") as f:
f.write(header)
for node in self.node_iter(unique_syms=True):
item = node.item
if item.__class__ is Symbol:
f.write(item.config_string)
elif expr_value(node.dep) and \
((item is MENU and expr_value(node.visibility)) or
item is COMMENT):
f.write("\n#\n# {}\n#\n".format(node.prompt[0]))
if verbose:
print("Configuration written to '{}'".format(filename))
def write_min_config(self, filename,
header="# Generated by Kconfiglib (https://github.com/ulfalizer/Kconfiglib)\n"):
"""
Writes out a "minimal" configuration file, omitting symbols whose value
matches their default value. The format matches the one produced by
'make savedefconfig'.
The resulting configuration file is incomplete, but a complete
configuration can be derived from it by loading it. Minimal
configuration files can serve as a more manageable configuration format
compared to a "full" .config file, especially when configurations files
are merged or edited by hand.
filename:
Self-explanatory.
header (default: "# Generated by Kconfiglib (https://github.com/ulfalizer/Kconfiglib)\n"):
Text that will be inserted verbatim at the beginning of the file. You
would usually want each line to start with '#' to make it a comment,
and include a final terminating newline.
"""
with self._open(filename, "w") as f:
f.write(header)
for sym in self.unique_defined_syms:
# Skip symbols that cannot be changed. Only check
# non-choice symbols, as selects don't affect choice
# symbols.
if not sym.choice and \
sym.visibility <= expr_value(sym.rev_dep):
continue
# Skip symbols whose value matches their default
if sym.str_value == sym._str_default():
continue
# Skip symbols that would be selected by default in a
# choice, unless the choice is optional or the symbol type
# isn't bool (it might be possible to set the choice mode
# to n or the symbol to m in those cases).
if sym.choice and \
not sym.choice.is_optional and \
sym.choice._get_selection_from_defaults() is sym and \
sym.orig_type is BOOL and \
sym.tri_value == 2:
continue
f.write(sym.config_string)
def sync_deps(self, path):
"""
Creates or updates a directory structure that can be used to avoid
doing a full rebuild whenever the configuration is changed, mirroring
include/config/ in the kernel.
This function is intended to be called during each build, before
compiling source files that depend on configuration symbols.
path:
Path to directory
sync_deps(path) does the following:
1. If the directory <path> does not exist, it is created.
2. If <path>/auto.conf exists, old symbol values are loaded from it,
which are then compared against the current symbol values. If a
symbol has changed value (would generate different output in
autoconf.h compared to before), the change is signaled by
touch'ing a file corresponding to the symbol.
The first time sync_deps() is run on a directory, <path>/auto.conf
won't exist, and no old symbol values will be available. This
logically has the same effect as updating the entire
configuration.
The path to a symbol's file is calculated from the symbol's name
by replacing all '_' with '/' and appending '.h'. For example, the
symbol FOO_BAR_BAZ gets the file <path>/foo/bar/baz.h, and FOO
gets the file <path>/foo.h.
This scheme matches the C tools. The point is to avoid having a
single directory with a huge number of files, which the underlying
filesystem might not handle well.
3. A new auto.conf with the current symbol values is written, to keep
track of them for the next build.
The last piece of the puzzle is knowing what symbols each source file
depends on. Knowing that, dependencies can be added from source files
to the files corresponding to the symbols they depends on. The source
file will then get recompiled (only) when the symbol value changes
(provided sync_deps() is run first during each build).
The tool in the kernel that extracts symbol dependencies from source
files is scripts/basic/fixdep.c. Missing symbol files also correspond
to "not changed", which fixdep deals with by using the $(wildcard) Make
function when adding symbol prerequisites to source files.
In case you need a different scheme for your project, the sync_deps()
implementation can be used as a template.
"""
if not os.path.exists(path):
os.mkdir(path, 0o755)
# This setup makes sure that at least the current working directory
# gets reset if things fail
prev_dir = os.getcwd()
try:
# cd'ing into the symbol file directory simplifies
# _sync_deps() and saves some work
os.chdir(path)
self._sync_deps()
finally:
os.chdir(prev_dir)
def _sync_deps(self):
# Load old values from auto.conf, if any
self._load_old_vals()
for sym in self.unique_defined_syms:
# Note: _write_to_conf is determined when the value is
# calculated. This is a hidden function call due to
# property magic.
val = sym.str_value
# Note: n tristate values do not get written to auto.conf and
# autoconf.h, making a missing symbol logically equivalent to n
if sym._write_to_conf:
if sym._old_val is None and \
sym.orig_type in _BOOL_TRISTATE and \
val == "n":
# No old value (the symbol was missing or n), new value n.
# No change.
continue
if val == sym._old_val:
# New value matches old. No change.
continue
elif sym._old_val is None:
# The symbol wouldn't appear in autoconf.h (because
# _write_to_conf is false), and it wouldn't have appeared in
# autoconf.h previously either (because it didn't appear in
# auto.conf). No change.
continue
# 'sym' has a new value. Flag it.
_touch_dep_file(sym.name)
# Remember the current values as the "new old" values.
#
# This call could go anywhere after the call to _load_old_vals(), but
# putting it last means _sync_deps() can be safely rerun if it fails
# before this point.
self._write_old_vals()
def _write_old_vals(self):
# Helper for writing auto.conf. Basically just a simplified
# write_config() that doesn't write any comments (including
# '# CONFIG_FOO is not set' comments). The format matches the C
# implementation, though the ordering is arbitrary there (depends on
# the hash table implementation).
#
# A separate helper function is neater than complicating write_config()
# by passing a flag to it, plus we only need to look at symbols here.
with self._open("auto.conf", "w") as f:
for sym in self.unique_defined_syms:
if not (sym.orig_type in _BOOL_TRISTATE and not sym.tri_value):
f.write(sym.config_string)
def _load_old_vals(self):
# Loads old symbol values from auto.conf into a dedicated
# Symbol._old_val field. Mirrors load_config().
#
# The extra field could be avoided with some trickery involving dumping
# symbol values and restoring them later, but this is simpler and
# faster. The C tools also use a dedicated field for this purpose.
for sym in self.unique_defined_syms:
sym._old_val = None
if not os.path.exists("auto.conf"):
# No old values
return
with self._open("auto.conf", "r") as f:
for line in f:
match = self._set_match(line)
if not match:
# We only expect CONFIG_FOO=... (and possibly a header
# comment) in auto.conf
continue
name, val = match.groups()
if name in self.syms:
sym = self.syms[name]
if sym.orig_type is STRING:
match = _conf_string_match(val)
if not match:
continue
val = unescape(match.group(1))
self.syms[name]._old_val = val
else:
# Flag that the symbol no longer exists, in
# case something still depends on it
_touch_dep_file(name)
def node_iter(self, unique_syms=False):
"""
Returns a generator for iterating through all MenuNode's in the Kconfig
tree. The iteration is done in Kconfig definition order (each node is
visited before its children, and the children of a node are visited
before the next node).
The Kconfig.top_node menu node is skipped. It contains an implicit menu
that holds the top-level items.
As an example, the following code will produce a list equal to
Kconfig.defined_syms:
defined_syms = [node.item for node in kconf.node_iter()
if isinstance(node.item, Symbol)]
unique_syms (default: False):
If True, only the first MenuNode will be included for symbols defined
in multiple locations.
Using kconf.node_iter(True) in the example above would give a list
equal to unique_defined_syms.
"""
if unique_syms:
for sym in self.unique_defined_syms:
sym._visited = False
node = self.top_node
while 1:
# Jump to the next node with an iterative tree walk
if node.list:
node = node.list
elif node.next:
node = node.next
else:
while node.parent:
node = node.parent
if node.next:
node = node.next
break
else:
# No more nodes
return
if unique_syms and node.item.__class__ is Symbol:
if node.item._visited:
continue
node.item._visited = True
yield node
def eval_string(self, s):
"""
Returns the tristate value of the expression 's', represented as 0, 1,
and 2 for n, m, and y, respectively. Raises KconfigError if syntax
errors are detected in 's'. Warns if undefined symbols are referenced.
As an example, if FOO and BAR are tristate symbols at least one of
which has the value y, then config.eval_string("y && (FOO || BAR)")
returns 2 (y).
To get the string value of non-bool/tristate symbols, use
Symbol.str_value. eval_string() always returns a tristate value, and
all non-bool/tristate symbols have the tristate value 0 (n).
The expression parsing is consistent with how parsing works for
conditional ('if ...') expressions in the configuration, and matches
the C implementation. m is rewritten to 'm && MODULES', so
eval_string("m") will return 0 (n) unless modules are enabled.
"""
# The parser is optimized to be fast when parsing Kconfig files (where
# an expression can never appear at the beginning of a line). We have
# to monkey-patch things a bit here to reuse it.
self._filename = None
# Don't include the "if " from below to avoid giving confusing error
# messages
self._line = s
self._tokens = self._tokenize("if " + s)
self._tokens_i = 1 # Skip the 'if' token
return expr_value(self._expect_expr_and_eol())
def unset_values(self):
"""
Resets the user values of all symbols, as if Kconfig.load_config() or
Symbol.set_value() had never been called.
"""
self._warn_for_no_prompt = False
try:
# set_value() already rejects undefined symbols, and they don't
# need to be invalidated (because their value never changes), so we
# can just iterate over defined symbols
for sym in self.unique_defined_syms:
sym.unset_value()
for choice in self.unique_choices:
choice.unset_value()
finally:
self._warn_for_no_prompt = True
def enable_warnings(self):
"""
See Kconfig.__init__().
"""
self._warnings_enabled = True
def disable_warnings(self):
"""
See Kconfig.__init__().
"""
self._warnings_enabled = False
def enable_stderr_warnings(self):
"""
See Kconfig.__init__().
"""
self._warn_to_stderr = True
def disable_stderr_warnings(self):
"""
See Kconfig.__init__().
"""
self._warn_to_stderr = False
def enable_undef_warnings(self):
"""
Enables warnings for assignments to undefined symbols. Disabled by
default unless the KCONFIG_WARN_UNDEF_ASSIGN environment variable was
set to 'y' when the Kconfig instance was created.
"""
self._warn_for_undef_assign = True
def disable_undef_warnings(self):
"""
See enable_undef_assign().
"""
self._warn_for_undef_assign = False
def enable_override_warnings(self):
"""
Enables warnings for duplicated assignments in .config files that set
different values (e.g. CONFIG_FOO=m followed by CONFIG_FOO=y, where
the last value set is used).
These warnings are enabled by default. Disabling them might be helpful
in certain cases when merging configurations.
"""
self._warn_for_override = True
def disable_override_warnings(self):
"""
See enable_override_warnings().
"""
self._warn_for_override = False
def enable_redun_warnings(self):
"""
Enables warnings for duplicated assignments in .config files that all
set the same value.
These warnings are enabled by default. Disabling them might be helpful
in certain cases when merging configurations.
"""
self._warn_for_redun_assign = True
def disable_redun_warnings(self):
"""
See enable_redun_warnings().
"""
self._warn_for_redun_assign = False
def __repr__(self):
"""
Returns a string with information about the Kconfig object when it is
evaluated on e.g. the interactive Python prompt.
"""
return "<{}>".format(", ".join((
"configuration with {} symbols".format(len(self.syms)),
'main menu prompt "{}"'.format(self.mainmenu_text),
"srctree is current directory" if not self.srctree else
'srctree "{}"'.format(self.srctree),
'config symbol prefix "{}"'.format(self.config_prefix),
"warnings " +
("enabled" if self._warnings_enabled else "disabled"),
"printing of warnings to stderr " +
("enabled" if self._warn_to_stderr else "disabled"),
"undef. symbol assignment warnings " +
("enabled" if self._warn_for_undef_assign else "disabled"),
"redundant symbol assignment warnings " +
("enabled" if self._warn_for_redun_assign else "disabled")
)))
#
# Private methods
#
#
# File reading
#
def _open_config(self, filename):
# Opens a .config file. First tries to open 'filename', then
# '$srctree/filename' if $srctree was set when the configuration was
# loaded.
try:
return self._open(filename, "r")
except IOError as e:
# This will try opening the same file twice if $srctree is unset,
# but it's not a big deal
try:
return self._open(os.path.join(self.srctree, filename), "r")
except IOError as e2:
# This is needed for Python 3, because e2 is deleted after
# the try block:
#
# https://docs.python.org/3/reference/compound_stmts.html#the-try-statement
e = e2
raise _KconfigIOError(
e, "Could not open '{}' ({}: {}). Check that the $srctree "
"environment variable ({}) is set correctly."
.format(filename, errno.errorcode[e.errno], e.strerror,
"set to '{}'".format(self.srctree) if self.srctree
else "unset or blank"))
def _enter_file(self, full_filename, rel_filename):
# Jumps to the beginning of a sourced Kconfig file, saving the previous
# position and file object.
#
# full_filename:
# Actual path to the file.
#
# rel_filename:
# File path with $srctree prefix stripped, stored in e.g.
# self._filename (which makes it indirectly show up in
# MenuNode.filename). Equals full_filename for absolute paths.
self.kconfig_filenames.append(rel_filename)
# The parent Kconfig files are represented as a list of
# (<include path>, <Python 'file' object for Kconfig file>) tuples.
#
# <include path> is immutable and holds a *tuple* of
# (<filename>, <linenr>) tuples, giving the locations of the 'source'
# statements in the parent Kconfig files. The current include path is
# also available in Kconfig._include_path.
#
# The point of this redundant setup is to allow Kconfig._include_path
# to be assigned directly to MenuNode.include_path without having to
# copy it, sharing it wherever possible.
# Save include path and 'file' object (via its 'readline' function)
# before entering the file
self._filestack.append((self._include_path, self._readline))
# _include_path is a tuple, so this rebinds the variable instead of
# doing in-place modification
self._include_path += ((self._filename, self._linenr),)
# Check for recursive 'source'
for name, _ in self._include_path:
if name == rel_filename:
raise KconfigError(
"\n{}:{}: recursive 'source' of '{}' detected. Check that "
"environment variables are set correctly.\n"
"Include path:\n{}"
.format(self._filename, self._linenr, rel_filename,
"\n".join("{}:{}".format(name, linenr)
for name, linenr in self._include_path)))
# Note: We already know that the file exists
try:
self._readline = self._open(full_filename, "r").readline
except IOError as e:
raise _KconfigIOError(
e, "{}:{}: Could not open '{}' ({}: {})"
.format(self._filename, self._linenr, full_filename,
errno.errorcode[e.errno], e.strerror))
self._filename = rel_filename
self._linenr = 0
def _leave_file(self):
# Returns from a Kconfig file to the file that sourced it. See
# _enter_file().
# __self__ fetches the 'file' object for the method
self._readline.__self__.close()
# Restore location from parent Kconfig file
self._filename, self._linenr = self._include_path[-1]
# Restore include path and 'file' object
self._include_path, self._readline = self._filestack.pop()
def _next_line(self):
# Fetches and tokenizes the next line from the current Kconfig file.
# Returns False at EOF and True otherwise.
# We might already have tokens from parsing a line and discovering that
# it's part of a different construct
if self._reuse_tokens:
self._reuse_tokens = False
# self._tokens_i is known to be 1 here, because _parse_properties()
# leaves it like that when it can't recognize a line (or parses
# a help text)
return True
# Note: readline() returns '' over and over at EOF, which we rely on
# for help texts at the end of files (see _line_after_help())
line = self._readline()
if not line:
return False
self._linenr += 1
# Handle line joining
while line.endswith("\\\n"):
line = line[:-2] + self._readline()
self._linenr += 1
self._line = line # Used for error reporting
self._tokens = self._tokenize(line)
# Initialize to 1 instead of 0 to factor out code from _parse_block()
# and _parse_properties(). They immediately fetch self._tokens[0].
self._tokens_i = 1
return True
def _line_after_help(self, line):
# Tokenizes a line after a help text. This case is special in that the
# line has already been fetched (to discover that it isn't part of the
# help text).
#
# An earlier version used a _saved_line variable instead that was
# checked in _next_line(). This special-casing gets rid of it and makes
# _reuse_tokens alone sufficient to handle unget.
# Handle line joining
while line.endswith("\\\n"):
line = line[:-2] + self._readline()
self._linenr += 1
self._line = line
self._tokens = self._tokenize(line)
self._reuse_tokens = True
#
# Tokenization
#
def _lookup_sym(self, name):
# Fetches the symbol 'name' from the symbol table, creating and
# registering it if it does not exist. If '_parsing_kconfigs' is False,
# it means we're in eval_string(), and new symbols won't be registered.
if name in self.syms:
return self.syms[name]
sym = Symbol()
sym.kconfig = self
sym.name = name
sym.is_constant = False
sym.rev_dep = sym.weak_rev_dep = sym.direct_dep = self.n
if self._parsing_kconfigs:
self.syms[name] = sym
else:
self._warn("no symbol {} in configuration".format(name))
return sym
def _lookup_const_sym(self, name):
# Like _lookup_sym(), for constant (quoted) symbols
if name in self.const_syms:
return self.const_syms[name]
sym = Symbol()
sym.kconfig = self
sym.name = name
sym.is_constant = True
sym.rev_dep = sym.weak_rev_dep = sym.direct_dep = self.n
if self._parsing_kconfigs:
self.const_syms[name] = sym
return sym
def _tokenize(self, s):
# Parses 's', returning a None-terminated list of tokens. Registers any
# new symbols encountered with _lookup(_const)_sym().
#
# Tries to be reasonably speedy by processing chunks of text via
# regexes and string operations where possible. This is the biggest
# hotspot during parsing.
#
# Note: It might be possible to rewrite this to 'yield' tokens instead,
# working across multiple lines. The 'option env' lookback thing below
# complicates things though.
# Initial token on the line
match = _command_match(s)
if not match:
if s.isspace() or s.lstrip().startswith("#"):
return (None,)
self._parse_error("unknown token at start of line")
# Tricky implementation detail: While parsing a token, 'token' refers
# to the previous token. See _STRING_LEX for why this is needed.
token = _get_keyword(match.group(1))
if not token:
# Backwards compatibility with old versions of the C tools, which
# (accidentally) accepted stuff like "--help--" and "-help---".
# This was fixed in the C tools by commit c2264564 ("kconfig: warn
# of unhandled characters in Kconfig commands"), committed in July
# 2015, but it seems people still run Kconfiglib on older kernels.
if s.strip(" \t\n-") == "help":
return (_T_HELP, None)
# If the first token is not a keyword (and not a weird help token),
# we have a preprocessor variable assignment (or a bare macro on a
# line)
self._parse_assignment(s)
return (None,)
tokens = [token]
# The current index in the string being tokenized
i = match.end()
# Main tokenization loop (for tokens past the first one)
while i < len(s):
# Test for an identifier/keyword first. This is the most common
# case.
match = _id_keyword_match(s, i)
if match:
# We have an identifier or keyword
# Check what it is. lookup_sym() will take care of allocating
# new symbols for us the first time we see them. Note that
# 'token' still refers to the previous token.
name = match.group(1)
keyword = _get_keyword(name)
if keyword:
# It's a keyword
token = keyword
# Jump past it
i = match.end()
elif token not in _STRING_LEX:
# It's a non-const symbol, except we translate n, m, and y
# into the corresponding constant symbols, like the C
# implementation
if "$" in name:
# Macro expansion within symbol name
name, s, i = self._expand_name(s, i)
else:
i = match.end()
token = self.const_syms[name] \
if name in ("y", "m", "n") else \
self._lookup_sym(name)
else:
# It's a case of missing quotes. For example, the
# following is accepted:
#
# menu unquoted_title
#
# config A
# tristate unquoted_prompt
#
# endmenu
token = name
i = match.end()
else:
# Neither a keyword nor a non-const symbol
# We always strip whitespace after tokens, so it is safe to
# assume that s[i] is the start of a token here.
c = s[i]
if c in "\"'":
if "$" not in s and "\\" not in s:
# Fast path for lines without $ and \. Find the
# matching quote.
end_i = s.find(c, i + 1) + 1
if not end_i:
self._parse_error("unterminated string")
val = s[i + 1:end_i - 1]
i = end_i
else:
# Slow path
s, end_i = self._expand_str(s, i)
# os.path.expandvars() and the $UNAME_RELEASE replace()
# is a backwards compatibility hack, which should be
# reasonably safe as expandvars() leaves references to
# undefined env. vars. as is.
#
# The preprocessor functionality changed how
# environment variables are referenced, to $(FOO).
val = os.path.expandvars(
s[i + 1:end_i - 1].replace("$UNAME_RELEASE",
platform.uname()[2]))
i = end_i
# This is the only place where we don't survive with a
# single token of lookback: 'option env="FOO"' does not
# refer to a constant symbol named "FOO".
token = \
val if token in _STRING_LEX or tokens[0] is _T_OPTION \
else self._lookup_const_sym(val)
elif s.startswith("&&", i):
token = _T_AND
i += 2
elif s.startswith("||", i):
token = _T_OR
i += 2
elif c == "=":
token = _T_EQUAL
i += 1
elif s.startswith("!=", i):
token = _T_UNEQUAL
i += 2
elif c == "!":
token = _T_NOT
i += 1
elif c == "(":
token = _T_OPEN_PAREN
i += 1
elif c == ")":
token = _T_CLOSE_PAREN
i += 1
elif c == "#":
break
# Very rare
elif s.startswith("<=", i):
token = _T_LESS_EQUAL
i += 2
elif c == "<":
token = _T_LESS
i += 1
elif s.startswith(">=", i):
token = _T_GREATER_EQUAL
i += 2
elif c == ">":
token = _T_GREATER
i += 1
else:
self._parse_error("unknown tokens in line")
# Skip trailing whitespace
while i < len(s) and s[i].isspace():
i += 1
# Add the token
tokens.append(token)
# None-terminating the token list makes token fetching simpler/faster
tokens.append(None)
return tokens
# Helpers for syntax checking and token fetching. See the
# 'Intro to expressions' section for what a constant symbol is.
#
# More of these could be added, but the single-use cases are inlined as an
# optimization.
def _expect_sym(self):
token = self._tokens[self._tokens_i]
self._tokens_i += 1
if token.__class__ is not Symbol:
self._parse_error("expected symbol")
return token
def _expect_nonconst_sym(self):
# Used for 'select' and 'imply' only. We know the token indices.
token = self._tokens[1]
self._tokens_i = 2
if token.__class__ is not Symbol or token.is_constant:
self._parse_error("expected nonconstant symbol")
return token
def _expect_str_and_eol(self):
token = self._tokens[self._tokens_i]
self._tokens_i += 1
if token.__class__ is not str:
self._parse_error("expected string")
if self._tokens[self._tokens_i] is not None:
self._trailing_tokens_error()
return token
def _expect_expr_and_eol(self):
expr = self._parse_expr(True)
if self._tokens[self._tokens_i] is not None:
self._trailing_tokens_error()
return expr
def _check_token(self, token):
# If the next token is 'token', removes it and returns True
if self._tokens[self._tokens_i] is token:
self._tokens_i += 1
return True
return False
#
# Preprocessor logic
#
def _parse_assignment(self, s):
# Parses a preprocessor variable assignment, registering the variable
# if it doesn't already exist. Also takes care of bare macros on lines
# (which are allowed, and can be useful for their side effects).
# Expand any macros in the left-hand side of the assignment (the
# variable name)
s = s.lstrip()
i = 0
while 1:
i = _assignment_lhs_fragment_match(s, i).end()
if s.startswith("$(", i):
s, i = self._expand_macro(s, i, ())
else:
break
if s.isspace():
# We also accept a bare macro on a line (e.g.
# $(warning-if,$(foo),ops)), provided it expands to a blank string
return
# Assigned variable
name = s[:i]
# Extract assignment operator (=, :=, or +=) and value
rhs_match = _assignment_rhs_match(s, i)
if not rhs_match:
self._parse_error("syntax error")
op, val = rhs_match.groups()
if name in self.variables:
# Already seen variable
var = self.variables[name]
else:
# New variable
var = Variable()
var.kconfig = self
var.name = name
var._n_expansions = 0
self.variables[name] = var
# += acts like = on undefined variables (defines a recursive
# variable)
if op == "+=":
op = "="
if op == "=":
var.is_recursive = True
var.value = val
elif op == ":=":
var.is_recursive = False
var.value = self._expand_whole(val, ())
else: # op == "+="
# += does immediate expansion if the variable was last set
# with :=
var.value += " " + (val if var.is_recursive else
self._expand_whole(val, ()))
def _expand_whole(self, s, args):
# Expands preprocessor macros in all of 's'. Used whenever we don't
# have to worry about delimiters. See _expand_macro() re. the 'args'
# parameter.
#
# Returns the expanded string.
i = 0
while 1:
i = s.find("$(", i)
if i == -1:
break
s, i = self._expand_macro(s, i, args)
return s
def _expand_name(self, s, i):
# Expands a symbol name starting at index 'i' in 's'.
#
# Returns the expanded name, the expanded 's' (including the part
# before the name), and the index of the first character in the next
# token after the name.
s, end_i = self._expand_name_iter(s, i)
name = s[i:end_i]
# isspace() is False for empty strings
if not name.strip():
# Avoid creating a Kconfig symbol with a blank name. It's almost
# guaranteed to be an error.
self._parse_error("macro expanded to blank string")
# Skip trailing whitespace
while end_i < len(s) and s[end_i].isspace():
end_i += 1
return name, s, end_i
def _expand_name_iter(self, s, i):
# Expands a symbol name starting at index 'i' in 's'.
#
# Returns the expanded 's' (including the part before the name) and the
# index of the first character after the expanded name in 's'.
while 1:
match = _name_special_search(s, i)
if match.group() == "$(":
s, i = self._expand_macro(s, match.start(), ())
else:
return (s, match.start())
def _expand_str(self, s, i):
# Expands a quoted string starting at index 'i' in 's'. Handles both
# backslash escapes and macro expansion.
#
# Returns the expanded 's' (including the part before the string) and
# the index of the first character after the expanded string in 's'.
quote = s[i]
i += 1 # Skip over initial "/'
while 1:
match = _string_special_search(s, i)
if not match:
self._parse_error("unterminated string")
if match.group() == quote:
# Found the end of the string
return (s, match.end())
elif match.group() == "\\":
# Replace '\x' with 'x'. 'i' ends up pointing to the character
# after 'x', which allows macros to be canceled with '\$(foo)'.
i = match.end()
s = s[:match.start()] + s[i:]
elif match.group() == "$(":
# A macro call within the string
s, i = self._expand_macro(s, match.start(), ())
else:
# A ' quote within " quotes or vice versa
i += 1
def _expand_macro(self, s, i, args):
# Expands a macro starting at index 'i' in 's'. If this macro resulted
# from the expansion of another macro, 'args' holds the arguments
# passed to that macro.
#
# Returns the expanded 's' (including the part before the macro) and
# the index of the first character after the expanded macro in 's'.
start = i
i += 2 # Skip over "$("
# Start of current macro argument
arg_start = i
# Arguments of this macro call
new_args = []
while 1:
match = _macro_special_search(s, i)
if not match:
self._parse_error("missing end parenthesis in macro expansion")
if match.group() == ")":
# Found the end of the macro
new_args.append(s[arg_start:match.start()])
prefix = s[:start]
# $(1) is replaced by the first argument to the function, etc.,
# provided at least that many arguments were passed
try:
# Does the macro look like an integer, with a corresponding
# argument? If so, expand it to the value of the argument.
prefix += args[int(new_args[0])]
except (ValueError, IndexError):
# Regular variables are just functions without arguments,
# and also go through the function value path
prefix += self._fn_val(new_args)
return (prefix + s[match.end():],
len(prefix))
elif match.group() == ",":
# Found the end of a macro argument
new_args.append(s[arg_start:match.start()])
arg_start = i = match.end()
else: # match.group() == "$("
# A nested macro call within the macro
s, i = self._expand_macro(s, match.start(), args)
def _fn_val(self, args):
# Returns the result of calling the function args[0] with the arguments
# args[1..len(args)-1]. Plain variables are treated as functions
# without arguments.
fn = args[0]
if fn in self.variables:
var = self.variables[fn]
if len(args) == 1:
# Plain variable
if var._n_expansions:
self._parse_error("Preprocessor variable {} recursively "
"references itself".format(var.name))
elif var._n_expansions > 100:
# Allow functions to call themselves, but guess that functions
# that are overly recursive are stuck
self._parse_error("Preprocessor function {} seems stuck "
"in infinite recursion".format(var.name))
var._n_expansions += 1
res = self._expand_whole(self.variables[fn].value, args)
var._n_expansions -= 1
return res
if fn in self._functions:
# Built-in or user-defined function
py_fn, min_arg, max_arg = self._functions[fn]
if len(args) - 1 < min_arg or \
(max_arg is not None and len(args) - 1 > max_arg):
if min_arg == max_arg:
expected_args = min_arg
elif max_arg is None:
expected_args = "{} or more".format(min_arg)
else:
expected_args = "{}-{}".format(min_arg, max_arg)
raise KconfigError("{}:{}: bad number of arguments in call "
"to {}, expected {}, got {}"
.format(self._filename, self._linenr, fn,
expected_args, len(args) - 1))
return py_fn(self, *args)
# Environment variables are tried last
if fn in os.environ:
self.env_vars.add(fn)
return os.environ[fn]
return ""
#
# Parsing
#
def _make_and(self, e1, e2):
# Constructs an AND (&&) expression. Performs trivial simplification.
if e1 is self.y:
return e2
if e2 is self.y:
return e1
if e1 is self.n or e2 is self.n:
return self.n
return (AND, e1, e2)
def _make_or(self, e1, e2):
# Constructs an OR (||) expression. Performs trivial simplification.
if e1 is self.n:
return e2
if e2 is self.n:
return e1
if e1 is self.y or e2 is self.y:
return self.y
return (OR, e1, e2)
def _parse_block(self, end_token, parent, prev):
# Parses a block, which is the contents of either a file or an if,
# menu, or choice statement.
#
# end_token:
# The token that ends the block, e.g. _T_ENDIF ("endif") for ifs.
# None for files.
#
# parent:
# The parent menu node, corresponding to a menu, Choice, or 'if'.
# 'if's are flattened after parsing.
#
# prev:
# The previous menu node. New nodes will be added after this one (by
# modifying their 'next' pointer).
#
# 'prev' is reused to parse a list of child menu nodes (for a menu or
# Choice): After parsing the children, the 'next' pointer is assigned
# to the 'list' pointer to "tilt up" the children above the node.
#
# Returns the final menu node in the block (or 'prev' if the block is
# empty). This allows chaining.
while self._next_line():
t0 = self._tokens[0]
if t0 is _T_CONFIG or t0 is _T_MENUCONFIG:
# The tokenizer allocates Symbol objects for us
sym = self._tokens[1]
if sym.__class__ is not Symbol or sym.is_constant:
self._parse_error("missing or bad symbol name")
if self._tokens[2] is not None:
self._trailing_tokens_error()
self.defined_syms.append(sym)
node = MenuNode()
node.kconfig = self
node.item = sym
node.is_menuconfig = (t0 is _T_MENUCONFIG)
node.prompt = node.help = node.list = None
node.parent = parent
node.filename = self._filename
node.linenr = self._linenr
node.include_path = self._include_path
sym.nodes.append(node)
self._parse_properties(node)
if node.is_menuconfig and not node.prompt:
self._warn("the menuconfig symbol {} has no prompt"
.format(_name_and_loc(sym)))
# Equivalent to
#
# prev.next = node
# prev = node
#
# due to tricky Python semantics. The order matters.
prev.next = prev = node
elif t0 is None:
# Blank line
continue
elif t0 in _SOURCE_TOKENS:
pattern = self._expect_str_and_eol()
# Check if the pattern is absolute and avoid stripping srctree
# from it below in that case. We must do the check before
# join()'ing, as srctree might be an absolute path.
isabs = os.path.isabs(pattern)
if t0 in _REL_SOURCE_TOKENS:
# Relative source
pattern = os.path.join(os.path.dirname(self._filename),
pattern)
# Sort the glob results to ensure a consistent ordering of
# Kconfig symbols, which indirectly ensures a consistent
# ordering in e.g. .config files
filenames = \
sorted(glob.iglob(os.path.join(self.srctree, pattern)))
if not filenames and t0 in _OBL_SOURCE_TOKENS:
raise KconfigError(
"{}:{}: '{}' not found (in '{}'). Check that "
"environment variables are set correctly (e.g. "
"$srctree, which is {}). Also note that unset "
"environment variables expand to the empty string."
.format(self._filename, self._linenr, pattern,
self._line.strip(),
"set to '{}'".format(self.srctree)
if self.srctree else "unset or blank"))
for filename in filenames:
self._enter_file(
filename,
# Unless an absolute path is passed to *source, strip
# the $srctree prefix from the filename. That way it
# appears without a $srctree prefix in
# MenuNode.filename, which is nice e.g. when generating
# documentation.
filename if isabs else
os.path.relpath(filename, self.srctree))
prev = self._parse_block(None, parent, prev)
self._leave_file()
elif t0 is end_token:
# We have reached the end of the block. Terminate the final
# node and return it.
if self._tokens[1] is not None:
self._trailing_tokens_error()
prev.next = None
return prev
elif t0 is _T_IF:
node = MenuNode()
node.item = node.prompt = None
node.parent = parent
node.dep = self._expect_expr_and_eol()
self._parse_block(_T_ENDIF, node, node)
node.list = node.next
prev.next = prev = node
elif t0 is _T_MENU:
node = MenuNode()
node.kconfig = self
node.item = t0 # _T_MENU == MENU
node.is_menuconfig = True
node.prompt = (self._expect_str_and_eol(), self.y)
node.visibility = self.y
node.parent = parent
node.filename = self._filename
node.linenr = self._linenr
node.include_path = self._include_path
self.menus.append(node)
self._parse_properties(node)
self._parse_block(_T_ENDMENU, node, node)
node.list = node.next
prev.next = prev = node
elif t0 is _T_COMMENT:
node = MenuNode()
node.kconfig = self
node.item = t0 # _T_COMMENT == COMMENT
node.is_menuconfig = False
node.prompt = (self._expect_str_and_eol(), self.y)
node.list = None
node.parent = parent
node.filename = self._filename
node.linenr = self._linenr
node.include_path = self._include_path
self.comments.append(node)
self._parse_properties(node)
prev.next = prev = node
elif t0 is _T_CHOICE:
if self._tokens[1] is None:
choice = Choice()
choice.direct_dep = self.n
else:
# Named choice
name = self._expect_str_and_eol()
choice = self.named_choices.get(name)
if not choice:
choice = Choice()
choice.name = name
choice.direct_dep = self.n
self.named_choices[name] = choice
self.choices.append(choice)
choice.kconfig = self
node = MenuNode()
node.kconfig = self
node.item = choice
node.is_menuconfig = True
node.prompt = node.help = None
node.parent = parent
node.filename = self._filename
node.linenr = self._linenr
node.include_path = self._include_path
choice.nodes.append(node)
self._parse_properties(node)
self._parse_block(_T_ENDCHOICE, node, node)
node.list = node.next
prev.next = prev = node
elif t0 is _T_MAINMENU:
self.top_node.prompt = (self._expect_str_and_eol(), self.y)
self.top_node.filename = self._filename
self.top_node.linenr = self._linenr
else:
# A valid endchoice/endif/endmenu is caught by the 'end_token'
# check above
self._parse_error(
"no corresponding 'choice'" if t0 is _T_ENDCHOICE else
"no corresponding 'if'" if t0 is _T_ENDIF else
"no corresponding 'menu'" if t0 is _T_ENDMENU else
"unrecognized construct")
# End of file reached. Terminate the final node and return it.
if end_token:
raise KconfigError(
"expected '{}' at end of '{}'"
.format("endchoice" if end_token is _T_ENDCHOICE else
"endif" if end_token is _T_ENDIF else
"endmenu",
self._filename))
prev.next = None
return prev
def _parse_cond(self):
# Parses an optional 'if <expr>' construct and returns the parsed
# <expr>, or self.y if the next token is not _T_IF
expr = self._parse_expr(True) if self._check_token(_T_IF) else self.y
if self._tokens[self._tokens_i] is not None:
self._trailing_tokens_error()
return expr
def _parse_properties(self, node):
# Parses and adds properties to the MenuNode 'node' (type, 'prompt',
# 'default's, etc.) Properties are later copied up to symbols and
# choices in a separate pass after parsing, in e.g.
# _add_props_to_sym().
#
# An older version of this code added properties directly to symbols
# and choices instead of to their menu nodes (and handled dependency
# propagation simultaneously), but that loses information on where a
# property is added when a symbol or choice is defined in multiple
# locations. Some Kconfig configuration systems rely heavily on such
# symbols, and better docs can be generated by keeping track of where
# properties are added.
#
# node:
# The menu node we're parsing properties on
# Dependencies from 'depends on'. Will get propagated to the properties
# below.
node.dep = self.y
while self._next_line():
t0 = self._tokens[0]
if t0 in _TYPE_TOKENS:
self._set_type(node, t0)
if self._tokens[1] is not None:
self._parse_prompt(node)
elif t0 is _T_DEPENDS:
if not self._check_token(_T_ON):
self._parse_error("expected 'on' after 'depends'")
node.dep = self._make_and(node.dep,
self._expect_expr_and_eol())
elif t0 is _T_HELP:
self._parse_help(node)
elif t0 is _T_SELECT:
if node.item.__class__ is not Symbol:
self._parse_error("only symbols can select")
node.selects.append((self._expect_nonconst_sym(),
self._parse_cond()))
elif t0 is None:
# Blank line
continue
elif t0 is _T_DEFAULT:
node.defaults.append((self._parse_expr(False),
self._parse_cond()))
elif t0 in _DEF_TYPE_TOKENS:
self._set_type(node, t0)
node.defaults.append((self._parse_expr(False),
self._parse_cond()))
elif t0 is _T_PROMPT:
self._parse_prompt(node)
elif t0 is _T_RANGE:
node.ranges.append((self._expect_sym(), self._expect_sym(),
self._parse_cond()))
elif t0 is _T_IMPLY:
if node.item.__class__ is not Symbol:
self._parse_error("only symbols can imply")
node.implies.append((self._expect_nonconst_sym(),
self._parse_cond()))
elif t0 is _T_VISIBLE:
if not self._check_token(_T_IF):
self._parse_error("expected 'if' after 'visible'")
node.visibility = self._make_and(node.visibility,
self._expect_expr_and_eol())
elif t0 is _T_OPTION:
if self._check_token(_T_ENV):
if not self._check_token(_T_EQUAL):
self._parse_error("expected '=' after 'env'")
env_var = self._expect_str_and_eol()
node.item.env_var = env_var
if env_var in os.environ:
node.defaults.append(
(self._lookup_const_sym(os.environ[env_var]),
self.y))
else:
self._warn("{1} has 'option env=\"{0}\"', "
"but the environment variable {0} is not "
"set".format(node.item.name, env_var),
self._filename, self._linenr)
if env_var != node.item.name:
self._warn("Kconfiglib expands environment variables "
"in strings directly, meaning you do not "
"need 'option env=...' \"bounce\" symbols. "
"For compatibility with the C tools, "
"rename {} to {} (so that the symbol name "
"matches the environment variable name)."
.format(node.item.name, env_var),
self._filename, self._linenr)
elif self._check_token(_T_DEFCONFIG_LIST):
if not self.defconfig_list:
self.defconfig_list = node.item
else:
self._warn("'option defconfig_list' set on multiple "
"symbols ({0} and {1}). Only {0} will be "
"used.".format(self.defconfig_list.name,
node.item.name),
self._filename, self._linenr)
elif self._check_token(_T_MODULES):
# To reduce warning spam, only warn if 'option modules' is
# set on some symbol that isn't MODULES, which should be
# safe. I haven't run into any projects that make use
# modules besides the kernel yet, and there it's likely to
# keep being called "MODULES".
if node.item is not self.modules:
self._warn("the 'modules' option is not supported. "
"Let me know if this is a problem for you, "
"as it wouldn't be that hard to implement. "
"Note that modules are supported -- "
"Kconfiglib just assumes the symbol name "
"MODULES, like older versions of the C "
"implementation did when 'option modules' "
"wasn't used.",
self._filename, self._linenr)
elif self._check_token(_T_ALLNOCONFIG_Y):
if node.item.__class__ is not Symbol:
self._parse_error("the 'allnoconfig_y' option is only "
"valid for symbols")
node.item.is_allnoconfig_y = True
else:
self._parse_error("unrecognized option")
elif t0 is _T_OPTIONAL:
if node.item.__class__ is not Choice:
self._parse_error('"optional" is only valid for choices')
node.item.is_optional = True
else:
# Reuse the tokens for the non-property line later
self._reuse_tokens = True
return
def _set_type(self, node, type_token):
new_type = _TOKEN_TO_TYPE[type_token]
# The 'is not UNKNOWN' comparison will usually fail, since single-def
# symbols/choices are more common
if node.item.orig_type is not UNKNOWN and \
node.item.orig_type is not new_type:
self._warn("{} defined with multiple types, {} will be used"
.format(_name_and_loc(node.item),
TYPE_TO_STR[new_type]))
node.item.orig_type = new_type
def _parse_prompt(self, node):
# 'prompt' properties override each other within a single definition of
# a symbol, but additional prompts can be added by defining the symbol
# multiple times
if node.prompt:
self._warn(_name_and_loc(node.item) +
" defined with multiple prompts in single location")
prompt = self._tokens[1]
self._tokens_i = 2
if prompt.__class__ is not str:
self._parse_error("expected prompt string")
if prompt != prompt.strip():
self._warn(_name_and_loc(node.item) +
" has leading or trailing whitespace in its prompt")
# This avoid issues for e.g. reStructuredText documentation, where
# '*prompt *' is invalid
prompt = prompt.strip()
node.prompt = (prompt, self._parse_cond())
def _parse_help(self, node):
if node.help is not None:
self._warn(_name_and_loc(node.item) + " defined with more than "
"one help text -- only the last one will be used")
# Micro-optimization. This code is pretty hot.
readline = self._readline
# Find first non-blank (not all-space) line and get its
# indentation
while 1:
line = readline()
self._linenr += 1
if not line:
self._empty_help(node, line)
return
if not line.isspace():
break
len_ = len # Micro-optimization
# Use a separate 'expline' variable here and below to avoid stomping on
# any tabs people might've put deliberately into the first line after
# the help text
expline = line.expandtabs()
indent = len_(expline) - len_(expline.lstrip())
if not indent:
self._empty_help(node, line)
return
# The help text goes on till the first non-blank line with less indent
# than the first line
# Add the first line
lines = [expline[indent:]]
add_line = lines.append # Micro-optimization
while 1:
line = readline()
if line.isspace():
# No need to preserve the exact whitespace in these
add_line("\n")
elif not line:
# End of file
break
else:
expline = line.expandtabs()
if len_(expline) - len_(expline.lstrip()) < indent:
break
add_line(expline[indent:])
self._linenr += len_(lines)
node.help = "".join(lines).rstrip()
if line:
self._line_after_help(line)
def _empty_help(self, node, line):
self._warn(_name_and_loc(node.item) +
" has 'help' but empty help text")
node.help = ""
if line:
self._line_after_help(line)
def _parse_expr(self, transform_m):
# Parses an expression from the tokens in Kconfig._tokens using a
# simple top-down approach. See the module docstring for the expression
# format.
#
# transform_m:
# True if m should be rewritten to m && MODULES. See the
# Kconfig.eval_string() documentation.
# Grammar:
#
# expr: and_expr ['||' expr]
# and_expr: factor ['&&' and_expr]
# factor: <symbol> ['='/'!='/'<'/... <symbol>]
# '!' factor
# '(' expr ')'
#
# It helps to think of the 'expr: and_expr' case as a single-operand OR
# (no ||), and of the 'and_expr: factor' case as a single-operand AND
# (no &&). Parsing code is always a bit tricky.
# Mind dump: parse_factor() and two nested loops for OR and AND would
# work as well. The straightforward implementation there gives a
# (op, (op, (op, A, B), C), D) parse for A op B op C op D. Representing
# expressions as (op, [list of operands]) instead goes nicely with that
# version, but is wasteful for short expressions and complicates
# expression evaluation and other code that works on expressions (more
# complicated code likely offsets any performance gain from less
# recursion too). If we also try to optimize the list representation by
# merging lists when possible (e.g. when ANDing two AND expressions),
# we end up allocating a ton of lists instead of reusing expressions,
# which is bad.
and_expr = self._parse_and_expr(transform_m)
# Return 'and_expr' directly if we have a "single-operand" OR.
# Otherwise, parse the expression on the right and make an OR node.
# This turns A || B || C || D into (OR, A, (OR, B, (OR, C, D))).
return and_expr \
if not self._check_token(_T_OR) else \
(OR, and_expr, self._parse_expr(transform_m))
def _parse_and_expr(self, transform_m):
factor = self._parse_factor(transform_m)
# Return 'factor' directly if we have a "single-operand" AND.
# Otherwise, parse the right operand and make an AND node. This turns
# A && B && C && D into (AND, A, (AND, B, (AND, C, D))).
return factor \
if not self._check_token(_T_AND) else \
(AND, factor, self._parse_and_expr(transform_m))
def _parse_factor(self, transform_m):
token = self._tokens[self._tokens_i]
self._tokens_i += 1
if token.__class__ is Symbol:
# Plain symbol or relation
if self._tokens[self._tokens_i] not in _RELATIONS:
# Plain symbol
# For conditional expressions ('depends on <expr>',
# '... if <expr>', etc.), m is rewritten to m && MODULES.
if transform_m and token is self.m:
return (AND, self.m, self.modules)
return token
# Relation
#
# _T_EQUAL, _T_UNEQUAL, etc., deliberately have the same values as
# EQUAL, UNEQUAL, etc., so we can just use the token directly
self._tokens_i += 1
return (self._tokens[self._tokens_i - 1], token,
self._expect_sym())
if token is _T_NOT:
# token == _T_NOT == NOT
return (token, self._parse_factor(transform_m))
if token is _T_OPEN_PAREN:
expr_parse = self._parse_expr(transform_m)
if self._check_token(_T_CLOSE_PAREN):
return expr_parse
self._parse_error("malformed expression")
#
# Caching and invalidation
#
def _build_dep(self):
# Populates the Symbol/Choice._dependents sets, which contain all other
# items (symbols and choices) that immediately depend on the item in
# the sense that changing the value of the item might affect the value
# of the dependent items. This is used for caching/invalidation.
#
# The calculated sets might be larger than necessary as we don't do any
# complex analysis of the expressions.
make_depend_on = _make_depend_on # Micro-optimization
# Only calculate _dependents for defined symbols. Constant and
# undefined symbols could theoretically be selected/implied, but it
# wouldn't change their value, so it's not a true dependency.
for sym in self.unique_defined_syms:
# Symbols depend on the following:
# The prompt conditions
for node in sym.nodes:
if node.prompt:
make_depend_on(sym, node.prompt[1])
# The default values and their conditions
for value, cond in sym.defaults:
make_depend_on(sym, value)
make_depend_on(sym, cond)
# The reverse and weak reverse dependencies
make_depend_on(sym, sym.rev_dep)
make_depend_on(sym, sym.weak_rev_dep)
# The ranges along with their conditions
for low, high, cond in sym.ranges:
make_depend_on(sym, low)
make_depend_on(sym, high)
make_depend_on(sym, cond)
# The direct dependencies. This is usually redundant, as the direct
# dependencies get propagated to properties, but it's needed to get
# invalidation solid for 'imply', which only checks the direct
# dependencies (even if there are no properties to propagate it
# to).
make_depend_on(sym, sym.direct_dep)
# In addition to the above, choice symbols depend on the choice
# they're in, but that's handled automatically since the Choice is
# propagated to the conditions of the properties before
# _build_dep() runs.
for choice in self.unique_choices:
# Choices depend on the following:
# The prompt conditions
for node in choice.nodes:
if node.prompt:
make_depend_on(choice, node.prompt[1])
# The default symbol conditions
for _, cond in choice.defaults:
make_depend_on(choice, cond)
def _add_choice_deps(self):
# Choices also depend on the choice symbols themselves, because the
# y-mode selection of the choice might change if a choice symbol's
# visibility changes.
#
# We add these dependencies separately after dependency loop detection.
# The invalidation algorithm can handle the resulting
# <choice symbol> <-> <choice> dependency loops, but they make loop
# detection awkward.
for choice in self.unique_choices:
for sym in choice.syms:
sym._dependents.add(choice)
def _invalidate_all(self):
# Undefined symbols never change value and don't need to be
# invalidated, so we can just iterate over defined symbols.
# Invalidating constant symbols would break things horribly.
for sym in self.unique_defined_syms:
sym._invalidate()
for choice in self.unique_choices:
choice._invalidate()
#
# Post-parsing menu tree processing, including dependency propagation and
# implicit submenu creation
#
def _finalize_tree(self, node, visible_if):
# Propagates properties and dependencies, creates implicit menus (see
# kconfig-language.txt), removes 'if' nodes, and finalizes choices.
# This pretty closely mirrors menu_finalize() from the C
# implementation, with some minor tweaks (MenuNode holds lists of
# properties instead of each property having a MenuNode pointer, for
# example).
#
# node:
# The current "parent" menu node, from which we propagate
# dependencies
#
# visible_if:
# Dependencies from 'visible if' on parent menus. These are added to
# the prompts of symbols and choices.
if node.list:
# The menu node is a choice, menu, or if. Finalize each child in
# it.
if node.item is MENU:
visible_if = self._make_and(visible_if, node.visibility)
# Propagate the menu node's dependencies to each child menu node.
#
# The recursive _finalize_tree() calls assume that the current
# "level" in the tree has already had dependencies propagated. This
# makes e.g. implicit submenu creation easier, because it needs to
# look ahead.
self._propagate_deps(node, visible_if)
# Finalize the children
cur = node.list
while cur:
self._finalize_tree(cur, visible_if)
cur = cur.next
elif node.item.__class__ is Symbol:
# Add the node's non-node-specific properties (defaults, ranges,
# etc.) to the Symbol
self._add_props_to_sym(node)
# See if we can create an implicit menu rooted at the Symbol and
# finalize each child menu node in that menu if so, like for the
# choice/menu/if case above
cur = node
while cur.next and _auto_menu_dep(node, cur.next):
# This also makes implicit submenu creation work recursively,
# with implicit menus inside implicit menus
self._finalize_tree(cur.next, visible_if)
cur = cur.next
cur.parent = node
if cur is not node:
# Found symbols that should go in an implicit submenu. Tilt
# them up above us.
node.list = node.next
node.next = cur.next
cur.next = None
if node.list:
# We have a parent node with individually finalized child nodes. Do
# final steps to finalize this "level" in the menu tree.
_flatten(node.list)
_remove_ifs(node)
# Empty choices (node.list None) are possible, so this needs to go
# outside
if node.item.__class__ is Choice:
# Add the node's non-node-specific properties to the choice, like
# _add_props_to_sym() does
choice = node.item
choice.direct_dep = self._make_or(choice.direct_dep, node.dep)
choice.defaults += node.defaults
_finalize_choice(node)
def _propagate_deps(self, node, visible_if):
# Propagates 'node's dependencies to its child menu nodes
# If the parent node holds a Choice, we use the Choice itself as the
# parent dependency. This makes sense as the value (mode) of the choice
# limits the visibility of the contained choice symbols. The C
# implementation works the same way.
#
# Due to the similar interface, Choice works as a drop-in replacement
# for Symbol here.
basedep = node.item if node.item.__class__ is Choice else node.dep
cur = node.list
while cur:
cur.dep = dep = self._make_and(cur.dep, basedep)
# Propagate dependencies to prompt
if cur.prompt:
cur.prompt = (cur.prompt[0],
self._make_and(cur.prompt[1], dep))
if cur.item.__class__ in _SYMBOL_CHOICE:
# Propagate 'visible if' dependencies to the prompt
if cur.prompt:
cur.prompt = (cur.prompt[0],
self._make_and(cur.prompt[1], visible_if))
# Propagate dependencies to defaults
if cur.defaults:
cur.defaults = [(default, self._make_and(cond, dep))
for default, cond in cur.defaults]
# Propagate dependencies to ranges
if cur.ranges:
cur.ranges = [(low, high, self._make_and(cond, dep))
for low, high, cond in cur.ranges]
# Propagate dependencies to selects
if cur.selects:
cur.selects = [(target, self._make_and(cond, dep))
for target, cond in cur.selects]
# Propagate dependencies to implies
if cur.implies:
cur.implies = [(target, self._make_and(cond, dep))
for target, cond in cur.implies]
cur = cur.next
def _add_props_to_sym(self, node):
# Copies properties from the menu node 'node' up to its contained
# symbol, and adds (weak) reverse dependencies to selected/implied
# symbols.
#
# This can't be rolled into _propagate_deps(), because that function
# traverses the menu tree roughly breadth-first, meaning properties on
# symbols defined in multiple locations could end up in the wrong
# order.
sym = node.item
# See the Symbol class docstring
sym.direct_dep = self._make_or(sym.direct_dep, node.dep)
sym.defaults += node.defaults
sym.ranges += node.ranges
sym.selects += node.selects
sym.implies += node.implies
# Modify the reverse dependencies of the selected symbol
for target, cond in node.selects:
target.rev_dep = self._make_or(
target.rev_dep,
self._make_and(sym, cond))
# Modify the weak reverse dependencies of the implied
# symbol
for target, cond in node.implies:
target.weak_rev_dep = self._make_or(
target.weak_rev_dep,
self._make_and(sym, cond))
#
# Misc.
#
def _check_sym_sanity(self):
# Checks various symbol properties that are handiest to check after
# parsing. Only generates errors and warnings.
def num_ok(sym, type_):
# Returns True if the (possibly constant) symbol 'sym' is valid as a value
# for a symbol of type type_ (INT or HEX)
# 'not sym.nodes' implies a constant or undefined symbol, e.g. a plain
# "123"
if not sym.nodes:
return _is_base_n(sym.name, _TYPE_TO_BASE[type_])
return sym.orig_type is type_
for sym in self.unique_defined_syms:
if sym.orig_type in _BOOL_TRISTATE:
# A helper function could be factored out here, but keep it
# speedy/straightforward
for target_sym, _ in sym.selects:
if target_sym.orig_type not in _BOOL_TRISTATE_UNKNOWN:
self._warn("{} selects the {} symbol {}, which is not "
"bool or tristate"
.format(_name_and_loc(sym),
TYPE_TO_STR[target_sym.orig_type],
_name_and_loc(target_sym)))
for target_sym, _ in sym.implies:
if target_sym.orig_type not in _BOOL_TRISTATE_UNKNOWN:
self._warn("{} implies the {} symbol {}, which is not "
"bool or tristate"
.format(_name_and_loc(sym),
TYPE_TO_STR[target_sym.orig_type],
_name_and_loc(target_sym)))
elif sym.orig_type in _STRING_INT_HEX:
for default, _ in sym.defaults:
if default.__class__ is not Symbol:
raise KconfigError(
"the {} symbol {} has a malformed default {} -- expected "
"a single symbol"
.format(TYPE_TO_STR[sym.orig_type], _name_and_loc(sym),
expr_str(default)))
if sym.orig_type is STRING:
if not default.is_constant and not default.nodes and \
not default.name.isupper():
# 'default foo' on a string symbol could be either a symbol
# reference or someone leaving out the quotes. Guess that
# the quotes were left out if 'foo' isn't all-uppercase
# (and no symbol named 'foo' exists).
self._warn("style: quotes recommended around "
"default value for string symbol "
+ _name_and_loc(sym))
elif not num_ok(default, sym.orig_type): # INT/HEX
self._warn("the {0} symbol {1} has a non-{0} default {2}"
.format(TYPE_TO_STR[sym.orig_type],
_name_and_loc(sym),
_name_and_loc(default)))
if sym.selects or sym.implies:
self._warn("the {} symbol {} has selects or implies"
.format(TYPE_TO_STR[sym.orig_type],
_name_and_loc(sym)))
else: # UNKNOWN
self._warn("{} defined without a type"
.format(_name_and_loc(sym)))
if sym.ranges:
if sym.orig_type not in _INT_HEX:
self._warn(
"the {} symbol {} has ranges, but is not int or hex"
.format(TYPE_TO_STR[sym.orig_type],
_name_and_loc(sym)))
else:
for low, high, _ in sym.ranges:
if not num_ok(low, sym.orig_type) or \
not num_ok(high, sym.orig_type):
self._warn("the {0} symbol {1} has a non-{0} "
"range [{2}, {3}]"
.format(TYPE_TO_STR[sym.orig_type],
_name_and_loc(sym),
_name_and_loc(low),
_name_and_loc(high)))
def _check_choice_sanity(self):
# Checks various choice properties that are handiest to check after
# parsing. Only generates errors and warnings.
def warn_select_imply(sym, expr, expr_type):
msg = "the choice symbol {} is {} by the following symbols, but " \
"select/imply has no effect on choice symbols" \
.format(_name_and_loc(sym), expr_type)
# si = select/imply
for si in split_expr(expr, OR):
msg += "\n - " + _name_and_loc(split_expr(si, AND)[0])
self._warn(msg)
for choice in self.unique_choices:
if choice.orig_type not in _BOOL_TRISTATE:
self._warn("{} defined with type {}"
.format(_name_and_loc(choice),
TYPE_TO_STR[choice.orig_type]))
for node in choice.nodes:
if node.prompt:
break
else:
self._warn(_name_and_loc(choice) + " defined without a prompt")
for default, _ in choice.defaults:
if default.__class__ is not Symbol:
raise KconfigError(
"{} has a malformed default {}"
.format(_name_and_loc(choice), expr_str(default)))
if default.choice is not choice:
self._warn("the default selection {} of {} is not "
"contained in the choice"
.format(_name_and_loc(default),
_name_and_loc(choice)))
for sym in choice.syms:
if sym.defaults:
self._warn("default on the choice symbol {} will have "
"no effect, as defaults do not affect choice "
"symbols".format(_name_and_loc(sym)))
if sym.rev_dep is not sym.kconfig.n:
warn_select_imply(sym, sym.rev_dep, "selected")
if sym.weak_rev_dep is not sym.kconfig.n:
warn_select_imply(sym, sym.weak_rev_dep, "implied")
for node in sym.nodes:
if node.parent.item is choice:
if not node.prompt:
self._warn("the choice symbol {} has no prompt"
.format(_name_and_loc(sym)))
elif node.prompt:
self._warn("the choice symbol {} is defined with a "
"prompt outside the choice"
.format(_name_and_loc(sym)))
def _parse_error(self, msg):
raise KconfigError("{}couldn't parse '{}': {}".format(
"" if self._filename is None else
"{}:{}: ".format(self._filename, self._linenr),
self._line.strip(), msg))
def _trailing_tokens_error(self):
self._parse_error("extra tokens at end of line")
def _open(self, filename, mode):
# open() wrapper:
#
# - Enable universal newlines mode on Python 2 to ease
# interoperability between Linux and Windows. It's already the
# default on Python 3.
#
# The "U" flag would currently work for both Python 2 and 3, but it's
# deprecated on Python 3, so play it future-safe.
#
# A simpler solution would be to use io.open(), which defaults to
# universal newlines on both Python 2 and 3 (and is an alias for
# open() on Python 3), but it's appreciably slower on Python 2:
#
# Parsing x86 Kconfigs on Python 2
#
# with open(..., "rU"):
#
# real 0m0.930s
# user 0m0.905s
# sys 0m0.025s
#
# with io.open():
#
# real 0m1.069s
# user 0m1.040s
# sys 0m0.029s
#
# There's no appreciable performance difference between "r" and
# "rU" for parsing performance on Python 2.
#
# - For Python 3, force the encoding. Forcing the encoding on Python 2
# turns strings into Unicode strings, which gets messy. Python 2
# doesn't decode regular strings anyway.
return open(filename, "rU" if mode == "r" else mode) if _IS_PY2 else \
open(filename, mode, encoding=self._encoding)
def _check_undef_syms(self):
# Prints warnings for all references to undefined symbols within the
# Kconfig files
def is_num(s):
# Returns True if the string 's' looks like a number.
#
# Internally, all operands in Kconfig are symbols, only undefined symbols
# (which numbers usually are) get their name as their value.
#
# Only hex numbers that start with 0x/0X are classified as numbers.
# Otherwise, symbols whose names happen to contain only the letters A-F
# would trigger false positives.
try:
int(s)
except ValueError:
if not s.startswith(("0x", "0X")):
return False
try:
int(s, 16)
except ValueError:
return False
return True
for sym in (self.syms.viewvalues if _IS_PY2 else self.syms.values)():
# - sym.nodes empty means the symbol is undefined (has no
# definition locations)
#
# - Due to Kconfig internals, numbers show up as undefined Kconfig
# symbols, but shouldn't be flagged
#
# - The MODULES symbol always exists
if not sym.nodes and not is_num(sym.name) and \
sym.name != "MODULES":
msg = "undefined symbol {}:".format(sym.name)
for node in self.node_iter():
if sym in node.referenced:
msg += "\n\n- Referenced at {}:{}:\n\n{}" \
.format(n