Config and ConfigFile¶
confattr (config attributes) is a python library to make applications configurable.
This library defines the Config
class to create attributes which can be changed in a config file.
It uses the descriptor protocol to return it’s value when used as an instance attribute.
from confattr import Config
class Car:
speed_limit = Config('traffic-law.speed-limit', 50, unit='km/h')
def __init__(self) -> None:
self.speed = 0
def accelerate(self, value: int) -> None:
new_speed = self.speed + value
if new_speed > self.speed_limit:
raise ValueError('you are going too fast')
self.speed = new_speed
If you want to access the Config object itself you need to access it as a class attribute:
def print_config(self) -> None:
print('{key}: {val}'.format(key=type(self).speed_limit.key, val=self.speed_limit))
You load a config file with a ConfigFile
object.
In order to create an ConfigFile instance you need to provide a callback function which informs the user if the config file contains invalid lines.
This callback function takes two arguments: (1) a NotificationLevel
which says whether the notification is an error or an information and (2) the message to be presented to the user, either a str
or a BaseException
.
When you load a config file with ConfigFile.load()
all Config
objects are updated automatically.
if __name__ == '__main__':
from confattr import ConfigFile
config_file = ConfigFile(appname=__package__)
config_file.load()
# Print error messages which occurred while loading the config file.
# In this easy example it would have been possible to register the callback before calling load
# but in the real world the user interface will depend on the values set in the config file.
# Therefore the messages are stored until a callback is added.
config_file.set_ui_callback(lambda lvl, msg: print(msg))
c1 = Car()
print('speed_limit: %s' % c1.speed_limit)
Given the following config file (the location of the config file is determined by ConfigFile.iter_config_paths()
):
set traffic-law.speed-limit = 30
The script will give the following output:
speed_limit: 30
You can save the current configuration with ConfigFile.save()
if you want to write it to the default location
or with ConfigFile.save_file()
if you want to specify the path yourself.
filename = config_file.save()
print('configuration was written to %s' % filename)
This will write the following file:
# traffic-law.speed-limit
# -----------------------
# an int in km/h
set traffic-law.speed-limit = 30
Config file syntax¶
I have looked at the config files of different applications trying to come up with a syntax as intuitive as possible.
Two extremes which have heavily inspired me are the config files of vim and ranger.
Quoting and inline comments work like in a POSIX shell (except that there is no difference between single quotes and double quotes) as I am using shlex.split()
to split the lines.
The set
command has two different forms.
I recommend to not mix them in order to improve readability.
set key1=val1 [key2=val2 ...]
(inspired by vimrc)set
takes an arbitrary number of arguments, each argument sets one setting.Has the advantage that several settings can be changed at once. This is useful if you want to bind a set command to a key and process that command with
ConfigFile.parse_line()
if the key is pressed.If the value contains one or more spaces it must be quoted.
set greeting='hello world'
andset 'greeting=hello world'
are equivalent.set key [=] val
(inspired by ranger config)set
takes two arguments, the key and the value. Optionally a single equals character may be added in between as third argument.Has the advantage that key and value are separated by one or more spaces which can improve the readability of a config file.
If the value contains one or more spaces it must be quoted:
set greeting 'hello world'
andset greeting = 'hello world'
are equivalent.
I recommend to not use spaces in key names so that they don’t need to be wrapped in quotes.
It is possible to include another config file:
include filename
If
filename
is a relative path it is relative to the directory of the config file it appears in.
Different values for different objects¶
A Config
object always returns the same value, regardless of the owning object it is an attribute of:
from confattr import Config
class Car:
speed_limit = Config('traffic-law.speed-limit', 50, unit='km/h')
c1 = Car()
c2 = Car()
print(c1.speed_limit, c2.speed_limit)
c2.speed_limit = 30 # don't do this, this is misleading!
print(c1.speed_limit, c2.speed_limit)
Output:
50 50
30 30
If you want to have different values for different objects you need to use MultiConfig
instead.
This requires the owning object to have a special attribute called config_id
.
All objects which have the same config_id
share the same value.
All objects which have different config_id
can have different values (but don’t need to have different values).
import enum
from confattr import Config, MultiConfig, ConfigId, ConfigFile
class Color(enum.Enum):
RED = 'red'
YELLOW = 'yellow'
GREEN = 'green'
BLUE = 'blue'
WHITE = 'white'
BLACK = 'black'
class Car:
speed_limit = Config('traffic-law.speed-limit', 50, unit='km/h')
color = MultiConfig('car.color', Color.BLACK)
def __init__(self, config_id: ConfigId) -> None:
self.config_id = config_id
if __name__ == '__main__':
config_file = ConfigFile(appname=__package__)
config_file.load()
config_file.set_ui_callback(lambda lvl, msg: print(msg))
cars = []
for config_id in MultiConfig.config_ids:
cars.append(Car(config_id))
cars.append(Car(ConfigId('another-car')))
for car in cars:
print('color of %s: %s' % (car.config_id, car.color))
Given the following config file:
set traffic-law.speed-limit = 30
[alices-car]
set car.color = red
[bobs-car]
set car.color = blue
It creates the following output:
color of alices-car: Color.RED
color of bobs-car: Color.BLUE
color of another-car: Color.BLACK
another-car
gets the default color black as it is not set in the config file.
You can change this default color in the config file by setting it before specifying a config id or after specifying the special config id general
(Config.default_config_id
).
Note how this adds general
to MultiConfig.config_ids
.
set traffic-law.speed-limit = 30
set car.color = white
[alices-car]
set car.color = red
[bobs-car]
set car.color = blue
Creates the following output:
color of general: Color.WHITE
color of alices-car: Color.RED
color of bobs-car: Color.BLUE
color of another-car: Color.WHITE
Custom data types¶
It is possible to use custom data types. You can use that for example to avoid repeating a part of help which applies to several settings:
import re
from confattr import Config, ConfigFile
class Regex(str):
type_name = 'regular expression'
help = 'https://docs.python.org/3/library/re.html#regular-expression-syntax'
class Parser:
re_mount_output = Config('udisksctl.mount-output-pattern', Regex(r'^.*?(?P<mountpath>/(\S+/)*[^/]+?)\.?$'),
help='a regular expression to parse the output of `udisksctl mount`. Must contain a named group called "mountpath".')
re_unlock_output = Config('udisksctl.unlock-output-pattern', Regex(r'^.*?(?P<unlockpath>/(\S+/)*[^/]+?)\.?$'),
help='a regular expression to parse the output of `udisksctl unlock`. Must contain a named group called "unlockpath".')
def compile_regex(self) -> None:
'''
This must be called every time after the config file has been loaded.
'''
self.reo_mount_output = re.compile(self.re_mount_output)
self.reo_unlock_output = re.compile(self.re_unlock_output)
if __name__ == '__main__':
ConfigFile(appname=__package__).save()
This exports the following config file:
# Data types
# ----------
# - regular expression
# https://docs.python.org/3/library/re.html#regular-expression-syntax
# udisksctl.mount-output-pattern
# ------------------------------
# a regular expression
# a regular expression to parse the output of `udisksctl mount`. Must contain a named group called "mountpath".
set udisksctl.mount-output-pattern = '^.*?(?P<mountpath>/(\S+/)*[^/]+?)\.?$'
# udisksctl.unlock-output-pattern
# -------------------------------
# a regular expression
# a regular expression to parse the output of `udisksctl unlock`. Must contain a named group called "unlockpath".
set udisksctl.unlock-output-pattern = '^.*?(?P<unlockpath>/(\S+/)*[^/]+?)\.?$'
__str__()
must return a string representation suitable for the config file and the constructor must create an equal object if it is passed the return value of __str__()
. This is fulfilled by inheriting from str
.
type_name
is a special str attribute which specifies how the type is called in the config file. If it is missing it is derived from the class name.
help
is a special str attribute which contains a description which is printed in the config file. If it is missing the doc string is used instead.
confattr.types
defines several such types, including the above definition of Regex
.
For more information on the supported data types see Config
.
Adding new commands to the config file syntax¶
You can extend this library by defining new commands which can be used in the config file.
All you need to do is subclass ConfigFileCommand
and implement the ConfigFileCommand.run()
method.
Additionally I recommend to provide a doc string explaining how to use the command in the config file. The doc string is used by get_help()
which may be used by an in-app help.
Optionally you can set ConfigFileCommand.name
and ConfigFileCommand.aliases
and implement the ConfigFileCommand.save()
method.
Alternatively ConfigFileArgparseCommand
can be subclassed instead, it aims to make the parsing easier and avoid redundancy in the doc string by using the argparse
module.
You must implement init_parser()
and run_parsed()
.
Note that init_parser()
is a class method.
You should give a doc string describing what the command does.
In contrast to ConfigFileCommand
argparse
adds usage and the allowed arguments to the output of ConfigFileArgparseCommand.get_help()
automatically.
For example you may want to add a new command to bind keys to whatever kind of command. The following example assumes urwid as user interface framework.
import argparse
from collections.abc import Sequence
import urwid
from confattr import ConfigFileArgparseCommand, ConfigFile, Config, NotificationLevel
class Map(ConfigFileArgparseCommand):
'''
bind a command to a key
'''
@classmethod
def init_parser(cls) -> None:
cls.parser.add_argument('key', help='http://urwid.org/manual/userinput.html#keyboard-input')
cls.parser.add_argument('cmd', help='any urwid command')
def run_parsed(self, args: argparse.Namespace) -> None:
urwid.command_map[args.key] = args.cmd
if __name__ == '__main__':
# config
choices = Config('choices', ['vanilla', 'strawberry'])
urwid.command_map['enter'] = 'confirm'
config_file = ConfigFile(appname=__package__)
config_file.load()
# show errors in config
palette = [(NotificationLevel.ERROR.value, 'dark red', 'default')]
status_bar = urwid.Pile([])
def on_config_message(lvl: NotificationLevel, msg: 'str|BaseException') -> None:
markup = (lvl.value, str(msg))
widget_options_tuple = (urwid.Text(markup), status_bar.options('pack'))
status_bar.contents.append(widget_options_tuple)
config_file.set_ui_callback(on_config_message)
# a simple example app showing check boxes and printing the user's choice to stdout
def key_handler(key: str) -> None:
cmd = urwid.command_map[key]
if cmd == 'confirm':
raise urwid.ExitMainLoop()
checkboxes = [urwid.CheckBox(choice) for choice in choices.value]
frame = urwid.Frame(urwid.Filler(urwid.Pile(checkboxes)), footer=status_bar)
urwid.MainLoop(frame, palette=palette, unhandled_input=key_handler).run()
for ckb in checkboxes:
print(f'{ckb.label}: {ckb.state}')
Given the following config file it is possible to move the cursor upward and downward with j
and k
like in vim:
map j 'cursor down'
map k 'cursor up'
map q 'confirm'
The help for the newly defined command looks like this:
print(Map.get_help())
usage: map key cmd
bind a command to a key
positional arguments:
key http://urwid.org/manual/userinput.html#keyboard-input
cmd any urwid command
(All subclasses of ConfigFileCommand
are saved in ConfigFileCommand.__init_subclass__()
and can be retrieved with ConfigFileCommand.get_command_types()
.
The ConfigFile
constructor uses that if commands
is not given.)
Writing custom commands to the config file¶
The previous example has shown how to define new commands so that they can be used in the config file.
Let’s continue that example so that calls to the custom command map
are written with ConfigFile.save()
.
All you need to do for that is implementing the ConfigFileCommand.save()
method.
Experimental support for type checking **kw
has been added in mypy 0.981.
SaveKwargs
depends on typing.TypedDict
and therefore is not available before Python 3.8.
import argparse
import typing
if typing.TYPE_CHECKING:
from typing_extensions import Unpack # This will hopefully be replaced by the ** syntax proposed in https://peps.python.org/pep-0692/
from confattr import SaveKwargs
import urwid
from confattr import ConfigFileArgparseCommand, ConfigFile
class Map(ConfigFileArgparseCommand):
'''
bind a command to a key
'''
@classmethod
def init_parser(cls) -> None:
cls.parser.add_argument('key', help='http://urwid.org/manual/userinput.html#keyboard-input')
cls.parser.add_argument('cmd', help='any urwid command')
def run_parsed(self, args: argparse.Namespace) -> None:
urwid.command_map[args.key] = args.cmd
def save(self, f: typing.TextIO, **kw: 'Unpack[SaveKwargs]') -> None:
for key, cmd in sorted(urwid.command_map._command.items(), key=lambda key_cmd: str(key_cmd[1])):
quoted_key = self.config_file.quote(key)
quoted_cmd = self.config_file.quote(cmd)
print(f'map {quoted_key} {quoted_cmd}', file=f)
if __name__ == '__main__':
ConfigFile(appname=__package__).save()
However, urwid.command_map
contains more commands than the example app uses so writing all of them might be confusing.
Therefore let’s add a keyword argument to write only the specified commands:
import argparse
from collections.abc import Sequence
import typing
import urwid
from confattr import ConfigFileArgparseCommand, ConfigFile
if typing.TYPE_CHECKING:
from typing_extensions import Unpack # This will hopefully be replaced by the ** syntax proposed in https://peps.python.org/pep-0692/
from confattr import SaveKwargs
class MapSaveKwargs(SaveKwargs, total=False):
urwid_commands: 'Sequence[str]'
class Map(ConfigFileArgparseCommand):
'''
bind a command to a key
'''
@classmethod
def init_parser(cls) -> None:
cls.parser.add_argument('key', help='http://urwid.org/manual/userinput.html#keyboard-input')
cls.parser.add_argument('cmd', help='any urwid command')
def run_parsed(self, args: argparse.Namespace) -> None:
urwid.command_map[args.key] = args.cmd
def save(self, f: typing.TextIO, **kw: 'Unpack[MapSaveKwargs]') -> None:
commands = kw.get('urwid_commands', sorted(urwid.command_map._command.values()))
for cmd in commands:
for key in urwid.command_map._command.keys():
if urwid.command_map[key] == cmd:
quoted_key = self.config_file.quote(key)
quoted_cmd = self.config_file.quote(cmd)
print(f'map {quoted_key} {quoted_cmd}', file=f)
if __name__ == '__main__':
urwid_commands = [urwid.CURSOR_UP, urwid.CURSOR_DOWN, urwid.ACTIVATE, 'confirm']
mapkw: 'MapSaveKwargs' = dict(urwid_commands=urwid_commands)
kw: 'SaveKwargs' = mapkw
config_file = ConfigFile(appname=__package__)
config_file.save(**kw)
This produces the following config file:
map up 'cursor up'
map down 'cursor down'
map ' ' activate
map enter activate
If you don’t care about Python < 3.8 you can import SaveKwargs
normally and save a line when calling ConfigFile.save()
:
kw: SaveKwargs = MapSaveKwargs(urwid_commands=...)
config_file.save(**kw)
Customizing the config file syntax¶
If you want to make minor changes to the syntax of the config file you can subclass the corresponding command, i.e. Set
or Include
.
For example if you want to use a key: value
syntax you could do the following.
I am setting name
to an empty string (i.e. confattr.DEFAULT_COMMAND
) to make this the default command which is used if an unknown command is encountered.
This makes it possible to use this command without writing out it’s name in the config file.
from confattr import Set, ParseException, ConfigId
import typing
if typing.TYPE_CHECKING:
from confattr import ParseSplittedLineKwargs, SaveKwargs
from typing_extensions import Unpack
from collections.abc import Sequence
class SimpleSet(Set, replace=True):
name = ''
SEP = ':'
def run(self, cmd: 'Sequence[str]', **kw: 'Unpack[ParseSplittedLineKwargs]') -> None:
ln = kw['line']
if self.SEP not in ln:
raise ParseException(f'missing {self.SEP} between key and value')
key, value = ln.split(self.SEP)
value = value.lstrip()
self.parse_key_and_set_value(key, value)
def save_config_instance(self, f: typing.TextIO, instance: 'Config[object]', config_id: 'ConfigId|None', **kw: 'Unpack[SaveKwargs]') -> None:
# this is called by Set.save
if kw['comments']:
self.write_help(f, instance)
value = self.format_value(instance, config_id)
#value = self.config_file.quote(value) # not needed because run uses line instead of cmd
ln = f'{instance.key}{self.SEP} {value}\n'
f.write(ln)
if __name__ == '__main__':
from confattr import Config, ConfigFile
color = Config('favorite color', 'white')
subject = Config('favorite subject', 'math')
config_file = ConfigFile(appname=__package__)
config_file.load()
config_file.set_ui_callback(lambda lvl, msg: print(msg))
print(color.value)
print(subject.value)
Then a config file might look like this:
favorite color: sky blue
favorite subject: computer science
Please note that it’s still possible to use the include
command.
If you want to avoid that use
from confattr import ConfigFileCommand, Include
ConfigFileCommand.delete_command_type(Include)
If you want to make bigger changes like using JSON you need to subclass ConfigFile
.
Config without classes¶
If you want to use Config
objects without custom classes you can access the value via the Config.value
attribute:
from confattr import Config, ConfigFile
backend = Config('urwid.backend', 'auto', allowed_values=('auto', 'raw', 'curses'))
config_file = ConfigFile(appname=__package__)
config_file.load()
config_file.set_ui_callback(lambda lvl, msg: print(msg))
print(backend.value)
Given the following config file (the location of the config file is determined by ConfigFile.iter_config_paths()
):
set urwid.backend = curses
The script will give the following output:
curses
Environment variables¶
This library is influenced by the following environment variables:
XDG_CONFIG_HOME
defines the base directory relative to which user-specific configuration files should be stored. [1] [2]XDG_CONFIG_DIRS
defines the preference-ordered set of base directories to search for configuration files in addition to theXDG_CONFIG_HOME
base directory. The directories inXDG_CONFIG_DIRS
should be separated with a colon. [1] [2]CONFATTR_FILENAME
defines the value ofConfigFile.FILENAME
. [2]