53. utils
— A collection of miscellaneous utility functions¶
53.1. Classes defined in module utils¶
- class utils.PyformexWarning[source]¶
Warning category for announcing changes in pyFormex code
This is used to warn the user about changes and deprecations in pyFormex. The messages from this category are normally stored in the messages module. The user can filter the messages out for the current and future sessions.
- class utils.PyformexRuntimeWarning[source]¶
Warning category for run time warnings
Run time warnings are warnings to the end user that can only be filtered out for the current session.
- class utils.PyformexRuntimeError[source]¶
Warning category for run time errors
Run time error are obvious errors that should always be notified to the user. They can not be filtered out.
53.2. Functions defined in module utils¶
- utils.pzf_register(clas)[source]¶
Class decorator to allow load from PZF format.
Adding this decoratot to a class registers the class with the
pzffile
module. Objects in a PZF file with class name equal to clas.__name__ will then be restored using this class.
- utils.memoize(func)[source]¶
Remember the result of an instance method call.
This is a decorator function that saves the result of an instance method call into the instance. Subsequent use of the function will return the result from memory instead of recomputing it.
Notes
If the decorated function has no arguments other than self, this decorator can be stacked with @property to create a cached property.
The result is saved in a dict with the function name and its arguments as key. The dict is stored as an instance attribute _memory. It is created automatically on first used of a memoizing method.
Examples
We create a class with a single method, returning a list of 10 random ints in the range from 0 to 20. The method’s result is memoized.
>>> class C: ... @memoize ... def random_ints(self): ... print("Computing random ints") ... return [random.randint(0,20) for i in range(10)]
We create an instance and call the random_ints method.
>>> c = C() >>> a = c.random_ints() Computing random ints >>> print(len(a), min(a) >= 0, max(a) <= 20) 10 True True
When calling the random_ints method again, the method is not actually executed, but the memoized values are returned, so they are the same as the previous.
>>> b = c.random_ints() >>> print(len(b), a == b) 10 True
If we create another instance, we get other values, because the memoizing is done per instance.
>>> b = C().random_ints() Computing random ints >>> print(len(b), a == b) 10 False
The results are stored in the _memory attribute. They can be deleted to force recomputation.
>>> print(c._memory) {'random_ints': [...]}
- utils.rev_lookup(dictionary, value, default=None)[source]¶
Reverse lookup in a dict
Lookup a value in a dict, and return its key (the first match).
- Parameters:
dictionary (dict) – The dict in which to lookup a value
value (anything) – The value to lookup in the dict
default (anything) – The key to return if the value was not found
- Returns:
key (anything) – The first key in the dict whose value matches the given value, or default if no match was found.
- utils.warningCategory(category)[source]¶
Return a warning category and its alias
The input can be a category or an alias. Returns both as a tuple (alias, category). Invalid values return the default category (‘W’, Warning).
- utils.warningAction(action)[source]¶
Return a warning action and its alias
The input can be an action string or an alias. Returns both as a tuple (alias, action). Invalid values return the default action (‘i’, ‘ignore’).
- utils.retrans(s)[source]¶
Transform a string to an re to match itself
Replaces all special characters.
- utils.filter_warning(action, message, category, module, save='')[source]¶
Add a warnings filter.
- Parameters:
action – The warning filter action or alias.
message (str | Warning | None) – The text of the message to be filtered.
category – The warning category or alias.
module (str) – The module in which the warning is to be filtered. ‘’ for all.
save (str) – The action for future sessions. If not empty, the filter is saved in the user settings.
- utils.resetWarningFilters()[source]¶
Reset the warning filters
Reset the warning filters to the Python defaults plus the ones listed in the ‘warnings/filters’ configuration variable.
- utils.format_warning(message, category, filename, lineno, line=None)[source]¶
Format a warning message.
This acts as a replacement for
warnings.formatwarning()
. Its parameters have the same meaning, with the special cases mentioned below.- Parameters:
message (str | Warning) – The message to be shown. If category is PyformexWarning, the message is a simple mnemonic string and is looked up with
messages.getMessage()
to get the complete message. If the resulting message does not start with ‘..’, it is decorated to create an ReST format message with a proper header.
Examples
>>> print(format_warning('test_message', PyformexWarning, 'unknown', 33)) .. pyFormex Warning ---------------- **This is the warning shown to the user** PyformexWarning called from: unknown, line: 33
- utils.show_warning(message, category, filename, lineno, file=None, line=None)[source]¶
Replace the default warnings.showwarning
We display the warnings using our interactive warning widget. This feature can be turned off by setting
cfg['warnings/popup'] = False
- utils.warn(message, category=<class 'utils.PyformexWarning'>, stacklevel=2, uplevel=0)[source]¶
Create a warning through Python’s warnings system.
This is like
warnings.warn()
, but the default category isPyformexWarning
and it takes an extra uplevel argument whose value is added to the stacklevel (because it is easier to specify a relative level).Examples
>>> warn('test_message')
- utils.warning(message, uplevel=0)[source]¶
Create a pyFormex run time warning.
This calls
warn()
with categoryPyformexRuntimeWarning
.
- utils.error(message, uplevel=0)[source]¶
Create a pyFormex run time warning.
This calls
warn()
with categoryPyformexRuntimeError
.
- utils.with_warning(message, category=<class 'utils.PyformexWarning'>, uplevel=0)[source]¶
Decorator to add a warning to a function.
Adding this decorator to a function will warn the user with the supplied message when the decorated function gets executed for the first time in a session. An option is provided to switch off this warning in future sessions.
This can be used as follows:
@utils.with_warning('test_message') def func(): ...
- utils.deprecated_by(replace=None, *, uplevel=0)[source]¶
Decorator to deprecate a function by another one.
Adding this decorator to a function will warn the user with a message that the function is deprecated and should be replaced with replace.
- utils.system(args, *, verbose=False, wait=True, **kargs)[source]¶
Execute a command through the operating system.
This is a wrapper around the
process.run()
function, aimed particularly at the pyFormex GUI user. It has one extra parameter: verbose. Seeprocess.run()
for the other parameters.- Parameters:
verbose (bool) – If True, the command, and a report of its outcome in case of failure, timeout or error exit, are written to stdout.
- Returns:
DoneProcess
or subprocess.Popen – If wait is True, returns aDoneProcess
with the outcome of the command. If wait is False, returns a subprocess.Popen which can be used to communicate with the started subprocess.
See also
process.run
run a system command in a subprocess
command
call
system()
with some other defaults
Examples
>>> P = system("pwd") >>> P.stdout.strip('\n') == os.getcwd() True
>>> P = system('true') >>> P DoneProcess(args=['true'], returncode=0, stdout='', stderr='') >>> P = system('false', capture_output=False) >>> P DoneProcess(args=['false'], returncode=1) >>> P = system('False', verbose=True) Running command: False DoneProcess report args: ['False'] Command failed to run! returncode: 127
>>> P = system("sleep 5", timeout=1, verbose=True) Running command: sleep 5 DoneProcess report args: ['sleep', '5'] returncode: -1 stdout: stderr: timedout: True
- utils.command(args, verbose=True, check=True, **kargs)[source]¶
Run an external command in a user friendly way.
This is equivalent with
system()
with verbose=True by default.
- utils.matchMany(regexps, target)[source]¶
Return multiple regular expression matches of the same target string.
- utils.matchAny(regexps, target)[source]¶
Check whether target matches any of the regular expressions.
- utils.matchNone(regexps, target)[source]¶
Check whether target matches none of the regular expressions.
- utils.matchAll(regexps, target)[source]¶
Check whether targets matches all of the regular expressions.
- utils.okURL(url)[source]¶
Check that an URL is displayable in the browser.
- Parameters:
url (URL) – The URL to be checked.
- Returns:
bool – True if
url
starts with a protocol that is either ‘http:’, ‘https:’ or ‘file:’; else False
- utils.projectName(fn)[source]¶
Derive a project name from a file name.
The project name is the basename of the file without the extension. It is equivalent with Path(fn).stem
Examples
>>> projectName('aa/bb/cc.dd') 'cc' >>> projectName('cc.dd') 'cc' >>> projectName('cc') 'cc'
- utils.findIcon(name)[source]¶
Return the file name for an icon with given name.
- Parameters:
name (str) – Name of the icon: this is the stem fof the filename.
- Returns:
str – The full path name of an icon file with the specified name, found in the pyFormex icon folder, or the question mark icon file, if no match was found.
Examples
>>> print(findIcon('view-xr-yu').relative_to(pf.cfg['pyformexdir'])) icons/view-xr-yu.xpm >>> print(findIcon('right').relative_to(pf.cfg['pyformexdir'])) icons/64x64/right.png >>> print(findIcon('xyz').relative_to(pf.cfg['pyformexdir'])) icons/question.xpm >>> print(findIcon('recording').relative_to(pf.cfg['pyformexdir'])) icons/recording.gif
- utils.listIconNames(dirs=None, types=None)[source]¶
Return the list of available icons by their name.
- Parameters:
- Returns:
list of str – A sorted list of the icon names available in the pyFormex icons folder.
Examples
>>> listIconNames()[:4] ['clock', 'dist-angle', 'down', 'down'] >>> listIconNames([pf.cfg['icondir'] / '64x64'])[:4] ['down', 'ff', 'info', 'lamp'] >>> listIconNames(types=['.xpm'])[:4] ['clock', 'dist-angle', 'down', 'empty']
- utils.sourceFiles(relative=False, symlinks=True, extended=False, includedir=None)[source]¶
Return the list of pyFormex .py source files.
- Parameters:
relative (bool) – If True, returned filenames are relative to the current directory.
symlinks (bool) – If False, files that are symbolic links are retained in the list. The default is to remove them.
extended (bool) – If True, also return the .py files in all the paths in the configured appdirs and scriptdirs.
- Returns:
list of str – A list of filenames of .py files in the pyFormex source tree, and, if
extended
is True, .py files in the configured app and script dirs as well.
- utils.grepSource(pattern, options='', relative=True, verbose=False)[source]¶
Finds pattern in the pyFormex source files.
Uses the grep program to find all occurrences of some specified pattern text in the pyFormex source .py files (including the examples). Extra options can be passed to the grep command. See man grep for more info.
Returns the output of the grep command.
- utils.findModuleSource(module)[source]¶
Find the path of the source file of a module
module is either an imported module (pkg.mod) or a string with the module name (‘pkg.mod’), imported or not. Returns the source file from which the module was/would be loaded when imported. Raises an error if the module can not be imported or does not have a source file.
- utils.humanSize(size, units, ndigits=-1)[source]¶
Convert a number to a human size.
Large numbers are often represented in a more human readable form using k, M, G prefixes. This function returns the input size as a number with the specified prefix.
- Parameters:
size (int or float) – A number to be converted to human readable form.
units (str) – A string specifying the target units. The first character should be one of k,K,M,G,T,P,E,Z,Y. ‘k’ and ‘K’ are equivalent. A second character ‘i’ can be added to use binary (K=1024) prefixes instead of decimal (k=1000).
ndigits (int, optional) – If provided and >=0, the result will be rounded to this number of decimal digits.
- Returns:
float – The input value in the specified units and possibly rounded to
ndigits
.
Examples
>>> humanSize(1234567890,'k') 1234567.89 >>> humanSize(1234567890,'M',0) 1235.0 >>> humanSize(1234567890,'G',3) 1.235 >>> humanSize(1234567890,'Gi',3) 1.15
- utils.setSaneLocale(localestring='')[source]¶
Set a sane local configuration for LC_NUMERIC.
localestring is the locale string to be set, e.g. ‘en_US.UTF-8’ or ‘C.UTF-8’ for no locale.
Sets the
LC_ALL
locale to the specified string if that is not empty, and (always) setsLC_NUMERIC
andLC_COLLATE
to ‘C.UTF-8’.Changing the LC_NUMERIC setting is a very bad idea! It makes floating point values to be read or written with a comma instead of a the decimal point. Of course this makes input and output files completely incompatible. You will often not be able to process these files any further and create a lot of troubles for yourself and other people if you use an LC_NUMERIC setting different from the standard.
Because we do not want to help you shoot yourself in the foot, this function always sets
LC_NUMERIC
back to a sane ‘C’ value and we call this function when pyFormex is starting up.
- utils.rreplace(source, old, new, count=1)[source]¶
Replace substrings starting from the right.
Replaces count occurrences of old substring with a new substring. This is like str.replace, but counting starts from the right. The default count=1 replaces only the last occurrence, and is identical to str.replace (which is then preferred).
- Parameters:
- Returns:
str – The string with the replacements made. If source was a subclass of str, the returned string will be of the same subclass.
Examples
>>> print(rreplace('abababa', 'ab', '+de')) abab+dea >>> print(rreplace('abababa', 'ab', '+de', 2)) ab+de+dea >>> for i in (0, 1, 2, 3, 4, -1): ... print(f"{i}: {rreplace('abcabcabc', 'ab', '-ef', i)}") 0: abcabcabc 1: abcabc-efc 2: abc-efc-efc 3: -efc-efc-efc 4: -efc-efc-efc -1: -efc-efc-efc >>> rreplace(Path('dirname/filename.ext'), 'name.e', 'name00.n') Path('dirname/filename00.nxt')
- utils.strNorm(s)[source]¶
Normalize a string.
Text normalization removes all ‘&’ characters and converts it to lower case.
>>> strNorm("&MenuItem") 'menuitem'
- utils.slugify(text, delim='-')[source]¶
Convert a string into a URL-ready readable ascii text.
Examples
>>> slugify("http://example.com/blog/[Some] _ Article's Title--") 'http-example-com-blog-some-article-s-title' >>> slugify("&MenuItem") 'menuitem'
- utils.sprint(*args, **kargs)[source]¶
Print to a string
This takes all the parameters as
print()
but returs the result in a string.
- utils.textFormat(text)[source]¶
Detect text format
- Parameters:
text (str) – A multiline string in one of the supported formats: plain text, html, rest, markdown
- Returns:
format (str) – The detected format: one of ‘plain’, ‘html’, ‘rest’ or ‘markdown’
Examples
>>> textFormat('''.. ... Header ... ------ ... ''') 'rest'
- utils.convertText(text, format='')[source]¶
Convert a text to a format recognized by Qt.
Input text format is plain, rest, markdown or html. Output text format is plain, markdown or html, with rest being converted to html.
- Parameters:
- Returns:
text (str) – The converted text, being either plain markdown or html.
format (str) – The output format: ‘plain’, ‘markdown’ or ‘html’
Notes
For the conversion of reStructuredText to work, the Python docutils have to be installed on the system.
Examples
>>> convertText('''.. ... Header ... ------ ... ''')[0].startswith('<?xml') True
- utils.forceRest(text, underline=True)[source]¶
Convert a text string to have it recognized as reStructuredText.
- Parameters:
- Returns:
str – The input text with two lines prepended: a line with ‘..’ and a blank line. The pyFormex text display functions will then recognize the text as being reStructuredText. Since the ‘..’ starts a comment in reStructuredText, the extra lines are not displayed. If underline=True, an extra line is added below the (original) first line, to make that line appear as a header.
Examples
>>> print(forceRest('Header\nBody', underline=True)) .. Header ------ Body
- utils.underlineHeader(s, c='-')[source]¶
Underline the first line of a text.
- Parameters:
s (str) – A multiline string.
c (char, optional) – The character to use for underlining. Default is ‘-‘.
- Returns:
str – A multiline string with the original text plus an extra line inserted below the first line. The new line has the same length as the first, but all characters are equal to the specified char.
Examples
>>> print(underlineHeader("Hello World")) Hello World -----------
- utils.sameLength(lines, length=-1, adjust='l')[source]¶
Make a sequence of strings the same length.
- Parameters:
length (int) – The required length of the lines. If negative, the length is set to the maximum input line length.
adjust ('l' | 'c' | 'r') – How the input lines are adjusted to respectively the left, the center or the right of the total length of the line.
Examples
>>> sameLength(['a', 'bb', 'ccc']) ['a ', 'bb ', 'ccc'] >>> sameLength(['a', 'bb', 'ccc'], adjust='c') [' a ', ' bb', 'ccc'] >>> sameLength(['a', 'bb', 'ccc'], adjust='r') [' a', ' bb', 'ccc'] >>> sameLength(['a', 'bb', 'ccc'], length=2) ['a ', 'bb', 'cc']
- utils.framedText(text, padding=[0, 2, 0, 2], border=[1, 2, 1, 2], margin=[0, 0, 0, 0], borderchar='####', adjust='l')[source]¶
Create a text with a frame around it.
- Parameters:
padding (list of int) – Number of blank spaces around text, at the top, right, bottom, left.
border (list of int) – Border width, at the top, right, bottom, left.
margin (list of int) – Number of blank spaces around border, at the top, right, bottom, left.
borderchar (str) – Border charater, at the top, right, bottom, left.
width (int) – Intended width of the
adjust ('l' | 'c' | 'r') – Adjust the text to the left, center or right.
- Returns:
str – A multiline string with the formatted framed text.
Examples
>>> print(framedText("Hello World,\nThis is me calling",adjust='c')) ########################## ## Hello World, ## ## This is me calling ## ########################## >>> print(framedText("Hello World,\nThis is me calling",margin=[1,0,0,3])) ########################## ## Hello World, ## ## This is me calling ## ##########################
- utils.prefixText(text, prefix)[source]¶
Add a prefix to all lines of a text.
- Parameters:
- Returns:
str – A multiline string with the input lines prefixed with prefix.
Examples
>>> print(prefixText("line1\nline2","** ")) ** line1 ** line2
- utils.is_script(appname)[source]¶
Checks whether an application name is rather a script file name
- Parameters:
appname (str) – The name of a script file or an app.
- Returns:
bool – True if appname ends with ‘.py’, or contains a ‘/’.
- utils.prefixDict(d, prefix='')[source]¶
Prefix all the keys of a dict with the given prefix.
- Parameters:
- Returns:
dict – A dict with the same contents as the input, but where all keys have been prefixed with the given prefix string.
Examples
>>> prefixDict({'a':0,'b':1},'p_') {'p_a': 0, 'p_b': 1}
- utils.subDict(d, prefix='', strip=True, remove=False)[source]¶
Return a dict with the items whose key starts with prefix.
- Parameters:
- Returns:
dict – A dict with all the items from
d
whose key starts withprefix
. The keys in the returned dict will have the prefix stripped off, unless strip=False is specified.
Examples
>>> subDict({'p_a':0,'q_a':1,'p_b':2}, 'p_') {'a': 0, 'b': 2} >>> subDict({'p_a':0,'q_a':1,'p_b':2}, 'p_', strip=False) {'p_a': 0, 'p_b': 2} >>> a = {'p_a':0,'q_a':1,'p_b':2} >>> b = subDict(a, 'p_', remove=True, strip=False) >>> a, b ({'q_a': 1}, {'p_a': 0, 'p_b': 2})
- utils.selectDict(d, keys, remove=False)[source]¶
Return a dict with the items whose key is in keys.
- Parameters:
- Returns:
dict – A dict with all the items from
d
whose key is inkeys
.
See also
removeDict
the complementary operation, returns items not in
keys
.
Examples
>>> d = dict([(c,c*c) for c in range(4)]) >>> print(d) {0: 0, 1: 1, 2: 4, 3: 9} >>> selectDict(d,[2,0]) {2: 4, 0: 0} >>> print(d) {0: 0, 1: 1, 2: 4, 3: 9} >>> selectDict(d,[2,0,6],remove=True) {2: 4, 0: 0} >>> print(d) {1: 1, 3: 9}
- utils.removeDict(d, keys)[source]¶
Return a dict with the specified keys removed.
- Parameters:
- Returns:
dict – A dict with all the items from
d
whose key is not inkeys
.
See also
selectDict
the complementary operation returning the items in
keys
Examples
>>> d = dict([(c,c*c) for c in range(6)]) >>> removeDict(d,[4,0]) {1: 1, 2: 4, 3: 9, 5: 25}
- utils.mutexkeys(d, keys)[source]¶
Enforce a set of mutually exclusive keys in a dictionary.
This makes sure that d has only one of the specified keys. It modifies the dictionary inplace.
- Parameters:
d (dict) – The input dictionary.
keys – A list of dictionary keys that are mutually exclusive.
Examples
>>> d = {'a':0, 'b':1, 'c':2} >>> mutexkeys(d, ['b', 'c', 'a']) >>> print(d) {'b': 1}
- utils.refreshDict(d, src)[source]¶
Refresh a dict with values from another dict.
The values in the dict d are update with those in src. Unlike the dict.update method, this will only update existing keys but not add new keys.
- utils.inverseDict(d)[source]¶
Return the inverse of a dictionary.
Returns a dict with keys and values interchanged.
Example:
>>> inverseDict({'a':0,'b':1}) {0: 'a', 1: 'b'}
- utils.selectDictValues(d, values)[source]¶
Return the keys in a dict which have a specified value
d: a dict where all the keys are strings.
values: a list/set of values.
The return value is a list with all the keys from d whose value is in keys.
Example:
>>> d = dict([(c,c*c) for c in range(6)]) >>> selectDictValues(d,range(10)) [0, 1, 2, 3]
- utils.listFonts(pattern='', include=None, exclude=None)[source]¶
List the fonts known to the system.
This uses the ‘fc-list’ command from the fontconfig package to find a list of font files installed on the user’s system. The list of files can be restricted by three parameters: a pattern to be passed to the fc-list command, an include regexp specifying which of the matching font files should be retained, and an exclude regexp specifying which files should be removed from the remaining list.
- Parameters:
pattern (str) – A pattern string to pass to the fc-list command. For example, a pattern ‘mono’ will only list monospaced fonts. Multiple elements can be combined with a colon as separator. Example: pattern=’family=DejaVuSans:style=Bold’. An empty string selects all font files.
include (str) – Regex for grep to select the font files to include in the result. If not specified, the pattern from the configuration variable ‘fonts/include’ is used. Example: the default configured include=’.ttf$’ will only return font files with a .ttf suffix. An empty string will include all files selected by the
pattern
.exclude (str) – Regex for grep to select the font files to include in the result. If not specified, the pattern from the configuration variable ‘fonts/include’ is used. Example: the default configured exclude=’Emoji’ will exclude font files that have ‘Emoji’ in their name. An empty string will exclude no files.
- Returns:
list of
Path
– A list of the font files found on the system. If fontconfig is not installed, produces a warning and returns an empty list.
Examples
>>> fonts = listFonts('mono') >>> print(len(fonts) > 0 and fonts[0].is_file()) True
- utils.listMonoFonts()[source]¶
List the monospace fonts found on the system
This is equivalent to
listFonts('mono')
See also
- utils.defaultMonoFont()[source]¶
Return a default monospace font for the system.
- Returns:
Path
– If the configured ‘fonts/default’ has a matching font file on the system, that Path is returned. Else, the first file fromfontList('mono')
is returned.- Raises:
ValuerError – If no monospace font was found on the system
Examples
>>> print(defaultMonoFont()) /...DejaVuSansMono.ttf
- utils.totalMemSize(o, handlers={}, verbose=False)[source]¶
Return the approximate total memory footprint of an object.
This function returns the approximate total memory footprint of an object and all of its contents.
Automatically finds the contents of the following builtin containers and their subclasses: tuple, list, deque, dict, set and frozenset. To search other containers, add handlers to iterate over their contents:
handlers = {SomeContainerClass: iter, OtherContainerClass: OtherContainerClass.get_elements}
Adapted from http://code.activestate.com/recipes/577504/
- utils.disk_usage(paths)[source]¶
Estimate the amount of disk space used by files
- Parameters:
paths (list of path_like) – A list of file or directory paths. Tilde expand is applied on the path names.
- Returns:
dict – A dict where each key is a path name and the value is the estimate disk space used by it. For a directory path, the result is the total recursive size for the directory. Inaccessible paths are silently ignored, so the number of results can be smaller than the paths list. Symbolic links are not followed, except those specified in paths.
Examples
>>> print(disk_usage([pf.cfg['pyformexdir'], '/etc/hosts'])) {'.../pyformex': ..., '/etc/hosts': ...}