49. software — Detecting and checking installed software

A module to help with detecting required software and helper software, and to check the versions of it.

49.1. Classes defined in module software

class software.Version(version)[source]

Version numbering with sane comparison.

Like so many other projects pyFormex has long time been using distutils.SaneVersion to compare version strings with possibly unknown and complex structure. After distutils got removed from Python we have had the source copied into pyFormex for some time. Then I had another look at it and realized that the pyformex.path.hsortkey() function was an ideal candidate to compare version numbers: it was developed to sort file names in a human way, sorting the numerical parts in the the strings in numerical order and the non-numeric parts in lexical order. It even treats dots as a special case.

So here’s a class that treats any string as a version number. It even handles cases where SaneVersion fails, because we guarantee the type of the subsequent components of the splitted string. And we can rebuild the original version string from the components.

A Version instance is a list of alternating str and int components, starting and ending wit a str. Empty strings are inserted where needed. Dots are converted into a -1 int value, so they are sorted before another value.

Examples

>>> v = Version('1.29a3')
>>> v
['', 1, '', -1, '', 29, 'a', 3, '']
>>> print(v)
1.29a3
>>> print(v.pure())
1.29.3
>>> print(v.pure(2))
1.29
>>> v > Version('1.29')
True
>>> v < Version('1.30')
True
>>> v < Version('1.29b')
True
pure(max=3)[source]

Return purely numeric parts

class software.Software(key, *args, **kargs)[source]

Register for software versions.

This class holds a register of the version of installed software. The class is not intended to be used directly, but rather through the derived classes Module and External.

Parameters:

name (str) – The software name as known in pyFormex: this is often the same as the real software name, but can be different if the real software name is complex. We try to use simple lower case names in pyFormex.

Examples

>>> np = Software('numpy')
>>> Software.print_all()
  numpy (** Not Found **)
>>> Software.has('numpy')
''
>>> np.detect('detected')
'detected'
>>> Software.has('numpy')
'detected'
>>> Software.require('numpy')
>>> Software.has('foo')
Traceback (most recent call last):
pyformex.software.RequirementError: foo is not a registered Software
>>> foo = Software('foo')
>>> Software.require('foo')
Traceback (most recent call last):
pyformex.software.RequirementError: Required Software 'foo' (foo) not found
>>> Software.print_all()
  numpy (detected)
  foo (** Not Found **)
>>> Software('foo')
Traceback (most recent call last):
...
ValueError: Software('foo') already exists
property version

Return the version of the software, if installed.

If the software has already been detected, just returns the stored version string. Else, runs the software detect() method and stores and returns the version string.

Returns:

str – The version of the software, or an empty string if the software can not be detected.

detect(version='', fatal=False, quiet=False)[source]

Detect the version of the software.

Parameters:
  • version (str) – The version string to be stored for a detected software. The default empty string means the software was not detected. Derived classes should call this method fromt their detect method and pass a non-empty string for detected softwares.

  • fatal (bool, optional) – If True and the software can not be loaded, a fatal exception is raised. Default is to silently ignore the problem and return an empty version string.

  • quiet (bool, optional) – If True, the whole operation is done silently. The only information about failure will be the returned empty version string.

Returns:

str – The version string of the software, empty if the software can not be loaded.

Notes

As a side effect, the detected version string is stored for later reuse. Thus subsequent tests will not try to re-detect.

classmethod detect_all(names=None)[source]

Detect all registered or required softwares.

Parameters:

names (list of str, optional) – The list of names of registered software components of type clas that need to be detected. If not provided, all the registered components of that class are probed.

Notes

Usually, detection is only performed when needed. This method can however be used to detect a list of requirements or to report a full detection report on all registered softwares.

classmethod detected(names=None, probe=True)[source]

Return the current detection state of softwares and their version.

Parameters:

names (list of str, optional) – The list of names of registered software components of this clas that need to be returned. If not provided, all the registered components of that class are returned.

Returns:

dict – A dict with the software name as key and the detected version as value.

classmethod print_all()[source]

Print the list of registered softwares

classmethod has(name, check=False, fatal=False, quiet=False)[source]

Test if we have the named software available.

Returns a nonzero (version) string if the software is available, or an empty string if it is not.

By default, the software is only checked on the first call. The optional argument check==True forces a new detection.

classmethod check(name, version)[source]

Check that we have a required version of a software.

classmethod require(name, version=None)[source]

Ensure that the named Python software/version is available.

Checks that the specified software is available, and that its version number is not lower than the specified version. If no version is specified, any version is ok.

Returns if the required software/version could be loaded, else an error is raised.

class software.Module(key, *args, **kargs)[source]

Register for Python module version detection rules.

This class holds a register of version detection rules for installed Python modules. Each instance holds the rule for one module, and it is automatically registered at instantiation. The modules used by pyFormex are declared in this module, but users can add their own just by creating a Module instance.

Parameters:
  • name (str) – The module name as known in pyFormex: this is often the same as the Python module name, but can be different if the Python module name is complex. We try to use simple lower case names in pyFormex.

  • modname (str, optional) – The correct Python package.module name. If not provided, it is equal to the pyFormex name.

  • attr (str or tuple of str, optional) – If a str, it is the name of the attribute holding the module version. This should be an attribute of the module modname. The default is ‘__version__’, as it is used by many projects. If the version is not stored in a direct attribute of the same module as used for the detection, then a tuple of strings can be specified, starting with the Python module name in which the version attribute is stored, and a list of subsequent attribute names leading to the version. In this case the first element of the tuple is always a module name. If it is the same as modname, an empty string may be specified. If the final attribute is a callable, it will be called to get the version. The result is always converted to str before being stored as the version.

  • incompatible (tuple of str, optional) – A list of incompatible modules. If any of these modules is loaded, the Module will not be tried. Beware: this should be the actual module names, not the pyFormex component name, which is all lower case.

Examples

>>> Module.dict.clear()
>>> Module.detect_all()
>>> Module.print_all()
>>> np = Module('numpy')
>>> pil = Module('pil', modname='PIL', attr='VERSION')
>>> Module.print_all()
  numpy (...)
  pil (** Not Found **)
>>> Version(Module.has('numpy')) >= Version('1.16')
True
>>> Module.print_all()
  numpy (...)
  pil (** Not Found **)
>>> Module.has('foo')
Traceback (most recent call last):
pyformex.software.RequirementError: foo is not a registered Module
>>> Module.require('foo')
Traceback (most recent call last):
pyformex.software.RequirementError: foo is not a registered Module
>>> foo = Module('foo','FooBar')
>>> Module.has('foo')
''
>>> Module.require('foo')
Traceback (most recent call last):
pyformex.software.RequirementError: Required Module 'foo' (FooBar) not found

Now fake a detection of Module ‘foo’

>>> Module['foo'].version = '1.2.3'
>>> Module.has('foo')
'1.2.3'
>>> Module.require('foo')
>>> Module.require('foo','>= 1.1.7')
>>> Module.require('foo','>= 1.3.0')
Traceback (most recent call last):
pyformex.software.RequirementError: Required version >= 1.3.0      of Module 'foo' (FooBar) not found
detect(fatal=False, quiet=False)[source]

Detect the version of the module.

Parameters:
  • fatal (bool, optional) – If True and the module can not be loaded, a fatal exception is raised. Default is to silently ignore the problem and return an empty version string.

  • quiet (bool, optional) – If True, the whole operation is done silently. The only information about failure will be the returned empty version string.

Returns:

str – The version string of the module, empty if the module can not be loaded.

Notes

As a side effect, the detected version string is stored for later reuse. Thus subsequent tests will not try to re-detect.

class software.External(key, *args, **kargs)[source]

Register for external application version detection rules.

This class holds a register of version detection rules for installed external applications. Each instance holds the rule for one application, and it is automatically registered at instantiation. The applications used by pyFormex are declared in this module, but users can add their own just by creating an External instance.

Parameters:
  • name (str) – The application name as known in pyFormex: this is often the same as the executable name, but can be different if the executable name is complex. We try to use simple lower case names in pyFormex.

  • command (str) – The command to run the application. Usually this includes an option to make the application just report its version and then exit. The command should be directly executable as-is, without invoking a new shell. If a shell is required, it should be made part of the command (see e.g. tetgen). Do not use commands that take a long time to load and run.

  • regex (r-string) – A regular expression that extracts the version from the output of the command. If the application does not have or report a version, any non-empty string is accepted as a positive detection (for example the executable’s name in a bin path). The regex string should contain one set of grouping parentheses, delimiting the part of the output that will be stored as version. If the output of the command does not match, an empty string is stored.

Examples

>>> External.dict.clear()
>>> External.detect_all()
>>> External.print_all()
detect(fatal=False, quiet=False)[source]

Detect the version of the external.

Parameters:
  • fatal (bool, optional) – If True and the external can not be run, a fatal exception is raised. Default is to silently ignore the problem and return an empty version string.

  • quiet (bool, optional) – If True, the whole operation is done silently. The only information about failure will be the returned empty version string.

Returns:

str – The version string of the external, empty if the external can not be run.

Notes

As a side effect, the detected version string is stored for later reuse. Thus subsequent tests will not try to re-detect.

49.2. Functions defined in module software

software.is_namespace(module)[source]

Check if a module is a namespace package

software.del_mod_parents(modname)[source]

Delete a module and its parents

software.listLibraries()[source]

Return a list with the acceleration libraries

software.listShaders()[source]

Return a list with the available GPU shader programs.

Returns:

list – A list of the shader versions available.

Notes

Shader programs are stored in the pyformex/glsl directory and consist of at least of two files: ‘vertex_shader_SHADER.c’ and ‘fragment_shader_SHADER.c’. The SHADER part is the version mnemomic which can be used in the ‘–shader SHADER’ option of the pyformex command.

software.detectSystem()[source]

Detect the system versions

software.detectedSoftware(probe=True, **kargs)[source]

Detect software and system parameters.

Parameters:
  • **kargs

    Keyword arguments specifying which components to detect. Currently, the recognized arguments are:

    • System: detect system software including Python and pyFormex,

    • Module: detect Python modules used by pyFormex,

    • External: detect software used by pyFormex as external commands.

    The value of the parameter specifies which parts of that component are detected. In all cases, a value of ‘all’ means detect all known parts of the component. For ‘Module’ and ‘External’ a list of part names or a dict with part names as keys may be given, and only those parts will be detected. Available part names can be found from Module.dict.keys() or External.dict.keys().

  • probe (bool) – If True, detects all the requested software components. If False, returns the current detection state of the requested softwares.

Returns:

dict – A dict with one or more of the keys ‘System’, ‘Module’ and ‘External’, each having a dict as value:

  • System: contains information about system, pyFormex, Python

  • Module: the detected Python modules

  • External: the detected external programs

software.reportSoftware(soft='all', probe=True, header=None, sort=False, color=False)[source]

Create a report about a software collection

Parameters:
  • soft (dict | 'all' | 'detected') – The software collection to be reported. This is either a dict as returned by detectedSoftware, or ‘all’ to report the already detected software or ‘probe’ to probe all registered software.

  • probe (bool) – If True, detects all the requested software components. If False, reports the current detection state of the requested softwares.

software.compareVersion(has, want)[source]

Check whether a detected version matches the requirements.

Parameters:
  • has (str) – The version that is to be checked

  • wanted (str or list/tuple of str) –

    One or more version strings to which has should be compared. Each version string can be preceded by one of the comparison operators. If no comparison operator is specified, ‘==’ is used. List and tuples are handled recursively and can thus be nested. If a list, the result is True if any of its items gives a match. If a tuple, all items in the tuple must produce a match.

    Note that any tail behind x.y.z version is considered to be later version than x.y.z.

Returns:

bool – The result of the comparison.

Examples

>>> compareVersion('2.7', '2.4.3')
False
>>> compareVersion('2.7', '>2.4.3')
True
>>> compareVersion('2.7', '>= 2.4.3')
True
>>> compareVersion('2.7', '>= 2.7-rc3')
False
>>> compareVersion('2.7-rc4', '>= 2.7-rc3')
True
>>> compareVersion('2.7', ('>= 2.4.3', '< 3.0'))
True
>>> compareVersion('3.5', [('>= 2.4.3', '< 3.0'), '3.5'])
True
software.checkDict(has, want)[source]

Check that software dict has has the versions required in want

software.checkSoftware(req, report=False)[source]

Check that we have the matching components

Returns True or False. If report=True, also returns a string with a full report.

software.registerSoftware(req)[source]

Register the current values of required software

software.formatDict(d, *, indent=2, sort_keys=False, mode='pprint')[source]

Format a possibly nested dict

>>> d = {'a':0, 'd':{'y':'s', 'x':'t'}, 'b':1}
>>> print(formatDict(d, mode='pprint'))
{'a': 0, 'd': {'y': 's', 'x': 't'}, 'b': 1}
>>> print(formatDict(d, mode='json'))
{
  "a": 0,
  "d": {
    "y": "s",
    "x": "t"
  },
  "b": 1
}
>>> print(formatDict(d, mode='python'))
{'a': 0, 'd': {'y': 's', 'x': 't'}, 'b': 1}
>>> d = dict((('a'*i, i) for i in range(8)))
>>> print(d)
{'': 0, 'a': 1, 'aa': 2, 'aaa': 3, 'aaaa': 4, 'aaaaa': 5, 'aaaaaa': 6,    'aaaaaaa': 7}
>>> print(formatDict(d, mode='pprint'))
{ '': 0,
  'a': 1,
  'aa': 2,
  'aaa': 3,
  'aaaa': 4,
  'aaaaa': 5,
  'aaaaaa': 6,
  'aaaaaaa': 7}
software.storeSoftware(soft, fn, mode='json', indent=2)[source]

Store the software collection on file.

The collection is by default stored in JSON format.

software.readSoftware(fn)[source]

Read the software collection from file.

Default mode is ‘python’ because it reads json as well as python. ‘legacy’ can be used to read old software files, though it is recommended to change the files by removing the ‘soft = ‘ at the start.