PyXMake Developer Guide 1.0
PyXMake
Loading...
Searching...
No Matches
PyXMake.Tools.Utility Namespace Reference

Module of basic functions. More...

Classes

class  AbstractBase
 Abstract meta class for all data class objects. More...
 
class  AbstractImport
 Inherited from built-in object. More...
 
class  AbstractMethod
 Class to create 2to3 compatible pickling dictionary. More...
 
class  ChangedWorkingDirectory
 Class to create 2to3 compatible pickling dictionary. More...
 
class  GetDataFromPickle
 Class to create 2to3 compatible pickling dictionary. More...
 
class  UpdateZIP
 Class to create 2to3 compatible pickling dictionary. More...
 

Functions

 autotest (*args, **kwargs)
 
 TemporaryDirectory (default=None)
 
 TemporaryEnvironment (environ={})
 
 ConsoleRedirect (to=os.devnull, stdout=None)
 
 MergedConsoleRedirect (f)
 
 FileOutput (FileName)
 
 FileUpload (url, filename, header={}, **kwargs)
 
 GetHyperlink (s)
 
 GetPyXMakePath ()
 
 GetPlatform ()
 
 GetArchitecture ()
 
 GetLink (path)
 
 GetExecutable (name, get_path=False, **kwargs)
 
 GetBoolean (v)
 
 GetLogger (name=None, **kwargs)
 
 GetRequirements (dirname, args=[], check=False, **kwargs)
 
 GetHostIPAddress ()
 
 GetWSLIPAddress ()
 
 GetSanitizedDataFromCommand (*args, **kwargs)
 
 GetEnvironmentFromCommand (command)
 
 GetDomainNameServer ()
 
 GetOpenAPIGenerator (output=None, **kwargs)
 
 GetBibliographyData (fulltext, verbose=0)
 
 GetTemporaryFileName (arg=None, filename="Temp", extension=".cpd", **kwargs)
 
 GetPathConversion (path, target_os=None, **kwargs)
 
 GetIterableAsList (Iterable)
 
 GetMergedRepositories (RepoID, RepoList, RepoBranch={}, MergeBranch=[], scratch=os.getcwd(), **kwargs)
 
 GetDockerContainer (container, encoding="utf-8")
 
 GetDockerRegistry (registry, port, https_keypath="", https_pemfile="", GUI=False, **kwargs)
 
 GetDockerUI (name="portainer", image="portainer/portainer-ce:latest")
 
 GetDockerRunner (runner="gitlab-runner", restart_policy="always", image="", token="", executor="shell", url='https://gitlab.dlr.de/', tags="docker,linux", ssl_verify=True, **kwargs)
 
 GetDockerPorts (**kwargs)
 
 GetDockerEncoding (*args, **kwargs)
 
 IsDockerContainer ()
 
 InQuotes (s, quote='"')
 
 IsNotEmpty (s)
 
 IsValidISBN (isbn)
 
 IsWSL ()
 
 AsDrive (s, sep=os.path.sep)
 
 Popen (command, verbosity=1, encoding="utf-8", **kwargs)
 
 SSHPopen (ssh_client, command, verbosity=1, **kwargs)
 
 ConvertExcel (excel_file, output=None, **kwargs)
 
 CreateArchive (output, source=None, exclude=[".git",".svn","__pycache__"], **kwargs)
 
 GetResizedImage (image, size, **kwargs)
 
 ConcatenateFiles (filename, files, source=os.getcwd(), ending='')
 
 ReplaceTextinFile (filename, outname, replace, inend='', outend='', source=os.getcwd(), **kwargs)
 
 RemoveDuplicates (ListofStrings, Duplicate="")
 
 DeleteFilesbyEnding (identifier)
 
 DeleteRedundantFolders (Identifier, Except=[], ignore_readonly=False)
 
 AddFunctionToObject (_func, _obj)
 
 PrepareObjectforPickling (_obj)
 
 RecoverDictionaryfromPickling (_dict)
 
 PathLeaf (path)
 
 ArbitraryFlattening (container)
 
 ArbitraryEval (expression)
 
 MoveIO (src, dest, verbose=0)
 
 FileWalk (source, path=os.getcwd())
 
 ClassWalk (reference, cls)
 
 PathWalk (path, startswith=None, endswith=None, contains=None, **kwargs)
 
 ObjectWalk (obj, path=(), memo=None)
 
 ProcessWalk (executable, terminate=True)
 

Variables

 FileNotFoundError = IOError
 
 autotest = Make.Coverage.add
 
 logger = logging.getLogger(__name__)
 Create an alias using default logger for all print statements.
 

Detailed Description

Module of basic functions.

Classes and functions defined for convenience.

@note: PyXMake module                   
Created on 15.07.2016    

@version:  1.0    
----------------------------------------------------------------------------------------------
@requires:
      - 

@change: 
      -    
  
@author: garb_ma                                                     [DLR-FA,STM Braunschweig]
----------------------------------------------------------------------------------------------
Author
Marc Garbade
Date
15.07.2017
Notes/Changes
  • Added documentation // mg 29.03.2018

Function Documentation

◆ AddFunctionToObject()

PyXMake.Tools.Utility.AddFunctionToObject ( _func,
_obj )
Bind a function to an existing object.

Definition at line 1895 of file Utility.py.

◆ ArbitraryEval()

PyXMake.Tools.Utility.ArbitraryEval ( expression)
Evaluate a given expression using ast.literal_eval while treating every string as raw. 

Definition at line 1954 of file Utility.py.

◆ ArbitraryFlattening()

PyXMake.Tools.Utility.ArbitraryFlattening ( container)
Restore the original dictionary by converting Python defaults to their 
numpy equivalents if required  (2to3 compatible).

Definition at line 1941 of file Utility.py.

◆ AsDrive()

PyXMake.Tools.Utility.AsDrive ( s,
sep = os.path.sep )
Return s as drive to start an absolute path with path.join(...).

Definition at line 1555 of file Utility.py.

◆ autotest()

PyXMake.Tools.Utility.autotest ( * args,
** kwargs )

Definition at line 98 of file Utility.py.

◆ ClassWalk()

PyXMake.Tools.Utility.ClassWalk ( reference,
cls )
Recursively find a class object by name and return its object from another class.

@note: Returns None if reference cannot be resolved. Can be safely used with AbstractBase classes and reload.

Definition at line 2002 of file Utility.py.

◆ ConcatenateFiles()

PyXMake.Tools.Utility.ConcatenateFiles ( filename,
files,
source = os.getcwd(),
ending = '' )
Concatenate all files into one.

Definition at line 1791 of file Utility.py.

◆ ConsoleRedirect()

PyXMake.Tools.Utility.ConsoleRedirect ( to = os.devnull,
stdout = None )
Redirect console output to a given file.

Definition at line 471 of file Utility.py.

◆ ConvertExcel()

PyXMake.Tools.Utility.ConvertExcel ( excel_file,
output = None,
** kwargs )
Utility function to convert a given *XLSX file to an *XLS.

Definition at line 1649 of file Utility.py.

◆ CreateArchive()

PyXMake.Tools.Utility.CreateArchive ( output,
source = None,
exclude = [".git",".svn","__pycache__"],
** kwargs )
Create an archive from the given source directory. Defaults to the current project.

@param: output - Absolute output path
@param: source - Source path. Defaults to the current project.
@param: exclude - Files and directories to be ignored

Definition at line 1702 of file Utility.py.

◆ DeleteFilesbyEnding()

PyXMake.Tools.Utility.DeleteFilesbyEnding ( identifier)
Delete all files from workspace

@author: Marc Garbade, 26.02.2018

@param: identifier: A tuple specifying the files to remove.
@type: Tuple

Definition at line 1857 of file Utility.py.

◆ DeleteRedundantFolders()

PyXMake.Tools.Utility.DeleteRedundantFolders ( Identifier,
Except = [],
ignore_readonly = False )
Delete all redundant folders from the current workspace

Definition at line 1870 of file Utility.py.

◆ FileOutput()

PyXMake.Tools.Utility.FileOutput ( FileName)
Redirect outputs to a given file.

Definition at line 539 of file Utility.py.

◆ FileUpload()

PyXMake.Tools.Utility.FileUpload ( url,
filename,
header = {},
** kwargs )
Post a given file as a binary string to a given URL. 

@note: LFS is available if request_toolbelt is installed.

Definition at line 558 of file Utility.py.

◆ FileWalk()

PyXMake.Tools.Utility.FileWalk ( source,
path = os.getcwd() )
Walk recursively through path. Check if all files listed in source are present. 
If True, return them. If False, return all files present in the given path.

Definition at line 1976 of file Utility.py.

◆ GetArchitecture()

PyXMake.Tools.Utility.GetArchitecture ( )
Get the underlying machine architecture. Returns either x86 or x64 which corresponds to 
32 or 64 bit systems.

Definition at line 618 of file Utility.py.

◆ GetBibliographyData()

PyXMake.Tools.Utility.GetBibliographyData ( fulltext,
verbose = 0 )
Perform a full text search using Googles' book API.

Definition at line 992 of file Utility.py.

◆ GetBoolean()

PyXMake.Tools.Utility.GetBoolean ( v)
Inspect the input variable and return a Boolean value if conversion is possible. 

@Original: https://stackoverflow.com/questions/15008758/parsing-boolean-values-with-argparse

Definition at line 695 of file Utility.py.

◆ GetDockerContainer()

PyXMake.Tools.Utility.GetDockerContainer ( container,
encoding = "utf-8" )
Check if a current given container is active and running. Returns boolean value.

@author: garb_ma    
@param: container
@type: string

Definition at line 1192 of file Utility.py.

◆ GetDockerEncoding()

PyXMake.Tools.Utility.GetDockerEncoding ( * args,
** kwargs )
Creates a base64 encoded string of a given username and password combination used for Docker authentifiction.
@author: garb_ma    
@param: username, password
@type: string

Definition at line 1479 of file Utility.py.

◆ GetDockerPorts()

PyXMake.Tools.Utility.GetDockerPorts ( ** kwargs)
Get all active Docker ports. Either from a given container, by a list of containers or all active containers.
Defaults to all active containers. Supports WSL2 with Docker installed.

@author: garb_ma
@param: container
@type: string

Definition at line 1446 of file Utility.py.

◆ GetDockerRegistry()

PyXMake.Tools.Utility.GetDockerRegistry ( registry,
port,
https_keypath = "",
https_pemfile = "",
GUI = False,
** kwargs )
Check if a current given registry is active and running. If not, start a local docker registry at the given port and name.
Optionally, secure connection by a HTTPS certificate (PEM) and alter the base image

@author: garb_ma    
@param: registry
@type: string

Definition at line 1214 of file Utility.py.

◆ GetDockerRunner()

PyXMake.Tools.Utility.GetDockerRunner ( runner = "gitlab-runner",
restart_policy = "always",
image = "",
token = "",
executor = "shell",
url = 'https://gitlab.dlr.de/',
tags = "docker,linux",
ssl_verify = True,
** kwargs )
Check if a current given registry is active and running. If not, start a local docker registry at the given port and name.
Optionally, secure connection by a HTTPS certificate (PEM) and alter the base image

@author: garb_ma    
@param: registry
@type: string

Definition at line 1295 of file Utility.py.

◆ GetDockerUI()

PyXMake.Tools.Utility.GetDockerUI ( name = "portainer",
image = "portainer/portainer-ce:latest" )
Create a custom web UI for docker using Portainer. 

@author: garb_ma    
@param: name
@type: string

Definition at line 1263 of file Utility.py.

◆ GetDomainNameServer()

PyXMake.Tools.Utility.GetDomainNameServer ( )
Return the local DNS IP address.

Definition at line 939 of file Utility.py.

◆ GetEnvironmentFromCommand()

PyXMake.Tools.Utility.GetEnvironmentFromCommand ( command)
Returns the active environment from a process after a given command is executed.

Definition at line 887 of file Utility.py.

◆ GetExecutable()

PyXMake.Tools.Utility.GetExecutable ( name,
get_path = False,
** kwargs )
Check whether name is on PATH and marked as executable.

@author: Six
@note: https://stackoverflow.com/questions/11210104/check-if-a-program-exists-from-a-python-script

Definition at line 660 of file Utility.py.

◆ GetHostIPAddress()

PyXMake.Tools.Utility.GetHostIPAddress ( )
Return the host IP address (IP address of the machine executing this code)

Definition at line 820 of file Utility.py.

◆ GetHyperlink()

PyXMake.Tools.Utility.GetHyperlink ( s)
Get all URLs present in a given string

Definition at line 585 of file Utility.py.

◆ GetIterableAsList()

PyXMake.Tools.Utility.GetIterableAsList ( Iterable)
Walk through an iterable input set and store the results in a list.

Definition at line 1092 of file Utility.py.

◆ GetLink()

PyXMake.Tools.Utility.GetLink ( path)
Return the link target of a symbolic soft link

@note: Supports .lnk files from windows. Returns the target as well as all arguments

Definition at line 629 of file Utility.py.

◆ GetLogger()

PyXMake.Tools.Utility.GetLogger ( name = None,
** kwargs )
Initialize a root logger if no settings prior to loading have been found. 
Otherwise, inherit a logger from supported system logging utilities when available.
Finally, return a reference to the created or already created logger.

Definition at line 712 of file Utility.py.

◆ GetMergedRepositories()

PyXMake.Tools.Utility.GetMergedRepositories ( RepoID,
RepoList,
RepoBranch = {},
MergeBranch = [],
scratch = os.getcwd(),
** kwargs )
Walk through an iterable input set and store the results in a list.

Definition at line 1101 of file Utility.py.

◆ GetOpenAPIGenerator()

PyXMake.Tools.Utility.GetOpenAPIGenerator ( output = None,
** kwargs )
Get the latest version of OpenAPI generator (requires Internet connection).

@author: garb_ma

Definition at line 949 of file Utility.py.

◆ GetPathConversion()

PyXMake.Tools.Utility.GetPathConversion ( path,
target_os = None,
** kwargs )
Return the given absolute path in its Linux/Windows counter part. 

Definition at line 1057 of file Utility.py.

◆ GetPlatform()

PyXMake.Tools.Utility.GetPlatform ( )
Get the underlying machine platform in lower cases.

Definition at line 612 of file Utility.py.

◆ GetPyXMakePath()

PyXMake.Tools.Utility.GetPyXMakePath ( )
Get the PyXMake path from *__init__.

Definition at line 605 of file Utility.py.

◆ GetRequirements()

PyXMake.Tools.Utility.GetRequirements ( dirname,
args = [],
check = False,
** kwargs )
Create a list of required sub-packages for a given Python project (given as full directory path).

@author: garb_ma

Definition at line 763 of file Utility.py.

◆ GetResizedImage()

PyXMake.Tools.Utility.GetResizedImage ( image,
size,
** kwargs )
Resize a given image (absolute path) and returns a resized image object.

@param: source - Source path. Must be an absolute path to the icon
@param: size - Single integer value representing the new icon size

Definition at line 1752 of file Utility.py.

◆ GetSanitizedDataFromCommand()

PyXMake.Tools.Utility.GetSanitizedDataFromCommand ( * args,
** kwargs )
Returns platform independent paths extracted from a given command or list

Definition at line 859 of file Utility.py.

◆ GetTemporaryFileName()

PyXMake.Tools.Utility.GetTemporaryFileName ( arg = None,
filename = "Temp",
extension = ".cpd",
** kwargs )
Create a temporary file name with extension *.cpd by default. Optional argument: Seed for random number generation.

Definition at line 1039 of file Utility.py.

◆ GetWSLIPAddress()

PyXMake.Tools.Utility.GetWSLIPAddress ( )
Return the WSL IP address (IP address of the machine executing this code)

Definition at line 836 of file Utility.py.

◆ InQuotes()

PyXMake.Tools.Utility.InQuotes ( s,
quote = '"' )
Return the given string in quotes.

Definition at line 1508 of file Utility.py.

◆ IsDockerContainer()

PyXMake.Tools.Utility.IsDockerContainer ( )
Check whether current package lives inside a docker container.

Definition at line 1500 of file Utility.py.

◆ IsNotEmpty()

PyXMake.Tools.Utility.IsNotEmpty ( s)
Check whether a string is empty and/or not given. Returns True otherwise.

Definition at line 1515 of file Utility.py.

◆ IsValidISBN()

PyXMake.Tools.Utility.IsValidISBN ( isbn)
Check whether a given string corresponds to a valid ISBN address.

Definition at line 1522 of file Utility.py.

◆ IsWSL()

PyXMake.Tools.Utility.IsWSL ( )
Detect if the script is running inside WSL or WSL2 on windows.

@note: WSL is thought to be the only common Linux kernel with Microsoft in the name, per Microsoft:
https://github.com/microsoft/WSL/issues/4071#issuecomment-496715404

@author: https://www.scivision.dev/python-detect-wsl/

Definition at line 1544 of file Utility.py.

◆ MergedConsoleRedirect()

PyXMake.Tools.Utility.MergedConsoleRedirect ( f)
Redirect all console outputs to a given stream

Definition at line 530 of file Utility.py.

◆ MoveIO()

PyXMake.Tools.Utility.MoveIO ( src,
dest,
verbose = 0 )
Move given src to defined dest while waiting for completion of the process.

Definition at line 1962 of file Utility.py.

◆ ObjectWalk()

PyXMake.Tools.Utility.ObjectWalk ( obj,
path = (),
memo = None )
Walk recursively through nested python objects.

@author: Yaniv Aknin, 13.12.2011

@param: obj, path, memo
@type: object, list, boolean

Definition at line 2045 of file Utility.py.

◆ PathLeaf()

PyXMake.Tools.Utility.PathLeaf ( path)
Return the last item of an arbitrary path (its leaf).

Definition at line 1934 of file Utility.py.

◆ PathWalk()

PyXMake.Tools.Utility.PathWalk ( path,
startswith = None,
endswith = None,
contains = None,
** kwargs )
Walk recursively through path. Exclude both folders and files if requested. 

Definition at line 2017 of file Utility.py.

◆ Popen()

PyXMake.Tools.Utility.Popen ( command,
verbosity = 1,
encoding = "utf-8",
** kwargs )
Run command line string "command" in a separate subprocess. 
Show output in current console window in dependence of verbosity level:
- 0 --> Quiet
- 1 --> Only show errors
- 2 --> Show every command line output.

@author: garb_ma    
@param: command, verbosity
@type: string, integer

Definition at line 1567 of file Utility.py.

◆ PrepareObjectforPickling()

PyXMake.Tools.Utility.PrepareObjectforPickling ( _obj)
Prepare a object for pickling and convert all numpy  
arrays to python defaults (2to3 compatible). 

Definition at line 1902 of file Utility.py.

◆ ProcessWalk()

PyXMake.Tools.Utility.ProcessWalk ( executable,
terminate = True )
Walk through all active processes on the host machine, 

@author: garb_ma

Definition at line 2074 of file Utility.py.

◆ RecoverDictionaryfromPickling()

PyXMake.Tools.Utility.RecoverDictionaryfromPickling ( _dict)
Restore the original dictionary by converting python defaults to their 
numpy equivalents if required  (2to3 compatible).

Definition at line 1919 of file Utility.py.

◆ RemoveDuplicates()

PyXMake.Tools.Utility.RemoveDuplicates ( ListofStrings,
Duplicate = "" )
Remove all duplicates in a list of strings

Definition at line 1833 of file Utility.py.

◆ ReplaceTextinFile()

PyXMake.Tools.Utility.ReplaceTextinFile ( filename,
outname,
replace,
inend = '',
outend = '',
source = os.getcwd(),
** kwargs )
Replace all occurrences of replace in filename.

Definition at line 1811 of file Utility.py.

◆ SSHPopen()

PyXMake.Tools.Utility.SSHPopen ( ssh_client,
command,
verbosity = 1,
** kwargs )
Run command line string "command" in a separate SSH client process. 
Show output in current console window in dependence of verbosity level:
- 0 --> Quiet
- 1 --> Only show errors
- 2 --> Show every command line output.

@author: garb_ma    
@param: command, verbosity
@type: string, integer

Definition at line 1614 of file Utility.py.

◆ TemporaryDirectory()

PyXMake.Tools.Utility.TemporaryDirectory ( default = None)
Create a temporary dictionary for use with the "with" statement. Its content is deleted after execution.

@param: default
@type: default: string

Definition at line 430 of file Utility.py.

◆ TemporaryEnvironment()

PyXMake.Tools.Utility.TemporaryEnvironment ( environ = {})
Temporarily set process environment variables.

Definition at line 459 of file Utility.py.

Variable Documentation

◆ autotest

PyXMake.Tools.Utility.autotest = Make.Coverage.add

Definition at line 96 of file Utility.py.

◆ FileNotFoundError

PyXMake.Tools.Utility.FileNotFoundError = IOError

Definition at line 49 of file Utility.py.

◆ logger

PyXMake.Tools.Utility.logger = logging.getLogger(__name__)

Create an alias using default logger for all print statements.

Definition at line 103 of file Utility.py.