wasp_general package

Subpackages

Submodules

wasp_general.cache module

class wasp_general.cache.WCacheStorage[source]

Bases: object

Abstract class for cache storage

class CacheEntry(has_value=False, cached_value=None)[source]

Bases: object

Cache request result, is used in WCacheStorage.get_cache() to determine if there is a cached value and what that value is.

exception CacheMissedException[source]

Bases: Exception

Exception is raised in WCacheStorage.get_result() and derived methods as an error for accessing cache record that doesn’t exist

clear(decorated_function=None)[source]

Remove results from this storage

Parameters:decorated_function – if specified, then results will be removed for this function only
Returns:None
get_cache(decorated_function, *args, **kwargs)[source]

Get cache entry (WCacheStorage.CacheEntry) for the specified arguments

Parameters:
  • decorated_function – called function (original)
  • args – args with which function is called
  • kwargs – kwargs with which function is called
Returns:

WCacheStorage.CacheEntry
get_result(decorated_function, *args, **kwargs)[source]

Get result from storage for specified function. Will raise an exception (WCacheStorage.CacheMissedException) if there is no cached result.

Parameters:
  • decorated_function – called function (original)
  • args – args with which function is called
  • kwargs – kwargs with which function is called
Returns:

(any type, even None)
has(decorated_function, *args, **kwargs)[source]

Check if there is a result for given function

Parameters:
  • decorated_function – called function (original)
  • args – args with which function is called
  • kwargs – kwargs with which function is called
Returns:

None
put(result, decorated_function, *args, **kwargs)[source]

Save (or replace) result for given function

Parameters:
  • result – result to be saved
  • decorated_function – called function (original)
  • args – args with which function is called
  • kwargs – kwargs with which function is called
Returns:

None
class wasp_general.cache.WGlobalSingletonCacheStorage[source]

Bases: wasp_general.cache.WCacheStorage

Simple storage that acts as global singleton. Result (singleton) is saved on the very first call. It doesn’t matter with which parameters function was called, result will be the same for all the rest calls.

clear(decorated_function=None)[source]

WCacheStorage.clear() method implementation

get_cache(decorated_function, *args, **kwargs)[source]

WCacheStorage.get_cache() method implementation

get_result(decorated_function, *args, **kwargs)[source]

WCacheStorage.get_result() method implementation

has(decorated_function, *args, **kwargs)[source]

WCacheStorage.has() method implementation

put(result, decorated_function, *args, **kwargs)[source]

WCacheStorage.put() method implementation

class wasp_general.cache.WInstanceSingletonCacheStorage(cache_record_cls=None, statistic=False)[source]

Bases: wasp_general.cache.WCacheStorage

This storage acts similar to WGlobalSingletonCacheStorage storage, but works with bounded methods only (class methods or object method). For every object it keeps results with “cache-record” class (WInstanceSingletonCacheStorage.InstanceCacheRecord), this class (is used by default) saves the very first result and returns it every time. For example, by default if we have two object derived from the same class, and the same method is called, then this storage will keep two separate results, one for each instance.

Exact behaviour can be tweaked through WInstanceSingletonCacheStorage.InstanceCacheRecord inheritance. So derived “cache-records” classes can do things in there own way, they may save every called result, or may not save anything.

This class was extended to support internal statistics with cache hits and misses. Still, this class is not thread safe, but accessing statistics from a separate thread should work. Statistics is calculated for records that was fetch through WInstanceSingletonCacheStorage.get_cache() method only

Note:This implementation uses weakrefs, so memory leak doesn’t happen (here).
class InstanceCacheRecord(result, decorated_function)[source]

Bases: object

Class is used to save cached results for the specified method and for the single instance. This class saves the very first result only. This class uses WCacheStorage.CacheEntry the same way as WCacheStorage storage does - it help to determine, whether there is a cached value or not.

Because derived class constructor signature may differ from this class constructor signature, then in order to create cache record there should be a unified method, which is WInstanceSingletonCacheStorage.InstanceCacheRecord.create()

cache_entry(*args, **kwargs)[source]

Return cache entry for the specified arguments

Parameters:
  • args – args with which bounded method was called
  • kwargs – kwargs with which bounded method was called
Returns:

WCacheStorage.CacheEntry
classmethod create(result, decorated_function, *args, **kwargs)[source]

Create new “cache-record” for the specified arguments

Parameters:
  • result – result to keep
  • decorated_function – called bounded method
  • args – args with which bounded method was called
  • kwargs – kwargs with which bounded method was called
Returns:

WInstanceSingletonCacheStorage.InstanceCacheRecord
decorated_function()[source]

Return original method

Returns:bounded method
update(result, *args, **kwargs)[source]

Update (or add other one) result, that was generated with specified arguments

Parameters:
  • result – result to keep
  • args – args with which bounded method was called
  • kwargs – kwargs with which bounded method was called
Returns:

None
cache_hit()[source]

Return cache hits (return None if class was constructed without ‘statistic’ flag)

Returns:int or None
cache_missed()[source]

Return cache misses (return None if class was constructed without ‘statistic’ flag)

Returns:int or None
clear(decorated_function=None)[source]

WCacheStorage.clear() method implementation (Clears statistics also)

get_cache(decorated_function, *args, **kwargs)[source]

WCacheStorage.get_cache() method implementation

put(result, decorated_function, *args, **kwargs)[source]

WCacheStorage.put() method implementation

wasp_general.cache.cache_control(validator=None, storage=None)[source]

Decorator that is used for caching result.

Parameters:
  • validator – function, that has following signature (decorated_function, *args, **kwargs), where decorated_function - original function, args - function arguments, kwargs - function keyword arguments. This function must return True if cache is valid (old result must be use if it there is one), or False - to generate and to store new result. So function that always return True can be used as singleton. And function that always return False won’t cache anything at all. By default (if no validator is specified), it presumes that cache is always valid.
  • storage – storage that is used for caching results. see WCacheStorage class.
Returns:

decorated function

wasp_general.composer module

class wasp_general.composer.WClassComposer(*composite_keys, constructor=None)[source]

Bases: wasp_general.composer.WCompositeComposer

class ClassConstructor(basic_cls, create_obj_fn=None)[source]

Bases: wasp_general.composer.InstanceConstructor

basic_cls()[source]
create_obj(construction_keys)[source]
class ClassKey(key, basic_composer, has_key_fn=None, get_key_fn=None, set_key_fn=None, required=False)[source]

Bases: wasp_general.composer.CompositeKey

get_key(obj)[source]
has_key(obj)[source]
set_key(obj, value)[source]
class GetterKey(key, basic_composer, required=False)[source]

Bases: wasp_general.composer.ClassKey

get_key(obj)[source]
set_key(obj, value)[source]
add_composite_key(composite_key)[source]
basic_cls()[source]
class wasp_general.composer.WComposerFactory(*entries, name_field=None, value_field=None)[source]

Bases: wasp_general.composer.WComposerProto

class Entry(composer, name=None)[source]

Bases: object

composer()[source]
name()[source]
add_entry(entry)[source]
compose(obj_spec)[source]
decompose(obj)[source]
entries()[source]
name_field()[source]
value_field()[source]
class wasp_general.composer.WComposerProto[source]

Bases: object

compose(obj_spec)[source]
decompose(obj)[source]
class wasp_general.composer.WCompositeComposer(*composite_keys, constructor=None)[source]

Bases: wasp_general.composer.WComposerProto

class CompositeKey(key, basic_composer, required=False)[source]

Bases: wasp_general.composer.WProxyComposer

compose(key_value)[source]
decompose(key_value)[source]
get_key(obj)[source]
has_key(obj)[source]
key()[source]
required()[source]
set_key(obj, value)[source]
class ConstructionKeys(*key_instances)[source]

Bases: object

add(key_instance)[source]
has_key(key)[source]
key_instances()[source]
class InstanceConstructor[source]

Bases: object

construct_obj(construction_keys)[source]
create_obj(construction_keys)[source]
class KeyInstance(composite_key, value)[source]

Bases: object

add_composite_key(composite_key)[source]
compose(obj_spec)[source]
constructor()[source]
decompose(obj)[source]
optional_keys()[source]
required_keys()[source]
class wasp_general.composer.WDictComposer(*composite_keys)[source]

Bases: wasp_general.composer.WCompositeComposer

class DictConstructor[source]

Bases: wasp_general.composer.InstanceConstructor

create_obj(construction_keys)[source]
class DictKey(key, basic_composer, required=False)[source]

Bases: wasp_general.composer.CompositeKey

get_key(obj)[source]
has_key(obj)[source]
set_key(obj, value)[source]
add_composite_key(composite_key)[source]
class wasp_general.composer.WIterComposer(basic_composer)[source]

Bases: wasp_general.composer.WProxyComposer

compose(obj_spec)[source]
decompose(obj)[source]
class wasp_general.composer.WPlainComposer(strict_cls=None, permit_none=False)[source]

Bases: wasp_general.composer.WComposerProto

compose(obj_spec)[source]
decompose(obj)[source]
permit_none()[source]
strict_cls()[source]
class wasp_general.composer.WProxyComposer(basic_composer)[source]

Bases: wasp_general.composer.WComposerProto

basic_composer()[source]

wasp_general.config module

class wasp_general.config.WConfig(defaults=None, dict_type=<class 'collections.OrderedDict'>, allow_no_value=False, *, delimiters=('=', ':'), comment_prefixes=('#', ';'), inline_comment_prefixes=None, strict=True, empty_lines_in_values=True, default_section='DEFAULT', interpolation=<object object>, converters=<object object>)[source]

Bases: configparser.ConfigParser

Improved ConfigParser. Has single method to merge config data (see WConfig.merge() method) and method, that split coma-separated option value (see WConfig.split_option() method).

merge(config)[source]

Load configuration from given configuration.

Parameters:config – config to load. If config is a string type, then it’s treated as .ini filename
Returns:None
merge_section(config, section_to, section_from=None)[source]

Load configuration section from other configuration. If specified section doesn’t exist in current configuration, then it will be added automatically.

Parameters:
  • config – source configuration
  • section_to – destination section name
  • section_from – source section name (if it is None, then section_to is used as source section name)
Returns:

None
split_option(section, option)[source]

Return list of strings that are made by splitting coma-separated option value. Method returns empty list if option value is empty string

Parameters:
  • section – option section name
  • option – option name
Returns:

list of strings

wasp_general.csv module

class wasp_general.csv.WCSVExporter(output_obj, titles=True, **csv_fmtparams)[source]

Bases: object

csv_fmtparams()[source]
export(dict_record)[source]
export_titles(dict_record)[source]
omit_field(field_name)[source]
omitted_fields()[source]
output_obj()[source]
titles()[source]

wasp_general.datetime module

wasp_general.datetime.local_datetime(dt=None, utc_value=True)[source]

Convert UTC datetime and/or datetime without timezone information to local datetime with timezone information

Parameters:
  • dt – datetime in UTC to convert. If is None, then system datetime value is used
  • utc_value – whether dt is a datetime in UTC or in system timezone without timezone information
Returns:

datetime for system (local) timezone with tz set
wasp_general.datetime.local_tz()[source]

Return current system timezone shift from UTC

Returns:datetime.timezone
wasp_general.datetime.utc_datetime(dt=None, local_value=True)[source]

Convert local datetime and/or datetime without timezone information to UTC datetime with timezone information.

Parameters:
  • dt – local datetime to convert. If is None, then system datetime value is used
  • local_value – whether dt is a datetime in system timezone or UTC datetime without timezone information
Returns:

datetime in UTC with tz set

wasp_general.io module

class wasp_general.io.WAESWriter(raw, cipher)[source]

Bases: _io.BufferedWriter

File-like writer with transparent encryption

flush()[source]
write(b)[source]

Encrypt and write data

Parameters:b – data to encrypt and write
Returns:None
class wasp_general.io.WBufferedIOReader(raw)[source]

Bases: _io.BufferedReader

classmethod append_buffer(buffer, data)[source]
classmethod create_buffer()[source]
read(size=-1)[source]
read_chunk(size)[source]
writable(*args, **kwargs)[source]
class wasp_general.io.WBzip2Reader(raw)[source]

Bases: wasp_general.io.WBufferedIOReader

close(*args, **kwargs)[source]
read_chunk(size)[source]
class wasp_general.io.WDiscardReaderResult(raw)[source]

Bases: wasp_general.io.WBufferedIOReader

classmethod append_buffer(buffer, data)[source]
class wasp_general.io.WDiscardWriterResult[source]

Bases: _io.BufferedWriter

write(b)[source]
class wasp_general.io.WGzipReader(raw)[source]

Bases: wasp_general.io.WBufferedIOReader

close(*args, **kwargs)[source]
read_chunk(size)[source]
class wasp_general.io.WHashCalculationReader(raw, hash_name)[source]

Bases: wasp_general.io.WBufferedIOReader, wasp_general.io.WHashIO

read_chunk(size)[source]
class wasp_general.io.WHashCalculationWriter(raw, hash_name)[source]

Bases: _io.BufferedWriter, wasp_general.io.WHashIO

write(b)[source]
class wasp_general.io.WHashIO(hash_name)[source]

Bases: object

hash_name()[source]
hexdigest()[source]
update_hash(b)[source]
class wasp_general.io.WIOChain(last_io_obj, *links)[source]

Bases: object

first_io()[source]
instance(io_cls)[source]

Bases: object

io_cls()[source]
io_obj(raw)[source]
class wasp_general.io.WIOCounter[source]

Bases: object

bytes_processed()[source]
increase_counter(processed_bytes)[source]
rate()[source]
reset()[source]
start_counter()[source]
stop_counter()[source]
class wasp_general.io.WReaderChain(last_io_obj, *links)[source]

Bases: wasp_general.io.WIOChain, _io.BufferedReader

close()[source]

Bases: wasp_general.io.WIOChainLink

class wasp_general.io.WResponsiveIO(stop_event)[source]

Bases: object

exception IOTerminated[source]

Bases: Exception

stop_event()[source]
class wasp_general.io.WResponsiveReader(raw, stop_event)[source]

Bases: wasp_general.io.WBufferedIOReader, wasp_general.io.WResponsiveIO

read_chunk(size)[source]
class wasp_general.io.WResponsiveWriter(raw, stop_event)[source]

Bases: _io.BufferedWriter, wasp_general.io.WResponsiveIO

write(b)[source]
class wasp_general.io.WThrottlingIO(throttling_to=None, maximum_timeout=None)[source]

Bases: wasp_general.io.WIOCounter

check_rate()[source]
maximum_timeout()[source]
throttling_to()[source]
class wasp_general.io.WThrottlingReader(raw, throttling_to=None, maximum_timeout=None)[source]

Bases: wasp_general.io.WBufferedIOReader, wasp_general.io.WThrottlingIO

close(*args, **kwargs)[source]
read_chunk(size)[source]
class wasp_general.io.WThrottlingWriter(raw, throttling_to=None, maximum_timeout=None)[source]

Bases: _io.BufferedWriter, wasp_general.io.WThrottlingIO

close(*args, **kwargs)[source]
write(b)[source]
class wasp_general.io.WWriterChain(last_io_obj, *links)[source]

Bases: wasp_general.io.WIOChain, _io.BufferedWriter

close()[source]
flush()[source]

Bases: wasp_general.io.WIOChainLink

wasp_general.mime module

wasp_general.mime.mime_type(filename)[source]

Guess mime type for the given file name

Note: this implementation uses python_magic package which is not thread-safe, as a workaround global lock is used for the ability to work in threaded environment

Parameters:filename – file name to guess
Returns:str

wasp_general.template module

class wasp_general.template.WMakoTemplate(template)[source]

Bases: wasp_general.template.WTemplate

template()[source]
class wasp_general.template.WTemplate[source]

Bases: object

template()[source]
class wasp_general.template.WTemplateFile(template_filename, **kwargs)[source]

Bases: wasp_general.template.WTemplate

template()[source]
class wasp_general.template.WTemplateLookup(template_id, template_collection)[source]

Bases: wasp_general.template.WTemplate

template()[source]
class wasp_general.template.WTemplateRenderer(template, context=None)[source]

Bases: object

context()[source]
render(**context_vars)[source]
template()[source]
update_context(**context_vars)[source]
class wasp_general.template.WTemplateText(text_template, **kwargs)[source]

Bases: wasp_general.template.WTemplate

template()[source]

wasp_general.thread module

class wasp_general.thread.WCriticalResource[source]

Bases: object

static critical_section(blocking=True, timeout=None, raise_exception=True)[source]
thread_lock()[source]
exception wasp_general.thread.WCriticalSectionError[source]

Bases: Exception

wasp_general.thread.critical_section_dynamic_lock(lock_fn=None, blocking=True, timeout=None, raise_exception=True)[source]
wasp_general.thread.critical_section_lock(lock=None, blocking=True, timeout=None, raise_exception=True)[source]

wasp_general.uri module

class wasp_general.uri.WURI(**components)[source]

Bases: object

class Component[source]

Bases: enum.Enum

An enumeration.

fragment = 'fragment'
hostname = 'hostname'
password = 'password'
path = 'path'
port = 'port'
query = 'query'
scheme = 'scheme'
username = 'username'
component(component, value=None)[source]
classmethod parse(uri)[source]

wasp_general.verify module

class wasp_general.verify.SubclassVerifier(*tags, env_var=None, silent_checks=False)[source]

Bases: wasp_general.verify.Verifier

Verifier that is used for type verification. Checks parameter if it is class or subclass of specified class or classes. Specification accepts type or list/tuple/set of types

Example:

@verify_subclass(c=A, e=(A, D))
def foo(a, b, c, d=None, **kwargs):
        pass
check(type_spec, arg_name, decorated_function)[source]

Return callable that checks function parameter for class validity. Checks parameter if it is class or subclass of specified class or classes

Parameters:
  • type_spec – type or list/tuple/set of types
  • arg_name – function parameter name
  • decorated_function – target function
Returns:

function
class wasp_general.verify.TypeVerifier(*tags, env_var=None, silent_checks=False)[source]

Bases: wasp_general.verify.Verifier

Verifier that is used for type verification. Checks parameter if it is instance of specified class or classes. Specification accepts type or list/tuple/set of types

Example:

@verify_type(a=int, b=str, d=(int, None), e=float)
def foo(a, b, c, d=None, **kwargs):
        pass
check(type_spec, arg_name, decorated_function)[source]

Return callable that checks function parameter for type validity. Checks parameter if it is instance of specified class or classes

Parameters:
  • type_spec – type or list/tuple/set of types
  • arg_name – function parameter name
  • decorated_function – target function
Returns:

function
class wasp_general.verify.ValueVerifier(*tags, env_var=None, silent_checks=False)[source]

Bases: wasp_general.verify.Verifier

Verifier that is used for value verification. Checks parameter if its value passes specified restrictions. Specification accepts function or list/tuple/set of functions. Each function must accept one parameter and must return True or False if it passed restrictions or not.

Example:

@verify_value(a=(lambda x: x > 5, lambda x: x < 10))
@verify_value(c=lambda x: x.a == 'foo', d=lambda x: x is None or x < 0)
def foo(a, b, c, d=None, **kwargs):
pass
check(value_spec, arg_name, decorated_function)[source]

Return callable that checks function parameter for value validity. Checks parameter if its value passes specified restrictions.

Parameters:
  • value_spec – function or list/tuple/set of functions. Each function must accept one parameter and must return True or False if it passed restrictions or not.
  • arg_name – function parameter name
  • decorated_function – target function
Returns:

function
class wasp_general.verify.Verifier(*tags, env_var=None, silent_checks=False)[source]

Bases: object

Base class for verifier implementation.

Verifiers are classes, that generates decorators (which are later used for runtime function arguments checking). Derived classes (such as TypeVerifier, SubclassVerifier and ValueVerifier) check arguments for type and/or value validity. But each derived class uses its own syntax for check declaration (see Verifier.check()).

Same checks can be grouped into one sentence if they are used for different arguments. Each statement can be marked by tag or tags for runtime disabling. If statement doesn’t have tag then it always run checks.

Checks can be simple (implemented by lambda-statements) or complex (implemented by standalone functions or classes). Because target function is decorated for checking it is possible to run checks sequentially.

Example:

@verify_type(a=int, b=str, d=(int, None), e=float)
@verify_subclass(c=A, args=B)
@verify_value(a=(lambda x: x > 5, lambda x: x < 10), args=lambda x: x > 0)
@verify_value(c=lambda x: x.a == 'foo', d=lambda x: x is None or x < 0)
def foo(a, b, c, d=None, *args, **kwargs):
        pass
check(arg_spec, arg_name, decorated_function)[source]

Return callable object that takes single value - future argument. This callable must raise an exception if error occurs. It is recommended to return None if everything is OK

Parameters:
  • arg_spec – specification that is used to describe check like types, lambda-functions, list of types just anything (see Verifier.decorator())
  • arg_name – parameter name from decorated function
  • decorated_function – target function (function to decorate)
Returns:

None
decorate_disabled()[source]

Return True if this decoration must be omitted, otherwise - False. This class searches for tags values in environment variable (Verifier.__default_environment_var__), Derived class can implement any logic

Returns:bool
decorator(**arg_specs)[source]

Return decorator that can decorate target function

Parameters:arg_specs – dictionary where keys are parameters name and values are theirs specification. Specific specification is passed as is to Verifier.check() method with corresponding parameter name.
Returns:function
static function_name(fn)[source]

Return function name in pretty style

Parameters:fn – source function
Returns:str
help_info(exc, decorated_function, arg_name, arg_spec)[source]

Print debug information to stderr. (Do nothing if object was constructed with silent_checks=True)

Parameters:
  • exc – raised exception
  • decorated_function – target function (function to decorate)
  • arg_name – function parameter name
  • arg_spec – function parameter specification
Returns:

None
wasp_general.verify.verify_subclass(*tags, **type_kwargs)[source]

Shortcut for SubclassVerifier

Parameters:
  • tags – verification tags. See Verifier.__init__()
  • type_kwargs – verifier specification. See SubclassVerifier.check()
Returns:

decorator (function)
wasp_general.verify.verify_type(*tags, **type_kwargs)[source]

Shortcut for TypeVerifier

Parameters:
  • tags – verification tags. See Verifier.__init__()
  • type_kwargs – verifier specification. See TypeVerifier.check()
Returns:

decorator (function)
wasp_general.verify.verify_value(*tags, **type_kwargs)[source]

Shortcut for ValueVerifier

Parameters:
  • tags – verification tags. See Verifier.__init__()
  • type_kwargs – verifier specification. See ValueVerifier.check()
Returns:

decorator (function)

wasp_general.version module

wasp_general.version.package_version()[source]

Module contents