wasp_general.task package

Submodules

wasp_general.task.base module

class wasp_general.task.base.WStoppableTask[source]

Bases: wasp_general.task.base.WTask

Task that can be stopped (graceful shutdown)

stop()[source]

Stop this task (graceful shutdown)

Returns:None
class wasp_general.task.base.WSyncTask[source]

Bases: wasp_general.task.base.WStoppableTask

This class is some kind of declaration, that the following task is executed in foreground.

stop()[source]

Stop this task. This implementation does nothing.

Returns:None
class wasp_general.task.base.WTask[source]

Bases: object

Basic task prototype. Must implement the only thing - to start

start()[source]

Start this task

Returns:None
class wasp_general.task.base.WTerminatableTask[source]

Bases: wasp_general.task.base.WStoppableTask

Task that can be terminated (rough shutdown)

terminate()[source]

Terminate this task (rough shutdown)

Returns:None

wasp_general.task.dependency module

class wasp_general.task.dependency.WDependentTask(name, bases, namespace)[source]

Bases: wasp_general.task.registry.WRegisteredTask

Metaclass for dependent tasks. It is used for automatic resolving required dependencies (starting required tasks). Derived classes must be able to be constructed with constructor without arguments or they must override WDependentTask.start_dependent_task() method.

If derived class inherits wasp_general.task.base.WStoppableTask class, then it could be stopped (automatically stopped via registry class, such as WTaskDependencyRegistry)

When __registry_tag__ is set to None, related task registry must have ‘__skip_none_registry_tag__’ flag set to False, otherwise - exception is raised. Such task (that have __registry_tag__ set to None) will be excluded from registry.

In order to include task to registry, task must define __registry_tag__, which has to be a str class.

start_dependent_task()[source]

Start this task and return its instance

Returns:WTask
class wasp_general.task.dependency.WTaskDependencyRegistry[source]

Bases: wasp_general.task.registry.WTaskRegistry

Registry for the WDependentTask classes. Registry storage must be

Derived classes must redefine __registry_storage__ property (which has to be WTaskDependencyRegistryStorage instance). (see WTaskRegistry.__registry_storage__)

classmethod all_stop()[source]

Stop every task started within this registry

Returns:None
classmethod registry_storage()[source]

Get registry storage

Returns:WTaskDependencyRegistryStorage
classmethod start_task(task_tag, skip_unresolved=False)[source]

Start task from registry

Parameters:
Returns:

None
classmethod stop_task(task_tag, stop_dependent=True, stop_requirements=False)[source]

Stop started task from registry

Parameters:
Returns:

None
class wasp_general.task.dependency.WTaskDependencyRegistryStorage[source]

Bases: wasp_general.task.registry.WTaskRegistryStorage

Storage that is used to store WDependentTask task.

add(task_cls)[source]

Add task to this storage. Multiple tasks with the same tag are not allowed

Parameters:task_cls – task to add
Returns:None
dependency_check(task_cls, skip_unresolved=False)[source]

Check dependency of task for irresolvable conflicts (like task to task mutual dependency)

Parameters:
  • task_cls – task to check
  • skip_unresolved – flag controls this method behaviour for tasks that could not be found. When False, method will raise an exception if task tag was set in dependency and the related task wasn’t found in registry. When True that unresolvable task will be omitted
Returns:

None
start_task(task_tag, skip_unresolved=False)[source]

Check dependency for the given task_tag and start task. For dependency checking see WTaskDependencyRegistryStorage.dependency_check(). If task is already started then it must be stopped before it will be started again.

Parameters:
  • task_tag – task to start. Any required dependencies will be started automatically.
  • skip_unresolved – flag controls this method behaviour for tasks that could not be found. When False, method will raise an exception if task tag was set in dependency and the related task wasn’t found in registry. When True that unresolvable task will be omitted
Returns:

None
started_tasks(task_registry_id=None, task_cls=None)[source]

Return tasks that was started. Result way be filtered by the given arguments.

Parameters:
  • task_registry_id – if it is specified, then try to return single task which id is the same as this value.
  • task_cls – if it is specified then result will be consists of this subclass only
Returns:

None or WTask or tuple of WTask
stop_task(task_tag, stop_dependent=True, stop_requirements=False)[source]

Stop task with the given task tag. If task already stopped, then nothing happens.

Parameters:
  • task_tag – task to stop
  • stop_dependent – if True, then every task, that require the given task as dependency, will be stopped before.
  • stop_requirements – if True, then every task, that is required as dependency for the given task, will be stopped after.
Returns:

None

wasp_general.task.health module

class wasp_general.task.health.WMeasurableTaskHealthSensor(name, severity, error_value, description=None, comparator=None)[source]

Bases: wasp_general.task.health.WTaskHealthSensor

Represent measurable sensor

error_value()[source]

Return error_value

Returns:(any type)
healthy()[source]

Return True if sensor is OK, else - False

Returns:bool
measurement_unit()[source]

Return measurement unit of sensor

Returns:string
value()[source]

Return current value sensor

Returns:(any type)
class wasp_general.task.health.WTaskHealth(*sensors, decorate_start=True, decorate_stop=True)[source]

Bases: object

This class presents API for task health probe. It can be used to probe if everything that task needs are ready or available (for example: disk space, file existing, socket is opened and so on).

This is optional for the most tasks.

healthy()[source]

Return task health. If None - task is healthy, otherwise - maximum severity of sensors

Returns:None or WTaskHealthSensor.WTaskSensorSeverity
sensor(sensor_name)[source]

Return sensor by its name

Parameters:sensor_name – name of sensor
Returns:WMeasurableTaskHealthSensor instance
sensors()[source]

Return list of sensor names

Returns:list of strings
class wasp_general.task.health.WTaskHealthSensor(name, severity, description=None)[source]

Bases: object

Represent single sensor

class WTaskSensorSeverity[source]

Bases: enum.Enum

Sensor severity

critical = 2
important = 1
recommended = 0
description()[source]

Return sensor description

Returns:string (or None if there is no description)
healthy()[source]

Return True if sensor is OK, else - False

Returns:bool
name()[source]

Return sensor name

Returns:string
severity()[source]

Return sensor severity

Returns:WTaskHealthSensor.WTaskSensorSeverity

wasp_general.task.registry module

class wasp_general.task.registry.WRegisteredTask(name, bases, namespace)[source]

Bases: abc.ABCMeta

Metaclass for task, that is stored in registry. Derived class must redefine __registry__ property (and __registry_tag__ depends on registry storage - see WTaskRegistryStorage.__multiple_tasks_per_tag__)

class wasp_general.task.registry.WTaskRegistry[source]

Bases: object

Basic task registry. Derived classes must redefine __registry_storage__ property (see WTaskRegistry.__registry_storage__)

classmethod add(task_cls)[source]

Add task class to storage

Parameters:task_cls – task to add
Returns:None
classmethod clear()[source]

Remove every task from storage

Returns:None
classmethod registry_storage()[source]

Get registry storage

Returns:WTaskRegistryBase
classmethod remove(task_cls)[source]

Remove task class to storage

Parameters:task_cls – task to remove
Returns:None
class wasp_general.task.registry.WTaskRegistryBase[source]

Bases: object

Prototype for registry storage. Derived class must redefined following methods

add(task)[source]

Add task to storage

Parameters:task – task to add (WTask class with WRegisteredTask metaclass)
Returns:None
clear()[source]

Remove every task from storage

Returns:None
count()[source]

Registered task count

Returns:int
remove(task)[source]

Remove task from storage

Parameters:task – task to remove (WTask class with WRegisteredTask metaclass)
Returns:None
tags()[source]

Return available registry tags

Returns:tuple of str
tasks(task_cls=None)[source]

Return tasks that was added to this registry

Parameters:task_cls – if it is not None, then result will be consist of this subclass only (useful fo

filtering tasks)

Returns:tuple of WRegisteredTask
tasks_by_tag(registry_tag)[source]

Get tasks from registry by its tag

Parameters:registry_tag – any hash-able object
Returns:Return task or list of tasks
class wasp_general.task.registry.WTaskRegistryStorage[source]

Bases: wasp_general.task.registry.WTaskRegistryBase

Simple registry storage implementation

add(task_cls)[source]

Add task to this storage. Depends on WTaskRegistryStorage.__multiple_tasks_per_tag__ tasks with the same __registry_tag__ can be treated as error.

Parameters:task_cls – task to add
Returns:None
clear()[source]

Removes every task from storage

Returns:None
count()[source]

Registered task count

Returns:int
remove(task_cls)[source]

Remove task from the storage. If task class are stored multiple times (if WTaskRegistryStorage.__multiple_tasks_per_tag__ is True) - removes all of them.

Parameters:task_cls – task to remove
Returns:None
tags()[source]

WTaskRegistryBase.tags() implementation

tasks(task_cls=None)[source]

WTaskRegistryBase.tasks() implementation

tasks_by_tag(registry_tag)[source]

Get tasks from registry by its tag

Parameters:registry_tag – any hash-able object
Returns:Return task (if WTaskRegistryStorage.__multiple_tasks_per_tag__ is not True) or list of tasks

wasp_general.task.thread module

class wasp_general.task.thread.WPollingThreadTask(thread_name=None, thread_join_timeout=None, polling_timeout=None)[source]

Bases: wasp_general.task.thread.WThreadTask

Create task, that will be executed in a separate thread, and will wait for stop event or ready event and till that will do small piece of work. This threaded task will be constructed with ‘join_on_stop’ and ‘ready_to_stop’ flags turned on

Polling timeout is a timeout after which next call for a small piece of work will be done. Real WPollingThreadTask.__polling_iteration() method implementation must be fast (faster then joining timeout), so it must do small piece of work each time only. It is crucial to do that, because busy thread can be terminated at any time, and so can not be finalized gracefully.

If one thread spawns other threads it is obvious to stop them from the same thread they are generated. And at this point, wrong joining and polling timeouts could break start-stop mechanics. So parent thread should have joining timeout not less then children threads have (it is better to have joining timeout greater then children timeout). And polling timeout should be not greater (as little as possible) then children threads have

polling_timeout()[source]

Task polling timeout

Returns:int or float
thread_started()[source]

Start polling for a stop event or ready event and do small work via WPollingThreadTask._polling_iteration() method call

Returns:None
class wasp_general.task.thread.WThreadCustomTask(task, thread_name=None, join_on_stop=True, ready_to_stop=False, thread_join_timeout=None)[source]

Bases: wasp_general.task.thread.WThreadTask

Class that can run any task in a separate thread. It just wraps start method, and for a WStoppableTask object it wraps stop method also. So for a WThreadTask class task, this object will create new thread “inside” new thread. Because of this, it is important that appropriate flags was set within constructor

task()[source]

Return original task

Returns:WTask
thread_started()[source]

Start original task

Returns:None
thread_stopped()[source]

If original task is WStoppableTask object, then stop it

Returns:None
exception wasp_general.task.thread.WThreadJoiningTimeoutError[source]

Bases: Exception

Exception is raised when thread joining timeout is expired

class wasp_general.task.thread.WThreadTask(thread_name=None, join_on_stop=True, ready_to_stop=False, thread_join_timeout=None)[source]

Bases: wasp_general.task.base.WStoppableTask

Task that runs in a separate thread. Since there is no right way in Python to stop or terminate neighbor thread, so it’s highly important for derived classes to be capable to stop correctly.

This class implements WTask.start() method by creating new thread. Thread that is call WTask.thread_started() method.

check_events()[source]

Check “stopping”-events (‘ready_event’, ‘stop_event’, ‘exception_event’) if one of them is set. Usually True value means that thread is meant to be stopped, means that it is finished its job or some error has happened or this thread was asked to stop

Returns:bool
close_thread()[source]

Clear all object descriptors for stopped task. Task must be joined prior to calling this method.

Returns:None
exception_event()[source]

Return event which is set if exception is raised inside thread function

Returns:Event
join_timeout()[source]

Return task join timeout

Returns:int or float
ready_event()[source]

Return readiness event object. Event will be available if object was constructed with ready_to_stop flag

Returns:Event or None
start()[source]

WStoppableTask.start() implementation that creates new thread

start_event()[source]

Return event which is set after the thread creation. Shows that a separate thread has been created already

Returns:Event
stop()[source]

WStoppableTask.stop() implementation that sets stop even (if available), calls WStoppableTask.threaded_stopped() and cleans up thread (if configured)

stop_event()[source]

Return stop event object. Event will be available if object was constructed with join_on_stop flag

Returns:Event or None
thread()[source]

Return current Thread object (or None if task wasn’t started)

Returns:Thread or None
thread_exception(raised_exception)[source]

Callback for handling exception, that are raised inside WThreadTask.thread_started()

Parameters:raised_exception – raised exception
Returns:None
thread_name()[source]

Return thread name with which this thread is or will be created

Returns:str
thread_started()[source]

Real task that do all the work :return: None

thread_stopped()[source]

Method is called when task is about to stop (is called before joining process). This method is called whenever exception was raised or not

Returns:None
class wasp_general.task.thread.WThreadedTaskChain(*threaded_task_chain, thread_name=None, thread_join_timeout=None, polling_timeout=None)[source]

Bases: wasp_general.task.thread.WPollingThreadTask

Threaded task, that executes given tasks sequentially

thread_stopped()[source]

WThreadTask._polling_iteration() implementation

wasp_general.task.thread_tracker module

class wasp_general.task.thread_tracker.WScheduleRecordTracker(task, policy=None, task_group_id=None, on_drop=None, on_wait=None, track_wait=True, track_drop=True)[source]

Bases: wasp_general.task.scheduler.proto.WScheduleRecord

Schedule record that may register scheduler events related to a scheduled task like ‘postponing’ event and ‘drop’ event.

Note:Since there is an extra work that should be done, this class may be inappropriate for low-latency

situation. Also, registering events should be done quickly because of this task joining timeout.

task_dropped()[source]

Track (if required) drop event and do the same job as WScheduleRecord.task_dropped() method do

Returns:None
task_postponed()[source]

Track (if required) postponing event and do the same job as WScheduleRecord.task_postponed() method do

Returns:None
track_drop()[source]

Return True if this task is tracking “drop” event otherwise - False

Returns:bool
track_wait()[source]

Return True if this task is tracking “postponing” event otherwise - False

Returns:bool
class wasp_general.task.thread_tracker.WSimpleTrackerStorage(records_limit=None, record_start=True, record_stop=True, record_termination=True, record_exception=True, record_wait=True, record_drop=True)[source]

Bases: wasp_general.thread.WCriticalResource, wasp_general.task.thread_tracker.WThreadTrackerInfoStorageProto

Simple WThreadTrackerInfoStorageProto implementation which stores events in a operation memory

class ExceptionRecord(task, exception, exception_details, event_details=None)[source]

Bases: wasp_general.task.thread_tracker.Record

Record for unhandled exception

class Record(record_type, thread_task, event_details=None)[source]

Bases: object

General record of single event

last_record(task_uid, *requested_events)[source]

Search over registered WScheduleTask instances and return the last record that matches search criteria.

Parameters:
  • task_uid – uid of WScheduleTask instance
  • requested_events – target events types
Returns:

WSimpleTrackerStorage.Record or None
record_drop()[source]

Return True if this storage is saving dropping events, otherwise - False

Returns:bool
record_exception()[source]

Return True if this storage is saving unhandled exceptions events, otherwise - False

Returns:bool
record_limit()[source]

Return maximum number of records to keep

Returns:int or None (for no limit)
record_start()[source]

Return True if this storage is saving start events, otherwise - False

Returns:bool
record_stop()[source]

Return True if this storage is saving normal stop events, otherwise - False

Returns:bool
record_termination()[source]

Return True if this storage is saving termination stop events, otherwise - False

Returns:bool
record_wait()[source]

Return True if this storage is saving postponing events, otherwise - False

Returns:bool
register_drop(task, event_details=None)[source]

WSimpleTrackerStorage.register_drop() method implementation

register_exception(task, raised_exception, exception_details, event_details=None)[source]

WSimpleTrackerStorage.register_exception() method implementation

register_start(task, event_details=None)[source]

WSimpleTrackerStorage.register_start() method implementation

register_stop(task, event_details=None)[source]

WSimpleTrackerStorage.register_stop() method implementation

register_termination(task, event_details=None)[source]

WSimpleTrackerStorage.register_termination() method implementation

register_wait(task, event_details=None)[source]

WSimpleTrackerStorage.register_wait() method implementation

class wasp_general.task.thread_tracker.WThreadTracker(tracker_storage=None, thread_name=None, join_on_stop=True, thread_join_timeout=None, track_start=True, track_stop=True, track_termination=True, track_exception=True)[source]

Bases: wasp_general.task.thread.WThreadTask

Threaded task that may register its events (start event, normal stop fact, task termination fact, unhandled exceptions)

Note:Since there is an extra work that should be done, this class may be inappropriate for low-latency

situation. Also, registering events should be done quickly because of this task joining timeout.

event_details(event)[source]

Return task details that should be registered with a tracker storage

Parameters:event – source event that requested details
Returns:str or None
start()[source]

WThreadTask.start() implementation. Register (if required) start event by a tracker storage

Returns:None
thread_exception(raised_exception)[source]

WThreadTask.thread_exception() implementation. Register (if required) unhandled exception event by a tracker storage

Parameters:raised_exception – unhandled exception
Returns:None
thread_stopped()[source]

WThreadTask.thread_stopped() implementation. Register (if required) stop and termination event by a tracker storage

Returns:None
thread_tracker_exception(raised_exception)[source]

Method is called whenever an exception is raised during registering a event

Parameters:raised_exception – raised exception
Returns:None
track_exception()[source]

Return True if this task is tracking unhandled exception event otherwise - False

Returns:bool
track_start()[source]

Return True if this task is tracking “start-event” otherwise - False

Returns:bool
track_stop()[source]

Return True if this task is tracking “stop-event” otherwise - False

Returns:bool
track_termination()[source]

Return True if this task is tracking “termination-event” otherwise - False

Returns:bool
tracker_storage()[source]

Return linked storage

Returns:WThreadTrackerInfoStorageProto or None
class wasp_general.task.thread_tracker.WThreadTrackerInfoStorageProto[source]

Bases: object

Prototype for a storage that keeps thread task events like start event, normal stop, termination, raised unhandled exceptions) or scheduler events

register_drop(task, event_details=None)[source]

Store event of task drop (this event is used in a scheduler classes)

Parameters:
  • task – task that was dropped
  • event_details – (optional) event details - any kind of data related to the given task and

event of task drop

Returns:None
register_exception(task, raised_exception, exception_details, event_details=None)[source]

Store exception event

Parameters:
  • task – task that was terminated by unhandled exception
  • raised_exception – unhandled exception
  • exception_details – any kind of data related to the raised exception
  • event_details – (optional) event details - any kind of data related to the given task and

exception event

Returns:None
register_start(task, event_details=None)[source]

Store start event

Parameters:
  • task – task that is starting
  • event_details – (optional) event details - any kind of data related to the given task and start event
Returns:

None
register_stop(task, event_details=None)[source]

Store stop event

Parameters:
  • task – task that stopped
  • event_details – (optional) event details - any kind of data related to the given task and stop event
Returns:

None
register_termination(task, event_details=None)[source]

Store termination event

Parameters:
  • task – task that was terminated
  • event_details – (optional) event details - any kind of data related to the given task and termination event
Returns:

None
register_wait(task, event_details=None)[source]

Store event of task postponing (this event is used in a scheduler classes)

Parameters:
  • task – task that was postponed
  • event_details – (optional) event details - any kind of data related to the given task and

postponing event

Returns:None
class wasp_general.task.thread_tracker.WTrackerEvents[source]

Bases: enum.Enum

Possible tracking events

drop = 6
exception = 4
start = 1
stop = 2
termination = 3
wait = 5

Module contents