PyREM package¶
pyrem.task module¶
task.py: Contains the main unit of execution in PyREM, the task.
-
class
pyrem.task.
Task
¶ Bases:
object
Abstract class, the main unit of execution in PyREM.
If you would like to define your own type of
Task
, you should at least implement the_start
,_wait
,_stop
, and_reset
methods.Every task that gets started will be stopped on Python exit, as long as that exit can be caught by the
atexit
module (e.g. pressing Ctrl+C will be caught, but sending SIGKILL will not be caught).-
return_values
¶ dict
Subclasses of
Task
should store all of their results in this field and document what the possible return values are.
-
start
(wait=False)¶ Start a task.
This function depends on the underlying implementation of _start, which any subclass of
Task
should implement.Parameters: wait (bool) – Whether or not to wait on the task to finish before returning from this function. Default False. Raises: RuntimeError
– If the task has already been started without a subsequent call toreset()
.
-
wait
()¶ Wait on a task to finish and stop it when it has finished.
Raises: RuntimeError
– If the task hasn’t been started or has already been stopped.Returns: The return_values
of the task.
-
stop
()¶ Stop a task immediately.
Raises: RuntimeError
– If the task hasn’t been started or has already been stopped.
-
reset
()¶ Reset a task.
Allows a task to be started again, clears the
return_values
.Raises: RuntimeError
– If the task has not been stopped.
-
-
class
pyrem.task.
SubprocessTask
(command, quiet=False, return_output=False, shell=False, require_success=False)¶ Bases:
pyrem.task.Task
A task to run a command as a subprocess on the local host.
This process will be killed when this task is stopped. The return code of the process will be stored in
return_values['retcode']
.Parameters: - command (list of str) – The command to execute. Each command-line
argument and flag should be a separate element in the command
list unless
shell = True
. - quiet (bool) – If True, the output of this command is not printed. Default False.
- return_output (bool) –
If True, the output of this command will be saved in
return_values['stdout']
andreturn_values['stderr']
when the subprocess is allowed to finish (i.e. when it is waited on instead of being stopped). Default False.quiet and return_output shouldn’t both be true.
- shell (bool) – If True, allocate a shell to execute the process.
See:
subprocess.Popen
. Default False. - require_success (bool) – If True and if this task is waited on instead
of being stopped, raises a
RuntimeError
if the subprocess has a return code other than 0. Default False.
- command (list of str) – The command to execute. Each command-line
argument and flag should be a separate element in the command
list unless
-
class
pyrem.task.
RemoteTask
(host, command, quiet=False, return_output=False, kill_remote=True)¶ Bases:
pyrem.task.SubprocessTask
A task to run a command on a remote host over ssh.
Any processes started on the remote host will be killed when this task is stopped (unless kill_remote=False is specified).
return_values['retcode']
will contain the return code of the ssh command, which should currently be ignored.-
host
¶ str
The name of the host the task will run on.
Parameters: - host (str) – The host to run on.
- command (list of str) – The command to execute.
- quiet (bool) – See
SubprocessTask
. - return_output (bool) – See
SubprocessTask
. - kill_remote (bool) – If True, all processes started on the remote server will be killed when this task is stopped.
-
-
class
pyrem.task.
Parallel
(tasks)¶ Bases:
pyrem.task.Task
A task that executes several given tasks in parallel.
Currently does not capture the return_values of the underlying tasks, this will be fixed in the future.
Parameters: tasks (list of Task
) – Tasks to execute.
-
class
pyrem.task.
Sequential
(tasks)¶ Bases:
pyrem.task.Task
A tasks that executes several given tasks in sequence.
Currently does not capture the return_values of the underlying tasks, this will be fixed in the future.
Parameters: tasks (list of Task
) – Tasks to execute.
pyrem.host module¶
host.py: Contains classes for managing remote hosts.
The Host
object is a simple wrapper around various sorts of Tasks.
-
class
pyrem.host.
Host
(hostname)¶ Bases:
object
Abstract class, an object representing some host.
-
hostname
¶ str
The name of the host.
-
run
(command, **kwargs)¶ Build a task to run the command on a remote host.
Parameters: - command (list of str) – The command to execute.
- **kwargs – Keyword args to be passed to the underlying
Task
‘s init method.
Returns: The resulting task.
Return type: pyrem.task.Task
-
-
class
pyrem.host.
RemoteHost
(hostname)¶ Bases:
pyrem.host.Host
A remote host.
Parameters: hostname (str) – The hostname of the remote host. -
run
(command, **kwargs)¶ Run a command on the remote host.
This is just a wrapper around
RemoteTask(self.hostname, ...)
-
send_file
(file_name, remote_destination=None, **kwargs)¶ Send a file to a remote host with rsync.
Parameters: - file_name (str) – The relative location of the file on the local host.
- remote_destination (str) – The destination for the file on the remote host. If None, will be assumed to be the same as file_name. Default None.
- **kwargs – Passed to
SubprocessTask
‘s init method.
Returns: The resulting task.
Return type: pyrem.task.SubprocessTask
-
get_file
(file_name, local_destination=None, **kwargs)¶ Get a file from a remote host with rsync.
Parameters: - file_name (str) – The relative location of the file on the remote host.
- local_destination (str) – The destination for the file on the local host. If None, will be assumed to be the same as file_name. Default None.
- **kwargs – Passed to
SubprocessTask
‘s init method.
Returns: The resulting task.
Return type: pyrem.task.SubprocessTask
-
-
class
pyrem.host.
LocalHost
¶ Bases:
pyrem.host.Host
The local host.
-
run
(command, **kwargs)¶
-
move_file
(file_name, destination, **kwargs)¶ Move a file on the local host.
Parameters: - file_name (str) – The relative location of the file.
- destination (str) – The relative destination of the file.
- **kwargs – Passed to
SubprocessTask
‘s init method.
Returns: The resulting task.
Return type: pyrem.task.SubprocessTask
-
pyrem.utils module¶
utils.py: Contains useful utilities to be used in other modules.
-
pyrem.utils.
synchronized
(func)¶ Function decorator to make function synchronized on
self._lock
.If the first argument to the function (hopefully self) does not have a _lock attribute, then this decorator does nothing.