Output¶
Note
Output modules submit data to external services.
Output module properties:
- They have a protocol encoder method mapped to
wishbone.module.OutputModule.encode()
in order to convert the desiredwishbone.event.Event
payload into the desired format prior to submitting it to the external service. - Should always provide a
selection
,payload
,native_events
andparallel_streams
module parameter. Ifpayload
is notNone
, then it takes precendence overselection
.Selection
defines the event key to submit whilst template comes up with a string to submit.``payload`` usually makes no sense with bulk events. - Should use
wishbone.module.OutputModule.getDataToSubmit()
to retrieve the actual data to submit to the external service.This automatically takes care of bulk events. - Through inheriting
wishbone.module.OutputModule
Output modules overridewishbone.actor.Actor._consumer()
with their own version which executes the registeredfunction
in parallel greenthreads by using a threadpool. The module’sparallel_streams
parameter defines the size of the pool and therefor the number of parallel greenthreads submitting the event data externally.It depends on the nature of your output protocol whether this makes sense.Normally you shouldn’t really bother with this as long a Gevent’s monkey patching works on the code you’re using to speak to the remote service. - If you’re setting a default encoder function make sure you use
wishbone.module.OutputModule.setEncoder()
as this method will prevent overwrite any user defined encoder set viawishbone.actorconfig.ActorConfig
.
Warning
Be aware that if parallel_streams
is larger than 1, the equal amount
of events will be processed concurrently by the function registered with
wishbone.actor.Actor.registerConsumer()
to consume the queue. Within
that function do NOT change shared (module) variables but only use
local (to the function) ones.
The builtin Wishbone Output modules:
Name | Description |
---|---|
wishbone.module.output.null |
Purges events. |
wishbone.module.output.stdout |
Prints event data to STDOUT. |
wishbone.module.output.syslog |
Submits event data to syslog. |
Output
modules must base wishbone.module.OutputModule
:
-
class
wishbone.module.
OutputModule
(config)[source]¶ Bases:
wishbone.actor.Actor
-
generateEvent
(data={}, destination=None)¶ Generates a new event.
This function can get overridden by
wishbone.module.InputModule._generateNativeEvent
.The provided
data
will be traversed in search of valid templates which then will be rendered.Parameters: Returns: An event containing
data
as a payload.Return type:
-
getDataToSubmit
(event)[source]¶ Derives the data to submit from
event
taking into accountnative_events
,payload
andselection
module parameters.Parameters: event¶ – The event to extract data from. Returns: The data to submit. Return type: dict/str/…
-
loop
()¶ The global lock for this module.
Returns: True when module is in running mode. False if not. Return type: bool
-
postHook
()¶ Is executed when module exits.
-
preHook
()¶ Is executed when module starts. Can be overriden by the user.
-
registerConsumer
(function, queue)¶ Registers <function> to process all events in <queue>
Don’t not trap errors here. When <function> fails then the event will be submitted to the “failed” queue, If <function> succeeds to the success queue.
Registering
function
to consumequeue
will also apply all the registered module functions against the events consumed from it.Parameters: Returns: None
-
renderEventKwargs
(event, queue=None)¶ Renders kwargs using the content of
event
and stores the result underevent.kwargs
.Parameters: Returns: The provided event instance.
Return type: wishbone.event.Event
-
renderKwargs
()¶ Renders kwargs without making use of event content. This is typically used when initiliazing a module and render the defined kwargs which do not need a event data for rendering.
Returns: None
-
sendToBackground
(function, *args, **kwargs)¶ Executes a function and sends it to the background. Such a function should never exit until
self.loop
returnsFalse
. This method wrapsfunction
again in a loop as longself.loop
returnsFalse
so thatfunction
is restarted and an error is logged.Parameters:
-
setEncoder
(name, *args, **kwargs)[source]¶ Sets the encoder with name <name> unless there’s already an encoder defined via
actorconfig.ActorConfig
.Parameters: Returns: - True if the encoder is set, False when an encoder was already
set via
actorconfig.ActorConfig
Return type: Bool
-
start
()¶ Starts the module.
Returns: None
-
stop
()¶ Makes
self.loop
returnFalse
and handles shutdown of of the registered background jobs.
-