Decorators

Introduction

Bionic decorators are Python decorators designed to be used in conjunction with a FlowBuilder. They modify the way functions are incorporated into flows.

The normal way (without decorators) of incorporating functions into flows is as follows:

import bionic as bn

builder = FlowBuilder('my_flow')

builder.assign('x', 1)

@builder
def x_plus_one(x):
    return x + 1

print(builder.build().get('x_plus_one'))  # Prints "2".

In the simple case above, the function is interpreted as a new entity named x_plus_one which depends on the existing entity x. However, in many cases we want Bionic to process the function in a more complex way. In these cases we can add additional decorators:

import bionic as bn

builder = FlowBuilder('my_flow')

builder.assign('x', 1)

@builder
@bn.outputs('x_plus_one', 'x_plus_two')
@bn.persist(False)
def some_function(x):
    return (x + 1), (x + 2)

print(builder.build().get('x_plus_one'))  # Prints "2".
print(builder.build().get('x_plus_two'))  # Prints "3".

These decorators tell Bionic that our function actually generates two values for two different entities (x_plus_one and x_plus_two), and these values should not be persisted to disk.

All Bionic decorators should be placed after the initial @builder decorator, but before any regular (non-Bionic) decorators. Finally, the @builder decorator returns the original function, so it can be called normally, as if it had been defined without any of the Bionic decorators. E.g.:

@builder
@bn.persist(False)
def f(x):
    return x + 1

assert f(7) == 8

Built-In Decorators

bionic.run_in_aip(machine, worker_count=None, worker_machine=None)[source]

Indicates that the decorated function should be computed in AIP. This decorator requires AIP based distributed execution to be enabled, which can be done by setting core__aip_execution__enabled core entity.

This decorator is currently experimental and does not have any additional user-facing documentation. It may change in non-backwards-compatible ways.

Parameters

machine (String) – The machine type that should be used to compute the function on AIP.
worker_count (String, optional) – The number of workers that should be used to compute the function on AIP.
worker_machine (String, optional) – The machine type that should be used by the worker nodes.

Returns

A decorator which can be applied to an entity function.

Return type

Function

bionic.changes_per_run(enabled=None)[source]

Indicates whether this function is non-deterministic: i.e., if it’s called multiple times with the same inputs, can it return different outputs?

When enabled is true, Bionic will recompute this function’s value (and potentially the values of anything depending on it) each time this flow is instantiated, rather than reusing a value cached on disk. For example, if the function queries data from an external database, the results may be different each time even if the query stays the same.

However, for practical reasons, Bionic won’t compute a new value more than once within a single run. That is, once it’s been computed for a particular Flow instance, that value will be saved in memory and reused. This is a compromise: logically it makes sense to recompute it every time, but it’s much simpler to have a single fixed value for each entity within a given flow instance. For this reason, when this decorator is enabled, memoization must not be disabled for this entity.

Note that @changes_per_run is not the same as @persist(False). For example, the following code will not necessarily query the database each time:

@builder
@bn.persist(False)
@builder
def current_data():
    return download_data()

@builder
def summary(current_data):
    return summarize(current_data)

This fails because if we call flow.get('summary') and Bionic finds a cached value, it will return the cached value because it doesn’t know that current_data ought to be recomputed. On the other hand, if we replace persist(False) with changes_per_run – as in the example below – then current_data will be recomputed each time (and summary will be recomputed if current_data changes).

Parameters: enabled (Boolean, optional (default True)) – Whether this function’s output changes per run.
Returns: A decorator which can be applied to an entity function.
Return type: Function

Example usage:

@builder
@bn.changes_per_run
@builder
def current_data():
    return download_data()

@builder
def summary(current_data):
    return summarize(current_data)

bionic.docs(*docs)[source]

Assigns documentation strings to the entities defined by the decorated function. Typically used in conjuction with @outputs for functions that return multiple entity values. (In the more common case where your function returns a single entity value, you can just use a regular Python docstring.)

Parameters: docs (Sequence of strings) – Documentation strings for each of the defined entities.
Returns: A decorator which can be applied to an entity function.
Return type: Function

bionic.gather(over, also=None, into='gather_df')[source]

Gathers multiple instances of entities into a single dataframe.

Gathers all values of the over entity (or entities) along with associated values of the also entity (or entities) into a single dataframe, which is provided to the decorated function as an argument whose name is determined by into.

Parameters

over (String or sequence of strings) – Primary names to collect. Any cases that differ only in these names will be grouped together in the same frame.
also (String or sequence of strings) – Secondary names to include. These entity values are added to the frame but don’t affect the grouping.
into (String, optional (default 'gather_df')) – The argument name of the gathered frame.

Returns

A decorator which can be applied to a entity function.

Return type

Function

Example usage:

builder = FlowBuilder('my_flow')

builder.assign('color', values=['red', 'blue'])
builder.assign('shape', values=['square', 'circle'])

@builder
def colored_shape(color, shape):
    return color + ' ' + shape

@builder
@gather('color', 'colored_shape', 'df')
def all_color_shapes(df):
    return ', '.join(df.colored_shape.sort_values())

flow = builder.build()

flow.get('colored_shape', set)
# Returns {'red square', 'blue square', 'red circle', 'blue circle'}

flow.get('all_color_shapes', set)
# Returns {'blue square, red square', 'blue circle, red circle'}
# Note that the colored shapes are gathered into two groups: within
# each group, the color varies but the shape does not.

bionic.immediate(func)

Guarantees that an entity can be computed during bootstrap resolution.

Currently @immediate is equivalent to @persist(False).

bionic.memoize(enabled)[source]

Indicates whether computed values should be cached in memory. Overrides the value of core__memoize_by_default when set.

Parameters: enabled (Boolean) – Whether this entity’s values should be memoized.
Returns: A decorator which can be applied to an entity function.
Return type: Function

bionic.output(name)[source]

Renames an entity. The entity function must have a single value.

When this is used to decorate an entity function, the provided name is used as the entity name, instead of using the function’s name.

Parameters: name (String) – The new name for the entity.
Returns: A decorator which can be applied to an entity function.
Return type: Function

bionic.outputs(*names)[source]

Indicates that a result produces a (fixed-size) collection of values, and assigns a name to each value.

When this is used to decorate an entity function, the function will actually define multiple entities, one for each provided name. The decorated function must return a sequence with exactly as many values as the provided list of names.

Any other decorators which would normally modify the definition of the entity (such as protocols) will be applied to each of the final entities.

Parameters: names (Sequence of strings) – The names of the defined entities.
Returns: A decorator which can be applied to an entity function.
Return type: Function

bionic.persist(enabled)[source]

Indicates whether computed values should be cached persistently. Overrides the value of core__persist_by_default when set.

Parameters: enabled (Boolean) – Whether this entity’s values should be persisted (e.g., to local disk).
Returns: A decorator which can be applied to an entity function.
Return type: Function

bionic.pyplot(name=None, savefig_kwargs=None)[source]

Provides a Matplotlib pyplot module to the decorated entity function.

By default the module is provided as an argument named "pyplot", but this can be changed with the name argument. The entity’s Python function should use the pyplot module to create a plot, but should not return any values. The output of the final entity will be a Pillow.Image containing the plot.

Parameters

name (String, optional (default "pyplot")) – The argument name of the module provided to the decorated function.
savefig_kwargs (Dict, optional) – Additional arguments to pass to matplotlib.pytplot.savefig when converting the plot to an image. By default, passes format=png and bbox_inches="tight"; any arguments passed in this dict will override the default values.

Returns

A decorator which can be applied to an entity function.

Return type

Function

bionic.version(major=None, minor=None, ignore_bytecode=None, suppress_bytecode_warnings=None)[source]

Identifies the version of a Python function. The version has two components: a major version and a minor version. Each of these can be either an integer or a string, and defaults to 0.

When you change the implementation of an entity function, you should update its version so that Bionic knows to whether invalidate any cached values of that function and re-compute them. An update in major version indicates a functional change: Bionic will assume that the function can return different output and won’t use any cached artifacts created by a previous version of the function. An update in minor version indicates a nonfunctional change, such as a refactoring or performance optimization: Bionic will assume that the function behaves the same for all inputs, and will continue using cached artifacts as long as the major version still matches. Updating the minor version is only required when using Bionic’s “assisted versioning” mode.

You may also want to update the major version when there are changes in functions or libraries that the entity function calls, or in any external data source (like a database) that the function accesses.

Parameters

major (Integer or string (default 0)) – An arbitrary identifier for a function’s behavior.
minor (Integer or string (default 0)) – An arbitrary identifier for a function’s nonfunctional characteristics.
ignore_bytecode (Boolean (default False)) – Whether this entity’s bytecode should be ignored.
suppress_bytecode_warnings (Boolean (default False)) – Whether warnings from this entity’s bytecode analysis should be ignored.

Returns

A decorator which can be applied to an entity function.

Return type

Function