Release Notes¶
This page attempts to document all non-trivial user-visible changes in each Bionic release. (“Non-trivial” means, for example, we won’t include fixes to typos in documentation.)
Versioning Scheme¶
The Bionic project follows a modified version of Semantic Versioning (“SemVer”), using a MAJOR.MINOR.PATCH version number. We try to avoid backwards-incompatible changes (even though SemVer doesn’t require this until version 1.0). However, since Bionic is still in a fairly early stage, we’d like to retain the option of making breaking changes to rarely-used features of Bionic. Thus, our policy is not to make any backwards-incompatible change before version 1 unless we’ve discussed it with known Bionic users beforehand. Currently all known Bionic users work at Square – if you’re not at Square but would like to be included in this group, let us know on our issue tracker.
We also tend to release new functionality with PATCH releases. (In SemVer this would happen in MINOR releases). We tend to issue a MINOR release when we’ve accumulated a significant amount of new functionality; when we make a breaking change (as described above); or when we make a Cache Compatibility Change (as described below).
Once we reach 1.0, we plan to follow SemVer exactly.
Types of Changes¶
For each release, we list the following types of change (in this order):
New Bugs: Notable bugs (i.e., failure to behave the way the documentation says it should) introduced by this release.
Breaking Changes: Any other non-backwards-compatible changes in behavior. Includes any difference in any returned value, thrown exception class, or value passed to a user-provided function. Does not include performance changes; bug fixes; changes in log output, error messages, or warnings; changes in visualizations; changes in documentation or docstrings; changes to features marked as experimental; changes when using Bionic in a way that the documentation tells you not to; or gracefully-handled changes to the cache data format (see next entry). We try to avoid breaking changes outside of major version releases.
Cache Compatibility Changes: Any case where this release of Bionic cannot read cached data generated by a previous release but handles the situation gracefully by automatically recomputing (and re-caching) values instead of loading old ones. If Bionic doesn’t handle this gracefully, this is marked as a bug instead.
Deprecated Features: Features which are still supported but may be removed in a future version. (This removal will count as a breaking change.) If a feature is deprecated, it will be noted in the appropriate docstring and Bionic will emit a warning message when the feature is used.
New Features: New capabilities which need to be explicitly activated by the user, generally through new functions or new arguments to existing functions.
Improvements: Any improvement that doesn’t require code changes from the user to activate, such as improved performance or clearer logging.
Documentation Changes: Significant changes to Bionic’s documentation; does not include changes associated with anything appearing in these notes (those are already assumed to be documented) or small corrections.
Development Changes: Significant changes to Bionic’s development process, such as changes to our Pytest configuration or our Continuous Integration (“CI”).
0.8.0 (Jul 6, 2020)¶
Cache Compatibility Changes¶
This release of Bionic cannot load cached data created by previous releases. Instead of attempting to load old an old cached value, Bionic will recompute (and re-cache) a new value.
New Features¶
Bionic flows can now be run in parallel, taking advantage of multiple CPUs at once.
Memoization can be globally disabled with the
core__memoize_by_default
entity, which means you can opt-in which entities are memoized instead of opting out.Bionic now allows entities to have no caching at all. Previously each entity needed to be cached either in memory or on disk; now both of these can be disabled, in which case it will be recomputed each time it’s needed.
GeoPandas DataFrames can now be serialized and cached. Bionic will do this automatically when an entity function returns a value of the appropriate type, but it can also be explicitly controlled with the new
@geodataframe
protocol.Bionic now provides an API for listing and deleting cached artifacts.
Documentation Changes¶
This Release Notes page was added!
Development Changes¶
Updated release process to use GitHub releases.
Moved CI and releases from Travis to GitHub Actions.
CI tests are now run using Python 3.8 (in addition to 3.7 and 3.6).
Add flake8-print, flake8-fixme to CI, and configure flake8 to warn on unused imports.
0.7.1 (May 11, 2020)¶
New Features¶
The
@picklable
decorator now accepts apickle_protocol_version
argument, which can be used to control the data format used by Python’s Pickle module.
Improvements¶
“Unset” entity (entities that are declared but have no value set) are now handled more cleanly. They now appear in the DAG visualization; if an entity value can’t be computed because one of its ancestors is unset, the exception message now describes the exact problem and the name of the problematic ancestor; and the
@gather
decorator now handles “missing” values more consistently by treating them as an empty set of values.Bionic now uses version 4 of the Pickle data format by default, so objects larger than 4 GB can be serialized without crashing.
Bug Fixes¶
The Protocol Interface now matches what is documented (it had drifted out of sync).
0.7.0 (Apr 7, 2020)¶
Cache Compatibility Changes¶
This release of Bionic cannot load cached data created by previous releases. Instead of attempting to load old an old cached value, Bionic will recompute (and re-cache) a new value.
New Features¶
Flow.render_dag
can now output the DAG as SVG in addition to existing formats. When SVG is used, entity docstrings appear as tooltips. SVG is the new default format for rendering in Jupyter Notebooks.The
@changes_per_run
decorator was added; this tells Bionic that a function is non-deterministic and should be re-run for each instance of a Flow.
Improvements¶
Exceptions originating in user-defined entity functions are now reported more clearly and include the name of the entity.
Cache invalidation now takes into account the hashed of immediate dependencies (not just top-level ancestors), reducing the number of unnecessary invalidations and recomputations.
The exception message for using protocol decorators with positional arguments (such as
@bn.protocol.frame('parquet')
), which is not allowed, is now clearer.
Bug Fixes¶
The failure to load cached data from older versions of Bionic, introduced in version 0.6.4, was fixed.
Documentation Changes¶
Some missing decorators were added to the API documentation.
The installation instructions were updated to work correctly on Zsh.
0.6.4 (Mar 3, 2020)¶
New Bugs¶
This release crashed when attempting to load cached data from older versions of Bionic. This was fixed in version 0.7.0.
Cache Compatibility Changes¶
This release of Bionic cannot load cached data created by previous releases. Instead of attempting to load old an old cached value, Bionic will recompute (and re-cache) a new value.
New Features¶
Improvements¶
The Bionic cache directory can now be renamed or moved without breaking it.
Deprecated Features¶
Development Changes¶
Travis config was updated to always use the most recent versions of all dependencies.
Regression tests were added to check that Bionic is capable of loading cached data from the previous versions. (This was intended to catch regressions like the one introduced in version 0.5.7, but unfortunately was too late to catch the one introduced in this version.)
0.6.3 (Dec 24, 2019)¶
Bug Fixes¶
Bionic no longer attempts to initialize Matplotlib in a special way on OS X, which was needed for versions of Matplotlib earlier than 3.1, but caused crashes for 3.1 and later.
0.6.2 (Dec 13, 2019)¶
Improvements¶
Entity-specific documentation is now visible in the docstrings for the shorthand
get
andset
methods, as inhelp(flow.get.my_entity_name)
.
0.6.1 (Nov 13, 2019)¶
Bug Fixes¶
The failure to load cached data from older versions of Bionic, introduced in version 0.5.7, was fixed.
0.6.0 (Nov 7, 2019)¶
Breaking Changes¶
Bionic no longer supports any Python version below 3.6. In particular, we no longer support Python 2.
0.5.7 (Nov 6, 2019)¶
New Bugs¶
This release crashed when attempting to load cached data from older versions of Bionic. This was fixed in version 0.6.1.
New Features¶
Flow.get
now has amode
argument, which can be used to retrieve the cached files for entities rather than their in-memory representations.Flow.merge
can now be called withkeep
values of"self"
or"arg"
; these do the same things as"old"
and"new"
, respectively, but were deemed clearer. The older values are still supported but may be deprecated in the future.
Improvements¶
Bionic is now released as a universal wheel, not just a source distribution.
Deprecated Features¶
Flow.get
’sfmt
argument is deprecated; instead, usecollection
, which does the same thing. (The namefmt
is confusing, since it would seem to do what the newmode
argument does rather than whatcollection
does.)Flow.export
is now deprecated; use the newmode
argument ofFlow.get
instead.
0.5.6 (Oct 23, 2019)¶
This release had no user-facing changes.
Development Changes¶
Bionic releases are now generated automatically by CI.
0.5.5 (Oct 23, 2019)¶
This release had no user-facing changes. It was an attempt to start automatically releasing via CI, but that didn’t work.
0.5.4 (Oct 22, 2019)¶
Improvements¶
Bionic no longer uses certain recent features of PyYAML, allowing it to be used with PyYAML 3.13. This in turn means it can be used with versions of Apache Beam (such as 2.15.0) which require PyYAML to have a version less than 4.0.
0.5.3 (Oct 21, 2019)¶
0.5.2 (Oct 17, 2019)¶
Bug Fixes¶
Fixed the bug introduced in 0.5.0, where
None
values could not be cached.
0.5.1 (Oct 11, 2019)¶
New Features¶
The
@pyplot
decorator now accepts asavefigs_kwargs
argument, which can be used to customize Matplotlib’s behavior when saving a plot.
Improvements¶
Matplotlib plots are now saved using
bbox_inches="tight"
by default.
Documentation Changes¶
The introductory documentation and tutorials were substantially reworded.
0.5.0 (Oct 9, 2019)¶
New Bugs¶
This release introduced a bug where Bionic would crash when attempting to cache a
None
value to disk. The bug was fixed in 0.5.2.
Cache Compatibility Changes¶
This release of Bionic cannot load cached data created by previous releases. Instead of attempting to load old an old cached value, Bionic will recompute (and re-cache) a new value.
New Features¶
Bionic now supports Automatic Versioning to help reduce the chance of forgetting to update the
@version
decorator.
Improvements¶
The “Computed …” log message is now logged immediately after an entity value is computed, so that the total compute time can be more accurately estimated from log timestamps.
Reading cache files is faster now that Bionic uses the C-based PyYAML loader.
Development Changes¶
Some Pytest tests are now designated “slow”, and will only be run when the
--slow
option is passed topytest
.
0.4.5 (Oct 8, 2019)¶
Improvements¶
Chained exceptions are now used (only in Python 3) to report caching errors more clearly.
A clearer error message is reported when attempting to serialize a Pandas DataFrame with duplicate column names with Parquet (which doesn’t work).
Writing cache files is faster now that Bionic uses the C-based PyYAML dumper.
0.4.4 (Oct 4, 2019)¶
0.4.3 (Sep 13, 2019)¶
Documentation Changes¶
Square’s CLA was added to our Contributing page.
0.4.2 (Sep 9, 2019)¶
Improvements¶
Bionic now uses
IncompatibleEntityError
to report problems where the user has defined multiple entities in an incompatible way. This is a subclass ofValueError
, which is what we used before.
Documentation Changes¶
The Flow Merging API, released in 0.4.0, is now correctly documented.