Changes#

The document records all past pytask releases and what went into them in reverse chronological order. Releases follow semantic versioning and all releases are available on PyPI and Anaconda.org.

0.4.7 - 2024-03-19#

0.4.6 - 2024-03-13#

  • #576 fixes accidentally collecting pytask.MarkGenerator when using from pytask import mark.

0.4.5 - 2024-01-09#

  • #515 enables tests with graphviz in CI. Thanks to @NickCrews.

  • #517 raises an error when the configuration file contains a non-existing path (fixes #514). It also warns if the path is configured as a string, not a list of strings.

  • #519 raises an error when builtin functions are wrapped with task(). Closes #512.

  • #521 raises an error message when imported functions are wrapped with @task in a task module. Fixes #513.

  • #522 improves the issue templates.

  • #523 refactors _pytask.console._get_file.

  • #524 improves some linting and formatting rules.

  • #525 enables pytask to work with remote files using universal_pathlib.

  • #528 improves the codecov setup and coverage.

  • #535 reenables and fixes tests with Jupyter.

  • #536 allows partialed functions to be task functions.

  • #538 updates the documentation. For example, colon fences are replaced by backticks to allow formatting all pages by mdformat.

  • #539 implements the hook_module configuration value and --hook-module commandline option to register hooks.

  • #540 changes the CLI entry-point and allow pytask.build(tasks=task_func) as the signatures suggested.

  • #542 refactors the plugin manager.

  • #543 fixes imports in tests and related issues.

  • #544 requires sqlalchemy >=2 and upgrades the syntax.

  • #545 finalizes the release.

0.4.4 - 2023-12-04#

0.4.3 - 2023-12-01#

  • #483 simplifies the teardown of a task.

  • #484 raises an informative error message when directories instead of files are used with path nodes.

  • #485 adds missing steps to unconfigure pytask after the job is done, which caused flaky tests.

  • #486 adds default names to PPathNode.

  • #488 raises an error when an invalid value is used in a return annotation.

  • #489 and #491 simplifies parsing products and does not raise an error when a product annotation is used with the argument name produces. And allow produces to intake any node.

  • #490 refactors and better tests parsing of dependencies.

  • #493 allows tasks to depend on other tasks.

  • #496 makes pytask even lazier. Now, when a task produces a node whose hash remains the same, the consecutive tasks are not executed. It remained from when pytask relied on timestamps.

  • #497 removes unnecessary code in the collection of tasks.

  • #498 fixes an error when using Task and TaskWithoutPath in task modules.

  • #500 refactors the dependencies for tests.

  • #501 removes MetaNode.

  • #508 catches objects that pretend to be a PTask. Fixes #507.

0.4.2 - 2023-11-08#

  • #449 simplifies the code building of the plugin manager.

  • #451 improves collect_command.py and renames graph.py to dag_command.py.

  • #454 removes more .svgs and replaces them with animations.

  • #455 adds more explanation when load() fails during the execution.

  • #456 refers to the source code on Github when clicking on a source link.

  • #457 refactors everything around formatting node names.

  • #459 adds a pre-commit hook to sort __all__.

  • #460 simplifies removing internal tracebacks from exceptions with a cause.

  • #463 raises an error when a task function is not defined inside the loop body.

  • #464 improves pinned dependencies.

  • #465 adds test to ensure internal tracebacks are removed by reports.

  • #466 implements hashing for files instead of modification timestamps.

  • #470 moves .pytask.sqlite3 to .pytask.

  • #472 adds is_product to load().

  • #473 adds signatures to nodes which decouples an identifier from a name.

  • #477 updates the PyPI action.

  • #478 replaces black with ruff-format.

  • #479 gives skips a higher precedence as an outcome than ancestor failed.

  • #480 removes the check for missing root nodes from the generation of the DAG. It is delegated to the check during the execution.

  • #481 improves coverage.

  • #482 correctly handles names and signatures of PythonNode.

0.4.1 - 2023-10-11#

  • #443 ensures that PythonNode.name is always unique by only handling it internally.

  • #444 moves all content of setup.cfg to pyproject.toml.

  • #446 refactors create_name_of_python_node and fixes PythonNodes as returns.

  • #447 simplifies the tree_map code while generating the DAG.

  • #448 fixes handling multiple product annotations of a task.

0.4.0 - 2023-10-07#

  • #323 remove Python 3.7 support and use a new Github action to provide mamba.

  • #384 allows to parse dependencies from every function argument if depends_on is not present.

  • #387 replaces pony with sqlalchemy.

  • #391 removes @pytask.mark.parametrize.

  • #394 allows to add products with typing.Annotated and Product.

  • #395 refactors all occurrences of pybaum to _pytask.tree_util.

  • #396 replaces pybaum with optree and adds paths to the name of pytask.PythonNode’s allowing for better hashing.

  • #397 adds support for typing.NamedTuple and attrs classes in @pytask.mark.task(kwargs=...).

  • #398 deprecates the decorators @pytask.mark.depends_on and @pytask.mark.produces.

  • #402 replaces ABCs with protocols allowing for more flexibility for users implementing their own nodes.

  • #404 allows to use function returns to define task products.

  • #406 allows to match function returns to node annotations with prefix trees.

  • #408 removes .value from Node protocol.

  • #409 make .from_annot an optional feature of nodes.

  • #410 allows to pass functions to PythonNode(hash=...).

  • #411 implements a new functional interface and adds experimental support for defining and running tasks in REPLs or Jupyter notebooks.

  • #412 adds protocols for tasks.

  • #413 removes scripts to generate .svgs.

  • #414 allow more ruff rules.

  • #416 removes .from_annot again.

  • #417 deprecates pytask.mark.task in favor of pytask.task().

  • #418 fixes and error and simplifies code in dag.py.

  • #420 converts DeprecationWarnings to FutureWarnings for the deprecated decorators.

  • #421 removes the deprecation warning when produces is used as an magic function keyword to define products.

  • #423 adds a notebook to explain the functional interface.

  • #424 fixes problems with ~_pytask.path.import_path.

  • #426 publishes the pytask.tree_util module.

  • #427 fixes type annotations for pytask.PTask.depends_on and pytask.PTask.produces.

  • #428 updates the example in the readme.

  • #429 implements a more informative error message when node.state() throws an exception. Now, it is easy to see which tasks are affected.

  • #430 updates some parts of the documentation.

  • #431 enables colors for WSL.

  • #432 fixes type checking of pytask.mark.xxx.

  • #433 fixes the ids generated for PythonNodes.

  • #437 fixes the detection of task functions and publishes pytask.is_task_function().

  • #438 clarifies some types.

  • #440 refines more types.

  • #441 updates more parts of the documentation.

  • #442 allows users to import from pytask import mark and use @mark.skip.

0.3.2 - 2023-06-07#

  • #345 updates the version numbers in animations.

  • #352 publishes db that is required by pytask-environment.

  • #354 adds a -f/--force flag to execute tasks even though nothing may have changed.

  • #355 refactors a lot of things related to nodes.

  • #357 add hashing for task files to detect changes when modification times do not match.

  • #364 updates update_plugin_list.py.

  • #365 reworks the panel on the index page with sphinx-design.

  • #366 adds light and dark logos and fixes some warnings when building the documentation.

  • #369 fixes an error in update_plugin_list.py introduced by #364.

  • #370 reverts the changes that turn Node.state() into a hook.

  • #371 renames Node to MetaNode.

  • #373 adds importing task modules to sys.modules and fully adopting pytest’s importlib mode. Thanks to @NickCrews. (Fixes #374.)

  • #376 enhances the documentation for pytask dag.

  • #378 conditionally skips test on MacOS.

  • #381 deprecates @pytask.mark.parametrize. (Closes #233.)

  • #501 removes pytask.MetaNode.

0.3.1 - 2023-12-25#

  • #337 fixes fallback to root path when pytask collect or pytask clean are used without paths.

0.3.0 - 2023-12-22#

  • #313 refactors the configuration. INI configurations are no longer supported.

  • #326 fixes the badge for status of the workflow.

  • #329 adds ruff to pre-commit hooks.

  • #330 add a guide for migrating from scripts to pytask.

  • #332 refactors database.py.

  • #333 requires attrs v21.3.0 and updates the code accordingly.

  • #334 adds target-version to ruff config.

0.2.7 - 2022-12-14#

  • #307 adds Python 3.11 to the CI.

  • #308 replaces pydot with pygraphviz.

  • #311 fixes a link in the documentation.

  • #311 adds refurb to pre-commit hooks.

  • #318 clarifies an example on nested dependencies and products.

  • #321 converts more choice options to enums.

  • #322 replaces SVGs with animations by termynal.

  • #325 allows to collect dynamically created tasks.

0.2.6 - 2022-10-27#

  • #297 moves non-hook functions from warnings.py to warnings_utils.py and publishes them so that pytask-parallel can import them.

  • #305 removes traces of colorama. Whatever it did should be handled by rich.

0.2.5 - 2022-08-02#

  • #288 fixes pinning pybaum to v0.1.1 or a version that supports tree_yield().

  • #289 shortens the task ids when using pytask collect. Fixes #286.

  • #290 implements a dry-run with pytask --dry-run to see which tasks would be executed.

  • #296 fixes a bug where the source code of the wrapped function could not be retrieved.

0.2.4 - 2022-06-28#

  • #279 enhances some tutorials with spell and grammar checking.

  • #282 updates the tox configuration.

  • #283 fixes an issue with coverage and tests using pexpect + pdb.set_trace().

  • #285 implements that pytask does not show the traceback of tasks that are skipped because their previous task failed. Fixes #284.

  • #287 changes that all files that are not produced by a task are displayed in the error message. Fixes #262.

0.2.3 - 2022-05-30#

  • #276 fixes pytask clean when git is not installed. Fixes #275.

  • #277 ignores DeprecationWarning and PendingDeprecationWarning by default. Previously, they were enabled, but they should be shown when testing the project with pytest, not after the execution with pytask. Fixes #269.

  • #278 counts multiple occurrences of a warning instead of listing the module or task name again and again. Fixes #270.

0.2.2 - 2022-05-14#

  • #267 fixes the info under the live execution table to show the total number of tasks also for pytask-parallel.

  • #273 reworks pytask clean so that it ignores files tracked by git. Resolves #146.

0.2.1 - 2022-04-28#

  • #259 adds an .svg for profiling tasks.

  • #261 adds a config file option to sort entries in live table

  • #262 allows pytask to capture warnings. Here is the guide.

0.2.0 - 2022-04-14#

  • #211 allows for flexible dependencies and products which can be any pytree of native Python objects as supported by pybaum.

  • #227 implements task.kwargs as a new way for a task to hold parametrized arguments. It also implements _pytask.models.CollectionMetadata to carry parametrized arguments to the task class.

  • #228 removes task.pytaskmark and moves the information to pytask.CollectionMetadata.markers.

  • #229 implements a new loop-based approach to parametrizations using the @pytask.mark.task decorator.

  • #230 implements _pytask.logging._TimeUnit as a typing.NamedTuple for better typing.

  • #232 moves the documentation to MyST.

  • #234 removes MetaTask. There is only pytask.Task.

  • #235 refactors the utility functions for dealing with marks in _pytask.mark_utils. (Closes #220.)

  • #236 refactors _pytask.collect and places shared functions in _pytask.collect_utils.

  • #237 publish more functions.

  • #238 allows any order of decorators with a @pytask.mark.task decorator.

  • #239 adds a warning on globals for parametrizations and some fixes.

  • #241 allows parametrizing over single dicts.

  • #242 removes tasks from global _pytask.task_utils.COLLECTED_TASKS to avoid re-collection when the programmatic interface is used.

  • #243 converts choice options to use enums instead of simple strings.

  • #245 adds choices on the command line to the help pages as metavars and show defaults.

  • #246 formalizes choices for click.Choice to enum.Enum.

  • #252 adds a counter at the bottom of the execution table to show how many tasks have been processed.

  • #253 adds support for pyproject.toml.

  • #254 improves test coverage, fixes a bug, and improves the deprecation message for the configuration.

  • #255 converts the readme to markdown and multiple pngs to svgs.

  • #256 adds even more svgs and scripts to generate them to the documentation and other improvements.

0.1.9 - 2022-02-23#

  • #197 publishes types, and adds classes and functions to the main namespace.

  • #217 enhances the tutorial on how to set up a project.

  • #218 removes depends_on and produces from the task function when parsed.

  • #219 removes some leftovers from pytest in Mark.

  • #221 adds more test cases for parametrizations.

  • #222 adds an automated Github Actions job for creating a list pytask plugins.

  • #225 fixes a circular import noticeable in plugins created by #197.

  • #226 fixes a bug where the number of items in the live table during the execution is not exhausted. (Closes #223.)

0.1.8 - 2022-02-07#

  • #210 allows __tracebackhide__ to be a callable that accepts the current exception as an input. Closes #145.

  • #213 improves coverage and reporting.

  • #215 makes the help pages of the CLI prettier.

0.1.7 - 2022-01-28#

  • #153 adds support for Python 3.10 which requires pony >= 0.7.15.

  • #192 deprecates Python 3.6.

  • #209 cancels previous CI jobs when a new job is started.

0.1.6 - 2022-01-27#

  • #191 adds a guide on how to profile pytask to the developer’s guide.

  • #192 deprecates Python 3.6.

  • #193 adds more figures to the documentation.

  • #194 updates the README.rst.

  • #196 references the two new cookiecutters for projects and plugins.

  • #198 fixes the documentation of @pytask.mark.skipif. (Closes #195)

  • #199 extends the error message when paths are ambiguous on case-insensitive file systems.

  • #200 implements the @pytask.mark.task decorator to mark functions as tasks regardless of whether they are prefixed with task_ or not.

  • #201 adds tests for _pytask.mark_utils.

  • #204 removes internal traceback frames from exceptions raised somewhere in pytask.

  • #208 fixes the best practices guide for parametrizations.

  • #209 cancels previous CI runs automatically.

  • #212 add .coveragerc and improve coverage.

0.1.5 - 2022-01-10#

  • #184 refactors reduce_node_name() and shorten task names in many places.

  • #185 fix issues with drawing a graph and adds the --rank-direction to change the direction of the DAG.

  • #186 enhance live displays by deactivating auto-refresh, among other things.

  • #187 allows to enable and disable showing tracebacks and potentially different styles in the future with show_traceback=True|False.

  • #188 refactors some code related to pytask.ExitCode.

  • #189 do not display a table in the execution if no task was run.

  • #190 updates the release notes.

0.1.4 - 2022-01-04#

  • #153 adds support and testing for Python 3.10.

  • #159 removes files for creating a conda package which is handled by conda-forge.

  • #160 adds rudimentary typing to pytask.

  • #161 removes a workaround for pyreadline which is also removed in pytest 7.

  • #163 allow forward slashes in expressions and marker expressions.

  • #164 allows to use backward slashes in expressions and marker expressions.

  • #167 makes small changes to the docs.

  • #172 embeds URLs in task ids. See editor_url_scheme for more information.

  • #173 replaces ColorCode with custom rich themes.

  • #174 restructures loosely defined outcomes to clear enum.Enum.

  • #176 and #177 implement a summary panel that holds aggregate information about the number of successes, fails and other status.

  • #178 adds stylistic changes like reducing tasks ids even more and dimming the path part.

  • #180 fixes parsing relative paths from the configuration file.

  • #181 adds correct formatting of running tasks.

  • #182 introduces that only the starting year is displayed in the license following https://hynek.me/til/copyright-years.

  • #183 enables tracing down the source of a function through decorators.

0.1.3 - 2021-11-30#

  • #157 adds packaging to the dependencies of the package.

  • #158 converts time units to the nearest integer.

0.1.2 - 2021-11-27#

  • #135 implements handling of version in docs as proposed by setuptools-scm.

  • #142 removes the display of skipped and persisted tasks from the live execution table for the default verbosity level of 1. They are displayed at 2.

  • #144 adds tryceratops to the pre-commit hooks for catching issues with exceptions.

  • #151 adds a limit on the number of items displayed in the execution table which is configurable with n_entries_in_table in the configuration file.

  • #152 makes the duration of the execution readable by humans by separating it into days, hours, minutes and seconds.

  • #155 implements functions to check for optional packages and programs and raises errors for requirements to draw the DAG earlier.

  • #156 adds the option show_errors_immediately to print/show errors as soon as they occur. Resolves #150.

0.1.1 - 2021-08-25#

  • #138 changes the default verbosity to 1 which displays the live table during execution and 0 display the symbols for outcomes (e.g. ., F, s).

  • #139 enables rich’s auto-refresh mechanism for live objects which causes almost no performance penalty for the live table compared to the symbolic output.

0.1.0 - 2021-07-20#

  • #106 implements a verbose mode for the execution which is available with pytask -v and shows a table with running and completed tasks. It also refines the collection status.

  • #116, #117, and #123 fix #104 which prevented to skip tasks with missing dependencies.

  • #118 makes the path to the configuration in the session header os-specific.

  • #119 changes that when marker or keyword expressions are used to select tasks, also the predecessors of the selected tasks will be executed.

  • #120 implements that a single KeyboardInterrupt stops the execution and previously collected reports are shown. Fixes #111.

  • #121 add skipped and persisted tasks to the execution footer.

  • #127 make the table during execution the default. Silence pytask with negative verbose mode integers and increase verbosity with positive ones.

  • #129 allows to hide frames from the traceback by using __tracebackhide__ = True.

  • #130 enables rendering of tracebacks from subprocesses with rich.

0.0.16 - 2021-06-25#

  • #113 fixes error when using pytask --version with click v8.

0.0.15 - 2021-06-24#

  • #80 replaces some remaining formatting using pprint with rich.

  • #81 adds a warning if a path is not correctly cased on a case-insensitive file system. This facilitates cross-platform builds of projects. Deactivate the check by setting check_casing_of_paths = false in the configuration file. See check_casing_of_paths for more information.

  • #83 replaces versioneer with setuptools_scm.

  • #84 fixes an error in the path normalization introduced by #81.

  • #85 sorts collected tasks, dependencies, and products by name.

  • #87 fixes that dirty versions are displayed in the documentation.

  • #88 adds the pytask profile command to show information on tasks like duration and file size of products.

  • #93 fixes the display of parametrized arguments in the console.

  • #94 adds show_locals which allows to print local variables in tracebacks.

  • #96 implements a spinner to show the progress during the collection.

  • #99 enables color support in WSL and fixes show_locals during collection.

  • #101 implement to visualize the project’s DAG. #108 refines the implementation.

  • #102 adds an example if a parametrization provides not the number of arguments specified in the signature.

  • #105 simplifies the logging of the tasks.

  • #107 adds and new hook pytask_unconfigure() which makes pytask return pdb.set_trace() at the end of a session which allows to use breakpoint() inside test functions using pytask.

  • #109 makes pytask require networkx>=2.4 since previous versions fail with Python 3.9.

  • #110 adds a “New Features” section to the README.rst.

0.0.14 - 2021-03-23#

  • #74 reworks the formatting of the command line output by using rich. Due to the new dependency, support for pytask with Python <3.6.1 on PyPI and with Python <3.7 on Anaconda will end.

  • #76 fixes #75 which reports a bug when a closest ancestor cannot be found to shorten node names in the CLI output. Instead a common ancestor is used.

0.0.13 - 2021-03-09#

  • #72 adds conda-forge to the README and highlights importance of specifying dependencies and products.

  • #62 implements the pytask.mark.skipif() marker to conditionally skip tasks. Many thanks to @roecla for implementing this feature and a warm welcome since she is the first pytask contributor!

0.0.12 - 2021-02-27#

  • #55 implements miscellaneous fixes to improve error message, tests and coverage.

  • #59 adds a tutorial on using plugins and features plugins more prominently.

  • #60 adds the MIT license to the project and mentions pytest and its developers.

  • #61 adds many changes to the documentation.

  • #65 adds versioneer to pytask and #66 corrects the coverage reports which were deflated due to the new files.

  • #67 prepares pytask to be published on PyPI and #68 fixes the pipeline, and #69 prepares releasing v0.0.12 and adds new shields.

0.0.11 - 2020-12-27#

  • #45 adds the option to stop execution after a number of tasks has failed. Closes #44.

  • #47 reduce node names in error messages while resolving dependencies.

  • #49 starts a style guide for pytask.

  • #50 implements correct usage of singular and plural in collection logs.

  • #51 allows to invoke pytask through the Python interpreter with python -m pytask which will add the current path to sys.path.

  • #52 allows to prioritize tasks with pytask.mark.try_last and pytask.mark.try_first.

  • #53 changes the theme of the documentation to furo.

  • #54 releases v0.0.11.

0.0.10 - 2020-11-18#

  • #40 cleans up the capture manager and other parts of pytask.

  • #41 shortens the task ids in the error reports for better readability.

  • #42 ensures that lists with one element and dictionaries with only a zero key as input for @pytask.mark.depends_on and @pytask.mark.produces are preserved as a dictionary inside the function.

0.0.9 - 2020-10-28#

  • #31 adds pytask collect to show information on collected tasks.

  • #32 fixes pytask clean.

  • #33 adds a module to apply common parameters to the command line interface.

  • #34 skips pytask_collect_task_teardown if task is None.

  • #35 adds the ability to capture stdout and stderr with the CaptureManager.

  • #36 reworks the debugger to make it work with the CaptureManager.

  • #37 removes reports argument from hooks related to task collection.

  • #38 allows to pass dictionaries as dependencies and products and inside the function depends_on and produces become dictionaries.

  • #39 releases v0.0.9.

0.0.8 - 2020-10-04#

  • #30 fixes or adds the session object to some hooks which was missing from the previous release.

0.0.7 - 2020-10-03#

  • #25 allows to customize the names of the task files.

  • #26 makes commands return the correct exit codes.

  • #27 implements the pytask_collect_task_teardown hook specification to perform checks after a task is collected.

  • #28 implements the @pytask.mark.persist decorator.

  • #29 releases 0.0.7.

0.0.6 - 2020-09-12#

  • #16 reduces the traceback generated from tasks, failure section in report, fix error passing a file path to pytask, add demo to README.

  • #17 changes the interface to subcommands, adds "-c/--config" option to pass a path to a configuration file and adds pytask clean (#22 as well), a command to clean your project.

  • #18 changes the documentation theme to alabaster.

  • #19 adds some changes related to ignored folders.

  • #20 fixes copying code examples in the documentation.

  • #21 enhances the ids generated by parametrization, allows to change them via the ids argument, and adds tutorials.

  • #23 allows to specify paths via the configuration file, documents the cli and configuration options.

  • #24 releases 0.0.6.

0.0.5 - 2020-08-12#

  • #10 turns parametrization into a plugin.

  • #11 extends the documentation.

  • #12 replaces pytest.mark with pytask.mark.

  • #13 implements selecting tasks via expressions or marker expressions.

  • #14 separates the namespace of pytask to pytask and _pytask.

  • #15 implements better tasks ids which consists of <path-to-task-file>::<func-name> and are certainly unique. And, it releases 0.0.5.

0.0.4 - 2020-07-22#

  • #9 adds hook specifications to the parametrization of tasks which allows pytask-latex and pytask-r to pass different command line arguments to a parametrized task and its script. Also, it prepares the release of 0.0.4.

0.0.3 - 2020-07-19#

  • #7 makes pytask exit with code 1 if a task failed and the skip_ancestor_failed decorator is only applied to descendant tasks not the task itself.

  • #8 releases v0.0.3

0.0.2 - 2020-07-17#

  • #2 provided multiple small changes.

  • #3 implements a class which holds the execution report of one task.

  • #4 makes adjustments after moving to main as the default branch.

  • #5 adds pytask_add_hooks to add more hook specifications and register hooks.

  • #6 releases v0.0.2.

0.0.1 - 2020-06-29#

  • #1 combined the whole effort which went into releasing v0.0.1.