Tomasz Sieprawski

Repositories

5

Projects Packages

3

Public activity Starred repositories

1

backup-orchestrator (0.19.0)

Published 2026-09-09 15:28:00 +00:00 by tsieprawski

Installation

To install the package using pip, run the following command:

pip install --index-url  backup-orchestrator

For more information on the PyPI registry, see the documentation.

About this package

Remote-first backup orchestration client and target-side server

backup-orchestrator

Remote-first backup orchestration client and target-side server.

Current scope:

• installable Python package with local and SSH-remote orchestration commands
• lease wait, renewal, release, and rest-server lifecycle handling
• backup, snapshots, restore, and prune operations
• explicit password modes and optional persistent restic cache plumbing
• unit and integration tests
• release script and post-merge release workflow
• Forgejo workflows for checks, integration, release, and conventional commit validation

Local Usage

Run checks:

./scripts/check.sh

Run the integration test:

./scripts/test-integration.sh

Run the release integration test:

./scripts/test-integration-release.sh

Run the manual restic and Borg benchmark harness:

./scripts/test-benchmark-restic-cache.sh --runs 5

Notes:

• this is a manual benchmark tool, not a normal CI test
• it runs real local restic and borg repositories against the same synthetic datasets
• it writes JSON output under ./benchmark-results by default
• use it first on one machine locally, then reuse it on gator and zotac for cross-host measurements
• practical repetition guidance:
  • smoke or harness validation: --runs 3
  • first comparative pass: --runs 10
  • cheap scenario percentile runs: --runs 30 to --runs 100
  • expensive large-repository scenarios: usually --runs 5 to --runs 20
• current default decision-focused variants are:
  • restic: default, cache-off, group-by-paths, explicit-parent, no-scan, ignore-ctime, compression-off, compression-max
  • borg: default, files-cache-disabled
• restic runs use --read-concurrency 4; inode checking remains enabled because production sources are not FUSE or pCloud mounts
• use --scenario-prefixes huge for the most decision-relevant large Nextcloud-like comparisons
• for light local dry runs, keep --huge-target-gib modest and use a native Linux filesystem workdir
• for serious host runs, use a large disposable workdir on representative direct local storage; the completed 100 GiB gator run is the current practical scale reference, while the slow USB 100 GiB attempt exceeded 16 hours and larger runs should not be assumed feasible
• large runs need significantly more free workspace than the nominal --huge-target-gib target because source data, repository growth, cache state, temp files, and tool-specific working state coexist during the benchmark; plan for a substantial multi-times space multiplier rather than target-size-only headroom

Profile a real source tree before changing synthetic benchmark data:

python3 scripts/profile-backup-dataset.py /path/to/source --output source-profile.json

The profiler is read-only. It reports file-count, byte, extension, size-bucket, top-level-directory, and sampled content entropy/compressibility statistics to guide synthetic benchmark data design.

Run the CLI smoke command:

python -m backup_orchestrator
backup-orchestrator scaffold

The package also provides these operation families:

• server, repos, repo, claim, renew, and release for target-side repository and lease control;
• backup, snapshots, ls, restore, and prune for local rest-server sessions;
• remote backup, remote snapshots, remote ls, remote restore, and remote prune for SSH-tunneled target-side sessions;
• pass --include-file PATH to restore for a one-off partial restore;
• pass --dry-run to restore to preview the selected restore without writing files.

The downstream partial-restore wrapper accepts the same optional --dry-run argument after its include-file argument.

Restore include files contain restic include patterns, not JSON or a verbatim path list. Paths containing restic wildcard characters such as *, ?, or [ must be escaped according to restic's pattern rules.

For remote usage, the target host must provide a configured backup-orchestrator server and local wrapper command. The client requires an SSH target, a known-hosts file, and the target server port. Use the downstream adoption reference for the complete container and workflow contract.

Restic cache configuration:

• pass --restic-cache-dir /path/to/persistent/cache to local or remote backup, snapshots, ls, restore, or prune
• the value is exported as RESTIC_CACHE_DIR to the restic process
• downstreams own the persistent path, capacity, and container mount
• omit the option to retain restic's no-explicit-cache behavior
• backup-orchestrator 0.17.0 is the first released version containing this cache contract

Downstream remote adoption:

• see docs/downstream-remote-adoption.md for the client-wrapper, target, workflow, cache, SSH container-path, and integration-test contract
• the Copier template and adoption tests are available under templates/remote-downstream and tests/test-copier-adoption.sh

Copier Template

templates/remote-downstream/ is a Copier template for a downstream repository that runs a local backup-orchestrator client in Docker Compose. The generated client connects over SSH to the target host, uses the configured repository name, and includes:

• backup, snapshots, ls, restore, partial restore, and prune wrappers;
• SSH, cache, source, and restore mounts;
• target-side server and local-wrapper examples;
• manual and scheduled Forgejo workflows;
• static checks and a real Forgejo workflow smoke test.

Install it into a downstream repository:

python3 -m pip install 'copier==9.17.1'
git clone https://code.sieprawski.pl/tsieprawski/backup-orchestrator.git backup-orchestrator-template
python3 -m copier copy --trust \
  backup-orchestrator-template/templates/remote-downstream \
  /path/to/downstream-repository

Answer the Copier questions, review the generated diff, and review the source, repository, SSH, cache, restore, and scheduling values before using the workflows. Copier writes .copier-answers.yml in the generated repository.

The generated workflow smoke test uses the pinned setup-forgejo fixture, but that fixture is test infrastructure and is not generated into the downstream runtime. Add it after generation:

cd /path/to/downstream-repository
git submodule add \
  https://code.sieprawski.pl/tsieprawski/setup-forgejo/ \
  vendor/setup-forgejo
git submodule update --init --recursive

Run the generated tests before enabling production scheduling:

./tests/test-adoption-contract.sh
./tests/test-forgejo-workflows.sh

Licensing

All code in this repository is proprietary. All rights reserved.

Release

Merges to main are intended to trigger a package release to the local Forgejo PyPI registry.

Version bumps are driven by conventional commits:

• breaking changes: major
• feat: minor
• fix, ci, refactor, test: patch
• docs, chore: no release

Requirements

Requires Python: >=3.13

Details

PyPI

2026-09-09 15:28:00 +00:00

10

25 KiB

Assets (1)

backup_orchestrator-0.19.0-py3-none-any.whl 25 KiB

Versions (24) View all

0.19.0 2026-09-09

0.18.1 2026-08-30

0.18.0 2026-08-17

0.17.0 2026-08-10

0.16.0 2026-08-09