Running phables from a single container

--use-conda's per-rule environment creation is convenient but has a real failure mode at HPC scale: every rule's conda env is named by a hash of its env file, so every sample's phables run needs the same env directory — but each sample is its own separate Snakemake process (a SLURM array task, say), and Snakemake only serializes env creation within one process's DAG. Two array tasks both needing the same not-yet-built env at the same moment can race mamba env create into the same directory, corrupting it (Fatal Python error: init_fs_encoding ... no codec search functions registered is what that looks like when it happens).

container/Dockerfile builds a single image with every per-rule tool this workflow needs already installed — --container then points the whole workflow at it, replacing conda entirely so there's nothing left to race.

Getting an image

CI (.github/workflows/build_container.yaml) builds and pushes an image to quay.io/<QUAY_USERNAME>/phables:<commit-sha> on every push and pull request — tag by the exact commit you want, there's no floating latest.

Running with it

phables run --input assembly_graph.gfa --reads fastq \
    --container quay.io/gbouras13/phables:<commit-sha> \
    --no-use-conda \
    --use-singularity

--use-singularity is Snakemake's own flag for running rules inside Apptainer/Singularity (the standard container runtime on HPC, incl. Setonix) — pass it through phables' existing snake_args passthrough, same as --dry-run/--keep-going/etc. --use-conda alone does not activate container execution; explicitly turn it off (--no-use-conda) when using --container, since combining the two makes Snakemake build a separate conda env inside the container instead of using what's already installed there — exactly the redundant-torch-reinstall problem this image exists to avoid.

What's actually in the image

Base: quay.io/pawsey/pytorch:2.7.1-rocm6.3.3 — the same verified-working ROCm+PyTorch build envs/prostt5-rocm.yaml pins, and the same base phold's own container (../../phold/container/hpci/Dockerfile) builds from. Nothing in phables' own Dockerfile reinstalls or touches torch — pip install pholdlib and everything else pure-Python go straight into that same system Python, so they use the base image's already-correct torch/ROCm stack rather than resolving a second, possibly conflicting one. This is the direct answer to "don't rebuild the ProstT5 conda env for this container" — there's no separate ProstT5 env in the container at all; predict_3di runs against the same Python everything else does.

Everything else — foldseek (direct binary, matching phold's own install), minimap2/samtools/coverm/mmseqs2/FragGeneScan/HMMER/MAFFT (via a standalone micromamba install into /opt/conda, kept deliberately separate from the system Python so it can't touch it), and phables' own Python dependencies (pyrodigal-gv, biopython, python-igraph, pysam, flowpaths, cogent3/piqtree, ...) — covers every envs/*.yaml this workflow has, so the whole DAG can run from this one image with --no-use-conda.

--container vs --prostt5-container

--prostt5-container already existed for pointing just predict_3di at a container (e.g. phold's own image, which also has pholdlib+torch). It still works exactly as before, and if set, it wins for that one rule. --container is new and broader: it's the default container for every rule, including predict_3di if --prostt5-container isn't also given. In practice, setting --container alone is enough — there's no reason to use --prostt5-container separately unless you specifically want predict_3di on a different image than the rest of the workflow.

Building it yourself

docker build -f container/Dockerfile -t phables:local .

The base image is large (~14GB compressed) — building on a standard GitHub Actions runner needs the disk-cleanup step already in build_container.yaml (jlumbroso/free-disk-space), or the very first FROM line fails with no space left on device. Building locally needs proportionate free disk space too.

Not yet built or run for real — the Dockerfile and the workflow-wide --container plumbing were written and the Snakemake DAG wiring was verified via real --dry-runs (with and without --container set, confirming both resolve identically at the structural level), but no actual docker build or real Apptainer/Singularity execution against Setonix hardware has happened yet. Build it, push a first tag, and run one real sample through it before trusting this for production batches.