Lab Containers¶

This guide is for Yen/lab-server workflows using Podman. Slurm execution and Apptainer image conversion are documented in slurm.md.

The lab setup keeps the same in-container paths as the local setup:

repo checkout at /hct-tower
datasets at /data
runtime outputs at /artifacts

The examples below assume the repo is cloned directly under your home directory on the lab server and that shared storage lives under /zfs/projects/faculty/jungho-idhlab/drstrange.

Node-Local Podman State¶

Podman images and named containers on the lab server are local to the cluster node where you build or create them. If you later connect to a different node, those images and containers may not be present. Rebuild the image and recreate the named container on that node if you need an interactive Podman workflow there.

Shared project data, artifacts, and exported Apptainer .sif files should live on shared storage. Use the Slurm/Apptainer workflow for jobs that need to run independent of a specific node's local Podman state.

Rootless Podman Prerequisite¶

When building Podman images on the Yen server, rootless Podman may fail to chown files on the underlying filesystem unless the overlay driver is told to ignore those errors.

Create or update ~/.config/containers/storage.conf:

[storage]
driver = "overlay"

[storage.options.overlay]
ignore_chown_errors = "true"

This only needs to be done once per user account.

The analyses image also tells R package source extraction not to preserve archive owners. This is needed because rootless Podman on the lab server does not map arbitrary package-archive UIDs/GIDs into the build user namespace.

Model Podman Image¶

The lab model image is a CUDA image built from model/Containerfile.

Build from inside model/:

cd model
podman build -t hct-tower-model-cuda .

On standard server nodes, GPU access is usually available only through Slurm. Build the CUDA image anyway so it can later be exported and used by Slurm jobs. For ordinary shell access on standard nodes, create the reusable container without a GPU flag:

podman create -it --name hct-tower-model-cuda \
  -v $HOME/hct-tower:/hct-tower \
  -v /zfs/projects/faculty/jungho-idhlab/drstrange/data:/data \
  -v /zfs/projects/faculty/jungho-idhlab/drstrange/artifacts:/artifacts \
  -w /hct-tower/model \
  hct-tower-model-cuda

Note that you can create this container to run code on the Yen's interactive nodes, but you should be mindful of the Community Guidelines. For GPU access, the lab server requires a Slurm GPU job. For these, an Apptainer image is used and created from the Podman image, but a Podman container is not used directly. See slurm.md for details.

Start and enter the container:

podman start hct-tower-model-cuda
podman exec -it hct-tower-model-cuda bash

After entering the container, model workflow commands are documented on the model landing page.

Analyses Podman Image¶

The lab analyses image is a CPU image built from analyses/Containerfile. By default it uses rocker/verse on amd64, which matches the lab-server architecture.

Build from inside analyses/:

cd analyses
podman build -t hct-tower-analyses-cpu .

Create a reusable analyses container:

podman create -it --name hct-tower-analyses-cpu \
  -v $HOME/hct-tower:/hct-tower \
  -v /zfs/projects/faculty/jungho-idhlab/drstrange/data:/data \
  -v /zfs/projects/faculty/jungho-idhlab/drstrange/artifacts:/artifacts \
  -w /hct-tower/analyses \
  hct-tower-analyses-cpu

Start and enter the container:

podman start hct-tower-analyses-cpu
podman exec -it hct-tower-analyses-cpu bash

The analyses image is R-only and intentionally does not include the Python environment needed for shared.data.* pipeline commands. Build shared processed data and reusable about embeddings from the model environment when those files are missing or stale, then consume the resulting parquet files from analyses. If they already exist at the configured /data/... paths, there is no need to rebuild them.

After entering the container, analyses workflow commands are documented on the analyses landing page.

Slurm And Apptainer¶

For cluster jobs, use the centralized Slurm interface in slurm.md. That guide covers:

exporting the Podman CUDA image to a container archive
converting the archive to an Apptainer .sif
running interactive Slurm checks
submitting model and shared data-building jobs