Skip to content

Model-output build workflow

Run this after the corresponding model checkpoint is complete. Direct commands run from /hct-tower/model in the model CUDA image (hct-tower-model-cuda-…) with the model-container PYTHONPATH configured. The Slurm tasks use the corresponding model CUDA Apptainer image.

export PYTHONPATH=/hct-tower:/hct-tower/model
  1. Score observed worker-firm periods for each profile-aware training checkpoint you intend to use, and optionally for the shared full-training checkpoint. The training mode is profile-specific; score worker and time separately if you need both held-out evaluations. full_training is profile-independent.

    python -m scoring.score_matches --config config/config_full.yaml --profile worker --mode training
    python -m scoring.score_matches --config config/config_full.yaml --profile time --mode training
    python -m scoring.score_matches --config config/config_full.yaml --mode full_training
    
    cd /hct-tower
    slurm/submit.sh --task score --profile worker --mode training --config config_full.yaml
    slurm/submit.sh --task score --profile time --mode training --config config_full.yaml
    slurm/submit.sh --task score --mode full_training --config config_full.yaml
    
    Produces
    Dataset File / dataset directory Folder
    Worker-firm-period scores (training, worker profile) score Parquet shards /data/full/analyses/profile_worker/training/worker_firm_period_scores/
    Worker-firm-period scores (training, time profile) score Parquet shards /data/full/analyses/profile_time/training/worker_firm_period_scores/
    Worker-firm-period scores (full_training) score Parquet shards /data/full/analyses/full_training/worker_firm_period_scores/
  2. Export embeddings only for downstream analyses that need them. Company vectors and worker vectors are separate targets. Worker exports can contain mover rows only or every worker relationship-period row; choose the desired population explicitly. See Embedding tables for separate Python and Slurm commands for each target and population.

Continue with the analysis-data workflow. In particular, destination ranks are analysis data because they combine model artifacts with the processed prior-year company employment counts that define their time-valid candidate pools.