Skip to content

Worker-firm-period scores

This is the central HCT output: model scores and predicted separation hazards for observed annual worker-firm periods.

flowchart LR
    periods[(Model relationship periods)]
    checkpoint[(Trained HCT checkpoint)]
    score[Score observed periods]
    output[(Worker-firm-period scores)]

    periods --> score
    checkpoint --> score
    score --> output

    classDef source fill:#e8eef7,stroke:#52739e
    classDef process fill:#fff0cc,stroke:#a66b00
    classDef data fill:#eee5f5,stroke:#76528c
    class periods,checkpoint source
    class score process
    class output data
Inputs
Dataset Location
Model relationship periods /data/<coverage>/processed/model_relationship_periods/
Processed firms /data/<coverage>/processed/companies_processed.parquet
Selected fitted checkpoint For training, training_profiles.<profile>.scoring_checkpoints.training; for full_training, full_training_artifacts.scoring_checkpoint
Build
Component Location
Builder model/scoring/score_matches.py
Runtime hct-tower-model-cuda-… (GPU task)
Entry point python -m scoring.score_matches / score
Output
Dataset File / location Grain Key and variables
Worker-firm-period scores (training, worker profile) Source-named Parquet shards in /data/<coverage>/analyses/profile_worker/training/worker_firm_period/ One scored annual worker-firm period Key: user_id, relationship_number, company_id, period_start
Variables: calendar_year, tenure_year, match_score, hazard_baseline_prob, hazard_prob, hazard_label, destination_company_id, log_wage, wage_interpolated, seniority, job_category
Worker-firm-period scores (training, time profile) Source-named Parquet shards in /data/<coverage>/analyses/profile_time/training/worker_firm_period/ One scored annual worker-firm period Same score fields as the worker-profile output, with the time-profile split labels
Worker-firm-period scores (full_training) Source-named Parquet shards in /data/<coverage>/analyses/full_training/worker_firm_period/ One scored annual worker-firm period Same score fields; uses the shared checkpoint trained on all configured splits

The output preserves worker, relationship, firm, period, and split keys. It contains match_score, predicted hazard quantities, observed hazard_label, wage and job fields, and in/out-of-sample indicators. One output shard is written for each source relationship-period shard, with expected filenames and row counts recorded in _metadata.yaml.

  • training --profile worker uses the worker-split checkpoint fitted on the worker-profile train and validation splits and preserves held-out workers.
  • training --profile time uses the time-split checkpoint fitted on periods before the configured test window and preserves future-period test rows.
  • full_training uses the profile-independent checkpoint fitted on all configured splits and supplies the final MQ measure for downstream analyses. Omit --profile for this mode.
Command reference
Option Required / default Source-defined effect
--config PATH Required Model YAML config, including coverage-specific input, checkpoint, and output paths.
--mode training\|full_training Required Selects held-out profile training or the profile-independent full-training checkpoint.
--profile worker\|time Required with --mode training; omit for full_training Selects the split labels, training checkpoint, and profile-specific output root.
--resume Off Continues from the first incomplete output shard.
--show-setup-logs Off Shows verbose company and employee metadata-loading logs.
Option Required / default Source-defined effect
--task score Required Selects slurm/model/score.slurm.
--config CONFIG.yaml Required Resolves the model config passed to the scorer.
--mode training\|full_training Required Passed to the scorer.
--profile worker\|time Required with --mode training; omit for full_training Selects the profile-specific checkpoint and score root.
--resume Off Passed to the scorer.
--partition, --time, --mem, --cpus, --gpus, --constraint, --exclude, --job-name, --dependency, --dry-run Wrapper defaults Standard Slurm resource/submission overrides; default request is one GPU, 8 CPUs, 64 GB, and 24 hours.
--repo-root, --data-root, --artifact-root, --container-root, --sif Wrapper defaults Override host paths or the Apptainer image.
python -m scoring.score_matches --config config/config_one_percent.yaml --profile worker --mode training
python -m scoring.score_matches --config config/config_one_percent.yaml --profile time --mode training
python -m scoring.score_matches --config config/config_one_percent.yaml --mode full_training
slurm/submit.sh --task score --profile worker --mode training --config config_one_percent.yaml
slurm/submit.sh --task score --profile time --mode training --config config_one_percent.yaml
slurm/submit.sh --task score --mode full_training --config config_one_percent.yaml

The scores feed the disaggregated, worker-firm, company-year, and linked-executive datasets.