Skip to content

Model relationship periods

This dataset expands worker-firm relationships into annual rows used by model training, evaluation, scoring, and several descriptive analyses.

flowchart TB
    events[(Employment events)]
    relationships[(Worker-firm relationships)]
    workers[(Workers)]
    firms[(Firms)]
    build[Create annual risk-set rows<br/>and model features]
    periods[(Model relationship periods)]

    events --> build
    relationships --> build
    workers --> build
    firms --> build
    build --> periods

    classDef source fill:#e8eef7,stroke:#52739e
    classDef process fill:#fff0cc,stroke:#a66b00
    classDef data fill:#e2f2e7,stroke:#3b7a4b
    class events,relationships,workers,firms source
    class build process
    class periods data

Grain: one annual worker-firm risk-set period
Location: /data/<coverage>/processed/model_relationship_periods/
Builder: model/data/build_relationship_periods.py
Entry point: model/data/build_model_datasets.py

Rows contain worker, relationship, and firm keys; period date; tenure and calendar year; current job and wage fields; encoded worker features; employment history; observed destination fields; and hazard_label. The configured split strategy assigns train, valid, or test; under the current worker strategy, all rows for a worker share one split.

Companion outputs under the same processed directory are:

  • model_relationship_metadata.yaml, with source, schema, and split counts;
  • model_relationship_company_counts.parquet, with company example counts by split.
python -m data.build_model_datasets --config config/config_one_percent.yaml
slurm/submit.sh --task build-model-periods --config config_one_percent.yaml

Output shard names match the processed relationship shards. Scoring and the disaggregated-data builder rely on that alignment.