Skip to content

Worker-firm relationships

This dataset collapses consecutive positions at the same firm into an employment relationship.

flowchart LR
    events[(Employment events)] --> group[Group by worker and<br/>relationship number]
    group --> relationships[(Worker-firm relationships)]

    classDef source fill:#e8eef7,stroke:#52739e
    classDef process fill:#fff0cc,stroke:#a66b00
    classDef data fill:#e2f2e7,stroke:#3b7a4b
    class events source
    class group process
    class relationships data
Inputs
Dataset Location
Employment events /data/<coverage>/processed/employment_events_processed/
Build
Component Location
Builder shared/data/build_processed.py
Runtime hct-tower-model-cuda-… (CPU execution)
Entry point python -m shared.data.build_processed --target relationships / build-processed
Output
Dataset File / location Grain Key and variables
Worker-firm relationship shards Source-named Parquet shards in /data/<coverage>/processed/employment_relationships_processed/ Consecutive worker-firm spell Key: user_id, relationship_number
Variables: company_id, relationship_start_date, relationship_end_date, next_company_id, num_positions
Command reference
Option Required / default Source-defined effect
--config PATH Required Shared YAML config that supplies inputs and output paths.
--target all\|employees\|companies\|relationships\|company-year-counts all Selects the artifact group; use relationships for this dataset.
Option Required / default Source-defined effect
--task build-processed Required Invokes the complete processed bundle (--target all); the wrapper does not expose --target.
--config CONFIG.yaml Required Resolves the shared config passed to the builder.
--partition, --time, --mem, --cpus, --job-name, --dependency, --dry-run Wrapper defaults Standard CPU overrides; default is 8 CPUs, 128 GB, and 6 hours.
python -m shared.data.build_processed \
  --config shared/config/config_full.yaml --target relationships
# The Slurm task builds the complete processed-employment bundle.
slurm/submit.sh --task build-processed --config config_full.yaml

For each relationship, the builder records:

  • company_id;
  • first position start as relationship_start_date;
  • relationship end date, left missing when any constituent position is open;
  • num_positions within the relationship;
  • next_company_id, taken from the worker's following relationship when one is observed.

The dataset distinguishes a relationship from a position: a worker can hold several roles at one firm without creating a new relationship. It feeds the model relationship-period builder and the observed firm-year employment counts.