Skip to content

Employment events

Employment events are the position-level record used to reconstruct worker histories and worker-firm relationships.

flowchart LR
    raw[(US employment-position records)] --> build[Clean and order positions]
    build --> events[(Employment events)]

    classDef source fill:#e8eef7,stroke:#52739e
    classDef process fill:#fff0cc,stroke:#a66b00
    classDef data fill:#e2f2e7,stroke:#3b7a4b
    class raw source
    class build process
    class events data
Inputs
Dataset Location
US employment-position records /data/full/raw/revelio_us_positions_users_education_user_*.parquet
Build
Component Location
Builder shared/data/build_processed.py
Runtime hct-tower-model-cuda-… (CPU execution)
Entry point python -m shared.data.build_processed --target employees / build-processed
Output
Dataset File / location Grain Key and variables
Employment-event shards Source-named Parquet shards in /data/<coverage>/processed/employment_events_processed/ Revelio employment position Key: position_id
Variables: user_id, relationship_number, company_id, start_date, end_date, total_compensation, start_salary, end_salary, salary, seniority, job_category, description_position
Command reference
Option Required / default Source-defined effect
--config PATH Required Shared YAML config that supplies raw inputs and processed output paths.
--target all\|employees\|companies\|relationships\|company-year-counts all Selects the artifact group; employees builds employment events and workers.
Option Required / default Source-defined effect
--task build-processed Required Invokes the complete processed bundle (--target all); the wrapper does not expose --target.
--config CONFIG.yaml Required Resolves the shared config passed to the builder.
--partition, --time, --mem, --cpus, --job-name, --dependency, --dry-run Wrapper defaults Standard CPU overrides; default is 8 CPUs, 128 GB, and 6 hours.
python -m shared.data.build_processed \
  --config shared/config/config_full.yaml --target employees
# The Slurm task builds the complete processed-employment bundle.
slurm/submit.sh --task build-processed --config config_full.yaml

The builder standardizes worker and firm identifiers, parses dates, cleans compensation and role fields, and sorts each worker's positions. It assigns a relationship_number whenever the employer changes in that ordered history. Multiple positions can therefore belong to the same worker-firm relationship.

Important columns are:

  • identifiers: user_id, position_id, company_id, relationship_number;
  • dates: start_date, end_date;
  • compensation: salary, start_salary, end_salary, total_compensation;
  • role attributes: seniority, job_category, description_position.

The output is sharded by worker. All positions for a worker remain together, which allows downstream relationship and model-period builders to process one shard at a time.