Skip to content

Sample description

This analysis summarizes the processed employment sample used by the paper. It does not use MQ scores or Compustat-linked variables.

flowchart LR
    employment[(Events, relationships,<br/>workers, firms, and periods)]
    summarize[[Compute native-grain statistics]]
    csv[(sample_description.csv)]
    latex[/sample_description.tex/]

    employment --> summarize
    summarize --> csv
    csv --> latex

    classDef upstream fill:#e8eef7,stroke:#52739e,stroke-width:2px
    classDef code fill:#f3f3f3,stroke:#666,stroke-width:2px
    classDef support fill:#fff0cc,stroke:#a66b00,stroke-width:2px
    classDef tableOutput fill:#e2f2e7,stroke:#3b7a4b,stroke-width:2px
    class employment upstream
    class summarize code
    class csv support
    class latex tableOutput
Inputs

A full recomputation requires all five processed datasets below. The paths are read from the corresponding data.* keys in the analyses config; <coverage> is full, samples, or one_percent.

Required dataset Config key Location Used for
Employment events data.employment_events_processed_path /data/<coverage>/processed/employment_events_processed/ Position count, salary, seniority, role tenure, and job-category shares
Worker-firm relationships data.employment_relationships_processed_path /data/<coverage>/processed/employment_relationships_processed/ Relationship count, positions per relationship, completed tenure, next-employer availability, and relationships per worker/firm
Workers data.employees_processed_path /data/<coverage>/processed/employees_processed.parquet Worker count, degree share, and predicted female share
Firms data.companies_processed_path /data/<coverage>/processed/companies_processed.parquet Firm count and business-description availability
Model relationship periods data.relationship_periods_path /data/<coverage>/processed/model_relationship_periods/ Worker-firm-period separation rate

The module checks for the required columns in each dataset when it opens them. Worker-firm-period score files are not an input: --mode is only recorded in the output metadata and does not select a score dataset.

If a valid sample_description.csv already exists in the configured sample_description.results_dir, the pipeline uses that cached summary to regenerate the LaTeX and metadata without opening these five datasets. If the cache is absent or lacks required rows/columns, it performs the full recomputation above.

Build
Component Location
Builder analyses/modules/sample_description.R
Runtime hct-tower-analyses-cpu
Entry point analyses/pipelines/sample_description.R / sample-description
Output
Dataset File / location Grain and variables
Sample description sample_description.csv, sample_description.tex, and metadata in /artifacts/<coverage>/analyses/sample_description/ Manuscript table of native-grain employment, relationship, worker, firm, and period summaries

The table contains employment-level, relationship-level, worker-level, and firm-level panels. Employment salary, seniority, role tenure, and job-category shares use processed employment events. Relationship tenure and next-employer availability use processed relationships. The annual separation rate uses the model relationship periods before training.

Command reference
Option Required / default Source-defined effect
--config PATH Required Analyses YAML config.
--mode VALUE Config sample_description.default_score_mode, else full_training Optional recorded mode; the source does not restrict values at argument parsing.
Option Required / default Source-defined effect
--task sample-description Required Selects the CPU analyses task; the wrapper rejects --mode.
--config CONFIG.yaml Required Resolves the analyses config passed to the pipeline.
--partition, --time, --mem, --cpus, --job-name, --dependency, --dry-run Wrapper defaults Standard CPU overrides; default is 8 CPUs, 128 GB, and 12 hours.
cd /hct-tower/analyses
Rscript --vanilla pipelines/sample_description.R \
  --config config/config_one_percent.yaml
cd /hct-tower
slurm/submit.sh --task sample-description \
  --config config_full.yaml

The configured mode is recorded for reproducibility, but the current table does not open the score dataset. Compustat rows are intentionally deferred.