Wage regressions¶
This analysis relates log wages to MQ across model samples, fixed-effect specifications, and relationship-tenure groups.
flowchart LR
data[(Available disaggregated data<br/>for five cohorts)]
fit[[Fit five specifications<br/>in three tenure contexts per cohort]]
models[(Compact fits, two-way covariance,<br/>and metrics)]
csv[(wage_regressions_metrics.csv)]
latex[/wage_regressions.tex/]
data --> fit --> models
models --> csv --> latex
classDef input fill:#eee5f5,stroke:#76528c,stroke-width:2px
classDef code fill:#f3f3f3,stroke:#666,stroke-width:2px
classDef support fill:#fff0cc,stroke:#a66b00,stroke-width:2px
classDef tableOutput fill:#e2f2e7,stroke:#3b7a4b,stroke-width:2px
class data input
class fit code
class models,csv support
class latex tableOutput
Inputs
| Dataset | Location |
|---|---|
| Disaggregated regression data | Full-training: /data/<coverage>/analyses/full_training/disaggregated_regression_data/; worker profile: /data/<coverage>/analyses/profile_worker/training/disaggregated_regression_data/; time profile: /data/<coverage>/analyses/profile_time/training/disaggregated_regression_data/ |
Build
| Component | Location |
|---|---|
| Builder | analyses/modules/wage_regressions.R |
| Runtime | hct-tower-analyses-cpu |
| Entry point | analyses/pipelines/wage_regressions.R / wage-regressions |
Output
| Dataset | File / location | Grain and variables |
|---|---|---|
| Wage-regression results | Metrics, compact fits, and wage_regressions.tex in /artifacts/<coverage>/analyses/wage_regression/<cohort>/ |
Cohorts are full_training_in_sample, profile_worker_in_sample, profile_worker_out_of_sample, profile_time_in_sample, and profile_time_out_of_sample; unavailable inputs are skipped |
With no selectors, one invocation attempts all five tables:
- full-training in-sample;
- worker-profile in-sample and out-of-sample;
- time-profile in-sample and out-of-sample.
The pipeline logs and skips a cohort if its configured data directory or
Parquet shards are unavailable, then continues with the remaining cohorts.
If --shards requests a shard that is absent for a cohort, that cohort is also
skipped. Model or data errors after a cohort's inputs are found still stop the
run.
Each table contains panels for all relationships, tenure 0–4, and tenure 5+.
Within each panel, five columns add observable controls, firm fixed effects,
worker fixed effects, and worker-plus-firm fixed effects to the MQ-only model.
Standard errors are clustered two ways by worker and firm. Saved metrics also
record the wage interpretation of a +0.1 MQ change.
All available calendar years are included, including 2024.
Command reference
| Option | Required / default | Source-defined effect |
|---|---|---|
--config PATH |
config/config_one_percent.yaml |
Analyses YAML config. |
--mode training\|full_training |
Optional | Restricts processing to the selected mode. full_training permits only in_sample. |
--profile worker\|time |
Optional | Restricts training-profile processing to one profile. Omit for both profiles. |
--sample in_sample\|out_of_sample |
Optional | Restricts processing to one sample status. Omit to process the available statuses. |
--shards ID[,ID,…] |
Unset | Restricts to user shards and writes isolated shard-scoped outputs. |
--rerun |
Off | Refits every saved model instead of reusing valid persisted fits. |
| Option | Required / default | Source-defined effect |
|---|---|---|
--task wage-regressions |
Required | Selects the CPU analyses task. |
--config CONFIG.yaml, --mode, --profile, --sample, --shards, --rerun |
Config required; selectors optional | Passed to the R pipeline. Without selectors the pipeline attempts all five cohorts and skips unavailable inputs. |
--partition, --time, --mem, --cpus, --job-name, --dependency, --dry-run |
Wrapper defaults | Standard CPU overrides; default is 8 CPUs, 800 GB, and two days. |
A normal rerun reuses valid model and metric bundles and can format the table
without reopening source data. --rerun refits every specification.
Sharded runs¶
Use --shards with comma-separated user-shard identifiers for a scoped,
exploratory result. Outputs are isolated under shards/user_<ids>/ beneath
each cohort. You can restrict the run, for example, to the time-profile
out-of-sample table with --mode training --profile time --sample out_of_sample.