Company-year match quality¶
This dataset averages worker-firm-period MQ within company-year and attaches firm identifiers and characteristics. It includes public and private firms.
Retirable
Read only by the CHRO and deals branches. Can be removed once those
migrate to rcid_year_mq.parquet.
Legacy workflow
This builder still references the retired
mapping_revelio_compustat.parquet input and is not runnable in the
supported workflow. Use rcid_year_mq.parquet and
mapping_rcid_gvkey.parquet for new analyses.
flowchart TB
scores[(Worker-firm-period scores)]
firms[(Firms)]
mapping[(Revelio-Compustat mapping)]
comp[(Processed annual Compustat)]
build[Average MQ by firm-year<br/>and attach firm fields]
output[(Company-year MQ)]
scores --> build
firms --> build
mapping --> build
comp --> build
build --> output
classDef source fill:#e8eef7,stroke:#52739e
classDef process fill:#fff0cc,stroke:#a66b00
classDef data fill:#eee5f5,stroke:#76528c
class scores,firms,mapping,comp source
class build process
class output data
Inputs
| Dataset | Location |
|---|---|
| Worker-firm-period scores, for the selected mode | /data/<coverage>/analyses/<mode>/worker_firm_period_scores/ |
| Processed firms | /data/<coverage>/processed/companies_processed.parquet |
| Revelio--Compustat mapping | /data/full/processed/mapping_revelio_compustat.parquet |
| Processed annual Compustat | /data/full/processed/compustat_annual.parquet |
Build
| Component | Location |
|---|---|
| Builder | analyses/data_build/build_mq_datasets.py |
| Runtime | hct-tower-model-cuda-… (CPU execution) |
| Entry point | python analyses/data_build/build_mq_datasets.py --step mq |
Output
| Dataset | File / location | Grain | Key and variables |
|---|---|---|---|
| Company-year match quality | company_year_mq.parquet in /data/<coverage>/analyses/<mode>/ |
Firm-year | Key: company_id, yearVariables: n_scores, mean_match_score, gvkey, company_name, observed_size_bucket |
Core columns are company_id, gvkey, year, n_scores, and
mean_match_score. The builder streams score shards, aggregates the score sum
and count, joins processed company fields, maps public firms to gvkey, and
attaches available annual Compustat fields. gvkey remains missing for private
or unmatched firms.
The current builder uses the shared ZFS host path directly:
Build this dataset for each scoring mode consumed by its public-firm, executive, and deal-analysis users. Destination-rank scoring instead uses the shared RCID-year MQ dataset.
Command reference
| Option | Required / default | Source-defined effect |
|---|---|---|
--sample one_percent\|full |
Required | Selects paths hard-coded by the Python builder. |
--model full_training\|training |
full_training |
Selects the score mode. |
--step mq\|dataset\|all |
all |
Runs company-MQ aggregation, prints the table-89 dataset handoff, or both. |
| Interface | Availability |
|---|---|
slurm/submit.sh task |
No project task invokes this Python builder. |
There is currently no project Slurm wrapper for this builder. Run the direct command in a CPU job using the model CUDA image.
Check uniqueness of (company_id, year), positive n_scores, and public-firm
mapping coverage. Consumers include CHRO data with model
MQ,
goodwill target MQ, and related public-firm
analyses.