Skip to content

Company-year match quality

This dataset averages worker-firm-period MQ within company-year and attaches firm identifiers and characteristics. It includes public and private firms.

Retirable

Read only by the CHRO and deals branches. Can be removed once those migrate to rcid_year_mq.parquet.

Legacy workflow

This builder still references the retired mapping_revelio_compustat.parquet input and is not runnable in the supported workflow. Use rcid_year_mq.parquet and mapping_rcid_gvkey.parquet for new analyses.

flowchart TB
    scores[(Worker-firm-period scores)]
    firms[(Firms)]
    mapping[(Revelio-Compustat mapping)]
    comp[(Processed annual Compustat)]
    build[Average MQ by firm-year<br/>and attach firm fields]
    output[(Company-year MQ)]

    scores --> build
    firms --> build
    mapping --> build
    comp --> build
    build --> output

    classDef source fill:#e8eef7,stroke:#52739e
    classDef process fill:#fff0cc,stroke:#a66b00
    classDef data fill:#eee5f5,stroke:#76528c
    class scores,firms,mapping,comp source
    class build process
    class output data
Inputs
Dataset Location
Worker-firm-period scores, for the selected mode /data/<coverage>/analyses/<mode>/worker_firm_period_scores/
Processed firms /data/<coverage>/processed/companies_processed.parquet
Revelio--Compustat mapping /data/full/processed/mapping_revelio_compustat.parquet
Processed annual Compustat /data/full/processed/compustat_annual.parquet
Build
Component Location
Builder analyses/data_build/build_mq_datasets.py
Runtime hct-tower-model-cuda-… (CPU execution)
Entry point python analyses/data_build/build_mq_datasets.py --step mq
Output
Dataset File / location Grain Key and variables
Company-year match quality company_year_mq.parquet in /data/<coverage>/analyses/<mode>/ Firm-year Key: company_id, year
Variables: n_scores, mean_match_score, gvkey, company_name, observed_size_bucket

Core columns are company_id, gvkey, year, n_scores, and mean_match_score. The builder streams score shards, aggregates the score sum and count, joins processed company fields, maps public firms to gvkey, and attaches available annual Compustat fields. gvkey remains missing for private or unmatched firms.

The current builder uses the shared ZFS host path directly:

Build this dataset for each scoring mode consumed by its public-firm, executive, and deal-analysis users. Destination-rank scoring instead uses the shared RCID-year MQ dataset.

Command reference
Option Required / default Source-defined effect
--sample one_percent\|full Required Selects paths hard-coded by the Python builder.
--model full_training\|training full_training Selects the score mode.
--step mq\|dataset\|all all Runs company-MQ aggregation, prints the table-89 dataset handoff, or both.
Interface Availability
slurm/submit.sh task No project task invokes this Python builder.
cd /hct-tower
python analyses/data_build/build_mq_datasets.py \
  --sample one_percent --model training --step mq
python analyses/data_build/build_mq_datasets.py \
  --sample one_percent --model full_training --step mq

There is currently no project Slurm wrapper for this builder. Run the direct command in a CPU job using the model CUDA image.

Check uniqueness of (company_id, year), positive n_scores, and public-firm mapping coverage. Consumers include CHRO data with model MQ, goodwill target MQ, and related public-firm analyses.