Compustat-CRSP MQ dataset
This reusable public-firm annual panel attaches Revelio MQ where available,
monthly CRSP information as of the fiscal year-end, and filed and issued KPSS
patent measures. The full_training version has one row for each processed
Compustat (gvkey, fyear). A worker- or time-profile training version
preserves each Compustat firm-year in both sample statuses, yielding one row
per (gvkey, fyear, sample_status) so the MQ from each status remains separate.
It is intentionally raw: analyses create ratios, logs, cumulative outcomes,
winsorisation, and samples themselves. To keep analyses dependent on one
reusable file, it also carries coverage-aware raw patent leads and lags through
t+5 and t-5.
flowchart TB
comp[(Processed annual Compustat)]
mapping[(RCID-GVKEY mapping)]
mq[(RCID-year MQ)]
ccm[(Project-raw CCM links)]
crsp[(Processed monthly CRSP)]
patents[(Patent firm-year by GVKEY)]
output[(Compustat-CRSP MQ panel)]
comp --> output
mapping --> output
mq --> output
ccm --> output
crsp --> output
patents --> output
classDef input fill:#e8eef7,stroke:#52739e,stroke-width:2px
classDef analysis fill:#eee5f5,stroke:#76528c,stroke-width:2px
class comp,mapping,mq,ccm,crsp,patents input
class output analysis
Inputs
| Dataset |
Location |
Role |
| Processed annual Compustat |
/data/full/processed/compustat_annual.parquet |
Panel spine; one row per (gvkey, fyear) |
| RCID-GVKEY mapping |
/data/full/processed/mapping_rcid_gvkey.parquet |
Preferred one-to-one public-firm mapping |
RCID-year MQ (training) |
/data/<coverage>/analyses/profile_<profile>/training/rcid_year_mq.parquet |
Annual employer MQ moments by sample_status; build this before Compustat-CRSP MQ |
RCID-year MQ (full_training) |
/data/<coverage>/analyses/full_training/rcid_year_mq.parquet |
Annual employer MQ moments |
| Raw CCM link table |
/data/full/raw/ccm_linktable.parquet |
Dated GVKEY-PERMNO links |
| Processed monthly CRSP |
/data/full/processed/crsp.parquet |
Returns and volatility as of fiscal year-end |
| Patent firm-year by GVKEY |
/data/full/processed/patent_firm_year.parquet |
Filed and issued patent measures |
Build
| Component |
Location |
| Builder |
analyses/modules/build_compustat_crsp_mq.R |
| Runtime |
hct-tower-analyses-cpu |
| Entry point |
analyses/pipelines/build_compustat_crsp_mq.R |
Output
| Dataset |
File / location |
Grain |
Compustat-CRSP MQ panel (training, worker profile) |
compustat_crsp_mq.parquet in /data/<coverage>/analyses/profile_worker/training/ |
Unique (gvkey, fyear, sample_status) |
Compustat-CRSP MQ panel (training, time profile) |
compustat_crsp_mq.parquet in /data/<coverage>/analyses/profile_time/training/ |
Unique (gvkey, fyear, sample_status) |
Compustat-CRSP MQ panel (full_training) |
compustat_crsp_mq.parquet in /data/<coverage>/analyses/full_training/ |
Unique (gvkey, fyear) |
Join rules
- The original zero-padded Compustat GVKEY is retained. A normalized form is
used internally to join the mapping and patent data.
- MQ joins on the mapped RCID and fiscal year (
rcid_year_mq.year = fyear);
profile builds also join on sample_status. In training mode, the Compustat
spine is crossed with in_sample and out_of_sample before joins. Thus every
firm-year remains in both status rows; MQ is missing for a status with no
scored relationships. Analyses using a profile output should filter the
intended sample_status before estimating results.
- CCM links use
LINKTYPE in {LU, LC} and LINKPRIM in {P, C}. Valid
links are evaluated at datadate; overlapping links use the latest start.
- CRSP uses the latest monthly observation at or before
datadate.
- Patent fields retain both timing conventions:
*_filed and *_issued, for
the current fiscal year and exact t-5 through t+5 fiscal years. A missing
GVKEY-year patent record is represented as zero activity only within the
fully covered KPSS window. Outside that window it is missing, with matching
*_available flags.
data.patent_last_complete_year records the final fully covered source year.
It is currently 2024: the KPSS 2025 delivery has filing observations only
through 2025-08-21, so 2025 cannot be treated as a full filing year.