Project-raw Compustat and identifier links
This stage converts immutable WRDS Funda, CCM, and SEC Suite historical-link SAS deliveries into
project-owned Parquet files. It selects only the variables declared in
shared/config/config_full.yaml, keeping ingestion memory bounded without
embedding a vendor-specific field list in code.
flowchart LR
funda[(Compustat Funda SAS)] --> stage[Stream configured fields]
ccm[(CCM link-table SAS)] --> stage
wcik[(WRDS historical CIK--GVKEY SAS)] --> stage
stage --> annual[(Project-raw annual Compustat)]
stage --> links[(Project-raw CCM links)]
stage --> ciklinks[(Project-raw historical CIK--GVKEY links)]
classDef source fill:#e8eef7,stroke:#52739e,stroke-width:2px
classDef support fill:#fff0cc,stroke:#a66b00,stroke-width:2px
class funda,ccm,wcik source
class annual,links,ciklinks support
Inputs
| Dataset |
Location |
| Compustat and CRSP vendor extracts |
/data/compustat/funda.sas7bdat, /data/compustat/ccmxpf_linktable.sas7bdat, and /data/compustat/wciklink_gvkey.sas7bdat |
Build
| Component |
Location |
| Builder |
shared/data/build_raw_compustat.py |
| Runtime |
hct-tower-model-cuda-… (CPU execution) |
| Entry point |
python -m shared.data.build_raw_compustat --config shared/config/config_full.yaml |
Output
| Dataset |
File / location |
Grain |
Key and variables |
| Project-raw annual Compustat |
compustat_annual.parquet in /data/full/raw/ |
Vendor financial statement record |
Identifiers and reporting metadata; financial fields including at, che, sale, xrd, csho, and prcc_f |
| Project-raw CCM links |
ccm_linktable.parquet in /data/full/raw/ |
GVKEY--security link interval |
gvkey, lpermno, lpermco, link type/primary status, and link dates |
| Project-raw historical CIK--GVKEY links |
wciklink_gvkey.parquet in /data/full/raw/ |
CIK--GVKEY-source interval |
CIK, GVKEY, source and quality labels, SEC/link dates, company names, and filing-count diagnostics |
The Funda output intentionally preserves all selected source records. The
processed Compustat builder applies the
industrial, domestic, standardized, consolidated USD record filter and chooses
one record per firm fiscal year. The CCM output supersedes the legacy CSV extract and is used by the
mapping and patent builders. The WRDS SEC Suite output preserves historical CIK--GVKEY evidence for
the audited Revelio--Compustat mapping.