Skip to content

Project-raw CRSP stock and security-history data

This stage streams the WRDS CRSP MSF and Stocknames V2 SAS deliveries into project-owned Parquet inputs. The selected monthly return fields and complete security identifier history are configured in shared/config/config_full.yaml.

flowchart LR
    msf[(CRSP MSF SAS)] --> stage[Stream configured fields]
    stocknames[(CRSP Stocknames V2 SAS)] --> stage
    stage --> monthly[(Project-raw CRSP monthly)]
    stage --> history[(Project-raw CRSP security history)]

    classDef source_style fill:#e8eef7,stroke:#52739e,stroke-width:2px
    classDef support fill:#fff0cc,stroke:#a66b00,stroke-width:2px
    class msf,stocknames source_style
    class monthly,history support
Inputs
Dataset Location
Compustat and CRSP vendor extracts /data/compustat/msf_v2.sas7bdat, /data/compustat/stocknames_v2.sas7bdat
Build
Component Location
Builder shared/data/build_raw_crsp.py
Runtime hct-tower-model-cuda-… (CPU execution)
Entry point python -m shared.data.build_raw_crsp --config shared/config/config_full.yaml
Output
Dataset File / location Grain Key and variables
Project-raw CRSP monthly crsp_monthly.parquet in /data/full/raw/ Security-month permno, mthcaldt, total and ex-distribution returns, price, shares, and security identifiers
Project-raw CRSP security history crsp_stocknames.parquet in /data/full/raw/ Security-name interval permno, namedt, nameenddt, historical cusip, ticker, issuer name, exchange, and security-status fields

The monthly input feeds processed CRSP, which produces trailing annualized return metrics at the security-month level. The security history is retained as the dated CUSIP source for firm-identifier mapping.

cd /hct-tower
python -m shared.data.build_raw_crsp \
  --config shared/config/config_full.yaml
slurm/submit.sh --task build-raw-crsp --config config_full.yaml