Skip to content

Processed-data build workflow

This is the ordered, full-coverage build runbook. Start only after the vendor files documented under Source data are mounted at their configured paths. Vendor files are inputs: do not overwrite them during a rebuild.

Run each family in the order below. Commands are shown relative to the owning container's project directory: /hct-tower for shared Python commands and /hct-tower/analyses for R commands.

Order Family Why it comes here Runbook
1 Employment Creates the canonical Revelio workers, firms, and employment histories used throughout the project. Employment workflow
2 Accounting and market Creates Compustat products and firm mappings used by public-firm and leadership work. Accounting and market workflow
3 Patent Creates an independent patent panel; it is not an input to employment or model training. Patent workflow
4 Executive and governance Depends on completed employment products and the Revelio--Compustat mapping. Executive and governance workflow

After these stages, follow the model workflow, then the model-output workflow, and finally the analysis-dataset workflow. That sequence takes the project from mounted vendor files to all currently runnable project products.

Coverage and destructive rebuilds

The commands below describe a clean full rebuild. Some builders replace existing output directories. Build and validate full first; derive one_percent and samples only afterwards. Do not mix old downstream outputs with newly rebuilt upstream data.