XBRL deals regression data¶
This builder creates the clean, non-counterfactual base panel for the goodwill analysis. It starts from the current reprocessed XBRL deal panel, keeps goodwill-bearing records, attaches selected target and acquirer RCIDs and GVKEYs, and adds observed pre-deal MQ and accounting controls. The separate counterfactual MQ scorer uses this panel's stable deal key and mappings.
flowchart LR
xbrl[(XBRL goodwill deals)]
maps[(Selected RCID mappings)]
mq[(RCID-year MQ)]
comp[(Annual Compustat)]
build[Build deal panel]
panel[(XBRL deals regression data)]
xbrl --> build
maps --> build
mq --> build
comp --> build
build --> panel
classDef input fill:#e8eef7,stroke:#52739e,stroke-width:2px
classDef process fill:#fff0d8,stroke:#b7791f,stroke-width:2px
classDef output fill:#e2f2e7,stroke:#3b7a4b,stroke-width:2px
class xbrl,maps,mq,comp input
class build process
class panel output
Inputs
| Dataset | Location | Join role |
|---|---|---|
| Clean XBRL deal panel | /data/full/processed/raw/xbrl_deals.parquet |
Goodwill-bearing issuer–target records assembled from reprocessed 10-K/10-Q facts |
| XBRL target and acquirer mappings | /data/full/processed/mapping_xbrl_targets_revelio.parquet; /data/full/processed/mapping_xbrl_acquirers_revelio.parquet |
Selected one-to-one links to target/acquirer RCIDs |
| RCID–GVKEY mapping | /data/full/processed/mapping_rcid_gvkey.parquet |
Selected RCID-to-Compustat link |
| RCID-year match quality | Full-training: /data/<coverage>/analyses/full_training/rcid_year_mq.parquet; profile training: /data/<coverage>/analyses/profile_<worker|time>/training/rcid_year_mq.parquet |
Observed target/acquirer MQ at year − 1; profile builds match on sample_status |
| Processed annual Compustat | /data/full/processed/compustat_annual.parquet |
Latest accounting observation available by filing date |
Build
| Component | Location | Purpose |
|---|---|---|
| Builder | analyses/modules/build_xbrl_deals_regression_data.R |
Performs all non-counterfactual joins with DuckDB |
| Entry point | analyses/pipelines/build_xbrl_deals_regression_data.R |
Direct CPU execution |
| Slurm task | build-xbrl-deals-regression-data |
CPU analyses-image execution |
Output
| Dataset | Location | Grain | Content |
|---|---|---|---|
| XBRL deals regression data | Full-training: /data/<coverage>/analyses/full_training/xbrl_deals_regression_data.parquet; worker/time: /data/<coverage>/analyses/profile_<worker|time>/training/xbrl_deals_regression_data.parquet |
Full-training: one row per deal; profile training: one row per deal and sample_status |
XBRL facts and dates, target/acquirer RCIDs and GVKEYs, pre-deal MQ, and prefixed annual Compustat variables |
| Build metadata | Beside the Parquet as xbrl_deals_regression_data_metadata.yaml |
One build | Inputs, timing conventions, coverage, row count, and creation details |
Join and timing contract¶
The panel is intentionally inclusive after the goodwill restriction: an
unmatched target, acquirer, RCID, or GVKEY leaves only that side's derived
fields missing; it does not remove the XBRL record. xbrl_deal_target_id is a
deterministic hash of the issuer, target label, and first filing identifier.
It is unique in full-training output and unique within each sample_status in
profile-training output. It is the join key for the counterfactual scoring
dataset. There is no year cutoff in this builder; available coverage follows
the source deal panel.
The builder filters each mapping file to its selected one-to-one rows and keeps only the resulting RCIDs and GVKEYs. It does not carry mapping confidence or audit fields; those remain available in the reusable mapping Parquets.
Observed MQ is measured in the calendar year before the deal panel's
first_fiscal_year: observed_mq_year = xbrl_year - 1. The regression's year
fixed effect instead uses the year of the selected goodwill fact's reference
date (xbrl_goodwill_date); the two years need not coincide. The complete
RCID-year aggregate is kept
for each side, including target_mq_mean/acquirer_mq_mean, support, spread,
and relationship counts. Annual Compustat is an as-of join: for each side the
builder selects the most recent datadate no later than xbrl_date_filed and
records the resulting accounting age in days. Every processed annual
Compustat variable is retained with target_compustat_ or
acquirer_compustat_ prefixes.
For profile training, the builder creates both in_sample and out_of_sample
rows for each deal, then joins target and acquirer MQ only to the matching
status from that profile's RCID-year dataset. This gives each profile a
separate output file and prevents MQ from one status entering the other. The
full-training mode keeps the original one-row-per-deal shape. The builder
does not attach CCM, PERMNO, or CRSP data.
Command reference
| Option | Required / default | Source-defined effect |
|---|---|---|
--config PATH |
config/config_one_percent.yaml |
Analyses YAML configuration containing source and output paths |
--mode training --profile worker|time |
Full-training mode by default | Builds one profile-specific file with both in_sample and out_of_sample rows |
--mode full_training |
Default | Builds the existing full-training panel; omit --profile |
--overwrite |
off | Explicitly replace the output Parquet and metadata |
| Option | Required / default | Source-defined effect |
|---|---|---|
--task build-xbrl-deals-regression-data |
Required | Selects the CPU analyses task |
--mode training --profile worker|time |
Full-training mode by default | Builds one profile-specific file with both sample statuses |
--mode full_training |
Default | Builds the full-training panel; omit --profile |
--config CONFIG.yaml |
Required | Resolves the analyses config passed to the builder |
--overwrite |
off | Rebuilds an existing output |
--partition, --time, --mem, --cpus, --job-name, --dependency, --exclude, --dry-run |
Wrapper defaults | Standard Slurm resource and submission overrides; default is 8 CPUs, 32 GB, and one hour |
cd /hct-tower/analyses
# Full-training panel
Rscript --vanilla pipelines/build_xbrl_deals_regression_data.R \
--config config/config_full.yaml --mode full_training --overwrite
# Worker-profile panel
Rscript --vanilla pipelines/build_xbrl_deals_regression_data.R \
--config config/config_full.yaml --mode training --profile worker --overwrite
# Time-profile panel
Rscript --vanilla pipelines/build_xbrl_deals_regression_data.R \
--config config/config_full.yaml --mode training --profile time --overwrite
These commands rebuild existing outputs, so they include --overwrite.
cd /hct-tower
# Full-training panel
slurm/submit.sh --task build-xbrl-deals-regression-data \
--mode full_training --config config_full.yaml --overwrite
# Worker-profile panel
slurm/submit.sh --task build-xbrl-deals-regression-data \
--mode training --profile worker --config config_full.yaml --overwrite
# Time-profile panel
slurm/submit.sh --task build-xbrl-deals-regression-data \
--mode training --profile time --config config_full.yaml --overwrite