Skip to content

XBRL deals regression data

This builder creates the clean, non-counterfactual base panel for the goodwill analysis. It starts from the current reprocessed XBRL deal panel, keeps goodwill-bearing records, attaches selected target and acquirer RCIDs and GVKEYs, and adds observed pre-deal MQ and accounting controls. The separate counterfactual MQ scorer uses this panel's stable deal key and mappings.

flowchart LR
    xbrl[(XBRL goodwill deals)]
    maps[(Selected RCID mappings)]
    mq[(RCID-year MQ)]
    comp[(Annual Compustat)]
    build[Build deal panel]
    panel[(XBRL deals regression data)]

    xbrl --> build
    maps --> build
    mq --> build
    comp --> build
    build --> panel

    classDef input fill:#e8eef7,stroke:#52739e,stroke-width:2px
    classDef process fill:#fff0d8,stroke:#b7791f,stroke-width:2px
    classDef output fill:#e2f2e7,stroke:#3b7a4b,stroke-width:2px
    class xbrl,maps,mq,comp input
    class build process
    class panel output
Inputs
Dataset Location Join role
Clean XBRL deal panel /data/full/processed/raw/xbrl_deals.parquet Goodwill-bearing issuer–target records assembled from reprocessed 10-K/10-Q facts
XBRL target and acquirer mappings /data/full/processed/mapping_xbrl_targets_revelio.parquet; /data/full/processed/mapping_xbrl_acquirers_revelio.parquet Selected one-to-one links to target/acquirer RCIDs
RCID–GVKEY mapping /data/full/processed/mapping_rcid_gvkey.parquet Selected RCID-to-Compustat link
RCID-year match quality Full-training: /data/<coverage>/analyses/full_training/rcid_year_mq.parquet; profile training: /data/<coverage>/analyses/profile_<worker|time>/training/rcid_year_mq.parquet Observed target/acquirer MQ at year − 1; profile builds match on sample_status
Processed annual Compustat /data/full/processed/compustat_annual.parquet Latest accounting observation available by filing date
Build
Component Location Purpose
Builder analyses/modules/build_xbrl_deals_regression_data.R Performs all non-counterfactual joins with DuckDB
Entry point analyses/pipelines/build_xbrl_deals_regression_data.R Direct CPU execution
Slurm task build-xbrl-deals-regression-data CPU analyses-image execution
Output
Dataset Location Grain Content
XBRL deals regression data Full-training: /data/<coverage>/analyses/full_training/xbrl_deals_regression_data.parquet; worker/time: /data/<coverage>/analyses/profile_<worker|time>/training/xbrl_deals_regression_data.parquet Full-training: one row per deal; profile training: one row per deal and sample_status XBRL facts and dates, target/acquirer RCIDs and GVKEYs, pre-deal MQ, and prefixed annual Compustat variables
Build metadata Beside the Parquet as xbrl_deals_regression_data_metadata.yaml One build Inputs, timing conventions, coverage, row count, and creation details

Join and timing contract

The panel is intentionally inclusive after the goodwill restriction: an unmatched target, acquirer, RCID, or GVKEY leaves only that side's derived fields missing; it does not remove the XBRL record. xbrl_deal_target_id is a deterministic hash of the issuer, target label, and first filing identifier. It is unique in full-training output and unique within each sample_status in profile-training output. It is the join key for the counterfactual scoring dataset. There is no year cutoff in this builder; available coverage follows the source deal panel.

The builder filters each mapping file to its selected one-to-one rows and keeps only the resulting RCIDs and GVKEYs. It does not carry mapping confidence or audit fields; those remain available in the reusable mapping Parquets.

Observed MQ is measured in the calendar year before the deal panel's first_fiscal_year: observed_mq_year = xbrl_year - 1. The regression's year fixed effect instead uses the year of the selected goodwill fact's reference date (xbrl_goodwill_date); the two years need not coincide. The complete RCID-year aggregate is kept for each side, including target_mq_mean/acquirer_mq_mean, support, spread, and relationship counts. Annual Compustat is an as-of join: for each side the builder selects the most recent datadate no later than xbrl_date_filed and records the resulting accounting age in days. Every processed annual Compustat variable is retained with target_compustat_ or acquirer_compustat_ prefixes.

For profile training, the builder creates both in_sample and out_of_sample rows for each deal, then joins target and acquirer MQ only to the matching status from that profile's RCID-year dataset. This gives each profile a separate output file and prevents MQ from one status entering the other. The full-training mode keeps the original one-row-per-deal shape. The builder does not attach CCM, PERMNO, or CRSP data.

Command reference
Option Required / default Source-defined effect
--config PATH config/config_one_percent.yaml Analyses YAML configuration containing source and output paths
--mode training --profile worker|time Full-training mode by default Builds one profile-specific file with both in_sample and out_of_sample rows
--mode full_training Default Builds the existing full-training panel; omit --profile
--overwrite off Explicitly replace the output Parquet and metadata
Option Required / default Source-defined effect
--task build-xbrl-deals-regression-data Required Selects the CPU analyses task
--mode training --profile worker|time Full-training mode by default Builds one profile-specific file with both sample statuses
--mode full_training Default Builds the full-training panel; omit --profile
--config CONFIG.yaml Required Resolves the analyses config passed to the builder
--overwrite off Rebuilds an existing output
--partition, --time, --mem, --cpus, --job-name, --dependency, --exclude, --dry-run Wrapper defaults Standard Slurm resource and submission overrides; default is 8 CPUs, 32 GB, and one hour
cd /hct-tower/analyses
# Full-training panel
Rscript --vanilla pipelines/build_xbrl_deals_regression_data.R \
  --config config/config_full.yaml --mode full_training --overwrite

# Worker-profile panel
Rscript --vanilla pipelines/build_xbrl_deals_regression_data.R \
  --config config/config_full.yaml --mode training --profile worker --overwrite

# Time-profile panel
Rscript --vanilla pipelines/build_xbrl_deals_regression_data.R \
  --config config/config_full.yaml --mode training --profile time --overwrite

These commands rebuild existing outputs, so they include --overwrite.

cd /hct-tower
# Full-training panel
slurm/submit.sh --task build-xbrl-deals-regression-data \
  --mode full_training --config config_full.yaml --overwrite

# Worker-profile panel
slurm/submit.sh --task build-xbrl-deals-regression-data \
  --mode training --profile worker --config config_full.yaml --overwrite

# Time-profile panel
slurm/submit.sh --task build-xbrl-deals-regression-data \
  --mode training --profile time --config config_full.yaml --overwrite