LLM methods
Text as data, embeddings, prompting and fine-tuning in empirical work, and the validity problems that come with them.
Test your LLM for look-ahead bias before you trust a backtest
A model trained on text through 2024 has already read how 2019 turned out. Four checks show whether your result survives that.
Keep paying for call transcripts, but only for the history, the identifiers and the rights
An agent now does the cleaning, splitting and scoring that used to justify a data budget. What it cannot produce is history, identifiers and the right to use the text.
Seven LLMs scoring the same earnings calls agree with a rank correlation of 0.52
Across thirteen text measures on S&P 500 transcripts, the choice of model changes the size, sign and significance of downstream coefficients.
LLM literature reviews hold up for broad claims and break down for paper-level ones
A test on economics papers that use rainfall as an instrument finds accuracy falls as the reading task needs more context.