EVALUATING LARGE LANGUAGE MODELS FOR FINANCIAL FORECASTING: ACCURACY, ROBUSTNESS, AND MODEL RISK

Abstract

Speaking of financial forecasting, Large Language Models (LLMs) have been the game changers. Coming fast into the world of financial analysis, these models can mangle through massive amounts of unstructured data, uncovering intricate patterns and forecasting predictions that have made them the go-to tool in modern financial analytics, according to Filippi and Motyl in their 2024 paper and Wang et al. In theirs, in 2024.

However, the practical application of LLMs in financial forecasting is marred by quite a few nasty issues.

Accuracy in predictions, steadiness in the face of market volatility and model risk are just a few examples. Balakrishnan et al. In 2025, Dong and Zhou in 2024 and Cummins et al. In 2023 all wrote about the significant problems posed by LLMs.

This paper provides an assessment of the functionality of LLMs in financial predictions from a multi-dimensional perspective, which concentrates on three main factors, namely, accuracy, robustness, and management of model risks. The benchmark metrics and comparative studies between the current LLM architectures and hybrid models are used to assess their accuracy, which reveals the strengths and weaknesses of the models (Balakrishnan et al., 2025; Ozupek et al., 2024; Strobel et al., 2024). Stress-testing and scenario analysis are used to study robustness, and are tested on how the models react to extreme market conditions and variability of data (Casini and Landes, 2024; Labijak-Kowalska and Kadzinski, 2023; Sorourkhah and Edalatpanah, 2022). It overcomes model risk by identifying the sources of possible bias, overfitting, and interpretability problems and presents a risk-reduction framework to use in financial decision-making (Dodgson, 2020; Singh et al., 2023; Yoshiura et al., 2023).

With the combination of predictive analytics, robustness assessment, and model risk assessment, this paper provides an extensive model of the implementation of LLMs in financial forecasting. The results can be applied to the field of academic research and practice, teaching the financial institution and the professional community to use AI-based forecasting tools to their advantage and reduce the risks involved.