Eddie Elizondo | No Hype Analytics blog homepage
Most of my work is building models to make decisions that manage uncertainty (1). For the last few years, LLMs have been added to the toolbox, primarily in the development workflow, but I was not (and am still not) enthusiastic how frequently LLMs of the “chatbot” variety would appear as a component of my models. They opened new potential use cases (hence the excitement), whether that’s new avenues for automation or reducing coding time, but for existing use cases, particularly part of an active pipeline in models that help make decisions to manage uncertainty, chatbot LLMs are generally not applicable. As these models generally use structured data for input, processing, and output, chatbot LLMs are not a better tool for these jobs, beyond making development a bit faster.
However, I think that’s finally changing as of September 2026, with the arrival of structured output LLM models, like TypeSafe’s Jev and the fast followers, like OpenAI’s Decisions API and Cloudfare’s Clef. TypeSafe is trying to brand these as “system one” models based on an idea by behavioral economst Daniel Kahneman; here is how TypeSafe explains them.
These new structured output models are still LLMs in their architecture and ability to ingest unstructured data, but instead of free text output, they are built to give two structured outputs: a numerical answer and a degree of confidence, like a probability. In essence, it’s a classifier that outputs a binary choice or ranking. Detractors may say that this has been machine learning lecture 1 for over a decade, which although true, misses the point: their new value is they do a decent job with no training, creating a zero-shot classifier for everyone. This opens an explosion of greater classification use, whether on a standalone basis or, especially, as a component in a modeling pipeline, like one part in a complex modeling workflow or as features in a predictive model. And they are fast and cheap to run.
The very low token cost makes these new LLMs practical in a wide variety of use cases, including to replace classification models in high volume applications when these general LLMs are good enough. Reducing time for development, model training, and (especially) model maintenance to prevent drift makes the LLMs even more attractive with their low fully loaded cost. Their speed could make them usable even in online systems.
I experimented with using structured output capabilities on traditional chatbot LLM models over the years, which was clunky and obviously a manipulated way to use the models. (If only I had also recognized the need to build a better LLM!) A model purposely designed to create structured outputs will be superior in every way: quality, speed, cost.
So, big picture, I think this is an inflection point for the ubiquity of LLMs in all analytics / data science applications, which will shift the daily work of most data science teams.
LLMs will be common in analytics / data science production pipelines
While these “structured output” LLM models won’t perform as well as fine-tuned classification models, there are enough reasons why they will supplant many fine-tuned classification models and be integrated other modeling pipelines.
- It will be “good enough” to replace fine-tuned classification models in many applications, especially considering there is near-zero development and model drift (or business requirements) maintenance required. This is a common failure mode in many models today.
- As a component of a modeling pipeline, it will make larger models better, by taking care of easy cases, routing to handle cases, improving with feature engineering, etc. I think this is the largest value of these models. Cases where developing and maintaining a classification model for a minor component was previously not worth the time expense can now be revisited.
So I see a fundamental shift in how LLMs will now interact across quantitative models that help us manage uncertainty.
Analytics and modeling teams (data science) will need to orchestrate, not reinvent
Many companies benefit the most from relatively simple, low maintenance models that enable broader use of analytics across functions. They do not need experts constantly building and maintaining a large library of bespoke models. They need small teams that solve standard problems with specificity and minimal maintenance, to rapidly move on to the next thing.
For these teams, and therefore the role of most data scientists, their value will be derived from clever and purposeful implementations of existing tools in a production pipeline that lead to minimal maintenance, not model building with heavy customization. The tools available increase every day. And vibe coding a solution with a traditional LLM chatbot will often create an over-engineered, difficult to maintain model - LLMs are best at generating new text, after all.
A distinguisher for data scientists is how to use out-of-the-box tools and light approaches to their maximum potential, by effectively integrating the broader context: the problem being solved, how much value it will drive vs difficulty of build, what resources will be devoted to maintenance, etc.
Experience and mentorship, or at least access to them for newer data scientists, will become even more important. Universities are good at teaching theory, but not the deployment and engineering realities that are gained from experience. This type of experience is not something LLMs can solve, at least yet.
Certainly some companies, whether because of their size or their unique business model, will derive significant value from custom/bespoke modeling in targeted cases. But that is the exception, not the norm. And bespoke building, as a percentage of work will likely become smaller and smaller as these general models that require no training get better.
Footnotes
(1) I’ll keep “models to make decisions that manage uncertainty” intentionally functional, since how this has been branded over time is a separate (unfortunate) issue dependent on the “term of the day” or the audience. It includes everything from operations research to statistics to machine learning to heuristics built using structured quantitative thinking to LLM-integrated models. “Terms of the day” have been data science or advanced analytics or now “AI” (when used in a general quantitative modeling sense). Operations research (OR) has persisted, covering applications like optimization and simulation, but OR has struggled from a lack of public awareness and marketing. INFORMS, the industry organization for OR, which I think may struggle to maintain professional relevance in the coming years, should strongly brand and market the timelessness and durable value of OR instead of tying its relevance to a fungible “term of the day”. The term “data science” which never had a durable definition and purpose, will probably disappear soon.
Questions, thoughts, feedback, comments? Please get in touch: eddie@betteroptima.com