Eddie Elizondo | No Hype Analytics blog homepage
Many large and medium sized organizations have a dedicated team for analytics and reporting - the team that designs and maintains core dashboards and data tables and assists with requests for curating new data or custom reports to drive new initiatives. Use of AI should be deeply embedded in this function as a basic tool in every business analytics team’s toolbox. However, like any tool, there are good and bad ways to integrate it.
Good
- Governed AI provider. Many organizations do not have best-in-class AI tools available to everyone. Sometimes the latest AI tools are available only to AI developers, while the rest of the organizations is relegated to Copilot. At this point, the lack of an AI tool will slow productivity, or worse, cause unsanctioned use of non-corporate versions of AI tools that lack information security controls.
- Strong data documentation. AI can be especially great at writing SQL queries. However, without good documentation, it will be as lost as humans guessing which tables and fields to use. Strong data catalogs are necessary.
- Sample query library. Even organizations with the cleanest governed data tables will inevitably still have “rules of thumb” in their queries - standard set of fields to join on, filters to apply, etc. Ensure the AI replicates this, along with the overall “writing style” of the team, by providing or training on a sample set of great queries. These could even be the same queries powering gold-standard reports and dashboards.
- Fewer dashboards, more customized reports. Many dashboards are not used, and often dashboards are used to export data to Excel for analysis. Businesses need frequent, custom analyses to drive change. AI should mean fewer standardized dashboards and more custom reports that solve the problem at hand immediately. Reporting teams should be more enabled than ever to respond quickly and efficiently.
Bad
- AI-generated data catalogs and dictionaries. AI will not (as of now) capture the context of the data’s origin and nuances of use which are the main value of this documentation.
- AI-generated code comments. AI often creates tautological code comments, expressing code (which is already very expressive in modern programming languages) simply in words without explaining underlying concepts and intent. Please use sparingly as it risks maintaining your codebase and onboarding new team members more difficult.
- Pure vibe coding. Most of the analytics world now runs on Python which makes it very easy to code in substandard ways, AI agents need to be supervised on code quality. Code running without errors is a minimal and far from sufficient standard for accepting code. Task out work to agents and review their plan up front, and add checkpoints to monitor execution.
Most issues in the “bad” section can be avoided by actively coaching team members and reviewing their work. Many of these tasks also fall on junior team members who may not yet know what “good” looks like, exactly the type of coaching and mentorship needed from leaders. Leaders also have a clearer path to ROI in data cataloging investments (which are often deprioritized) that the “good” section relies on. Just as companies invest in trainings for employees because it improves performance, investment in building blocks like the tools and structure will generate higher productivity from AI agents.
Do you have questions, thoughts, feedback, comments? Please get in touch - I would love to hear from you: eddie@betteroptima.com