What does a typical marketing analytics project look like?
To understand the impact of AI on the day-to-day of an analyst, I think it's best to start with the question “what does an analyst do day-to-day?”.
Over the last decade of my career, one only thing has remained stoically constant when working on any analytics project - the “stages” which every project goes through. This rings true, regardless of the “type” of work and the outputs we are looking to get to.
Stop any analyst in the corridor of the office on any given day - and ask them about their day - and it will broadly fit into one of these three stages:
- Data Collection: Any analysis needs correctly defined, cleaned & validated data to work from.
- Data Specification & Modelling: Where most of the advanced work takes place - ‘where the magic happens’
- Interpretation, Visualisation & Learning: Achieving the projects “purpose” by inferring analysis to answer the key questions
Colloquially, we refer to this process as the “bow-tie” - which is representative of the efforts required for each stage. While a lot of the most advanced work takes place in the middle, a lot of the time and effort is spent on the edges.
The aim of any analyst as a hallmark is trying to improve the accuracy & efficiency in which they move through these three stages to deliver value to wider teams.
My advice is to do this through developing an intrinsic understanding of each of these stages, identifying the parts within each which require their skills & attention alongside those where they trust it can appropriately be handled by automated systems. This is where I have noticed AI systems become a part of their day-to-day functions.
How does AI tooling fit into this?
With this in mind, what are the tasks that can be automated fully in this process when weighing up the risk & reward to the analyst?
- Data Collection
Any good analyst should spend time understanding their data before they try to apply any models - AI has the possibility to streamline this process step to highlight where there appears to be missing context or where data inputs have changed significantly.
However, context is everything in the world of model preparation. Whilst LLM models can be used to provide an overview story behind the data being analysed in record speed, they can often miss the contextual changes that show up in data which only an analyst with ample knowledge of the world surrounding the information they have to hand would spot and adjust for.
As an example, a paid social team changes campaigns from UK targeted to EU targeted without altering campaign naming structures - context will have a fundamental impact on any model results, but won't be listed in the raw API data connections which the LLM bases its assumptions on.
- Model Specification & Modelling
Specifying and building the model for any piece of analysis is where the levels of excitement in the team increase. Despite it being a key role motivator for most in these teams (I personally love building regression models…), the current trend is for AI to be used to assist this building in real-time based on "bootlegged" parameter definitions. This has the benefits of providing teams with a read-made, high-velocity output which they can work with - and should allow them to focus on day-to-day value-creation - despite removing some of the “fun” of the job.
With a model specification being set, and suitable time allowed for training - AI modelling capabilities can indeed improve efficiency significantly. However, there is a risk that the ultimate nuance around marketing analytics - which is where a skilled analyst differentiates themselves through the decisions they make - is missed, in favour of speed, and key questions to be answered by an analysis missed.. This is a risk that needs to be assessed by teams, but will improve as AI models in this space become ever more sophisticated.
- Interpretating, Visualisation & Learning
Understanding outcomes from the technical work we do is what really allows my team to generate value internally, and externally. This usually consists of analysing & visualising modelling outputs - and marrying this up to wider data sets to create a story on the questions the project itself was based on. This is incredibly time intensive, and requires an analyst to first have a deep understanding of the outputs their work has delivered - and how the meanings to these shift as they become a “solution” or a “strategy”.
AI provides help to bridge the gap between analytics teams and the wider business by making outcomes modelling work more digestible. With a model to read from, stakeholders with less technical interests can now just ask questions of models in direct fashion, and get relevant outputs which are cut in the required manner without the need for an analyst to build relevant visualisations - all the while it can learn how particular stakeholders process information and cater its outcomes for individuals.
This is particularly useful for visualisation - with AI assistance, analytics teams are now able to deliver visualisations on outputs at speed and to very specific requirements for each request, removing the need to build a “answers-all” dashboarding solution. Yet, this again comes with risks - without a full understanding of the context in which a question was asked or the data around it we walk the tightrope of different stakeholders understanding outcomes in different ways due to the questions they have posed being contextually different. A key observation of AI models is their eagerness to please can often mean they provide a response regardless of certainty - which is a layer of clarity an Analyst would provide.
What does the future look like?
Despite its current limitations, and the risks they pose - the embedding of AI tooling in and how it sits within marketing analytics teams is becoming an increasingly relevant question, as teams look to provide maximum value in minimal time.
When adopted in the right capacity, marketing analytics teams can become more efficient “deliverers of value” - which means greater cross-team engagement, improved trust in analytics teams capabilities - and recognise the value these projects identify at scale. However, adoption of AI tech in the wrong way can severely limit effectiveness - due to both its current technical limitations highlighted above within project, and through the misunderstanding of decision makers that AI is a replacement for work where value is often underestimated.
