Putting ChatGPT To The Test On GA4
Fifty-Five, a UK technology consulting, have reported that leading generative AI tool, ChatGPT had a success rate of just 29% when asked to answer questions related to Google Analytics 4. The consultancy firm took the opportunity to put ChatGPT to the test by asking it a range of 42 questions that clients commonly ask consultants, relating to the use of GA4.
ChatGPT Got Just 29% Correct on GA4
On the 42% questions, ChatGPT only provided 12 acceptable answers, which the consultancy would consider worthy of providing to a client (a success rate of just 29%).
A further 19% of responses (8 of 42 questions) were deemed partially correct, which were made up of a combination of misinterpreted questions, where the AI had provided a different answer to what was asked (although the answer provided was factually correct) and a smaller number of responses which included some misinformation in the response.
The remaining 52% of answers (22 of 42) were factually incorrect, and in some instances actively misleading.
Conclusions
The most common reason of the inaccuracies in the date was due to the fact that ChatGPT does not use training data beyond 2021 and so it was unaware of many of the recent updates.
While ChatGPT isn’t the best place to go with your Google Analytics questions right now it will only get better in time and does have some amazing potential to assist with knowledge and insights in the very near future.
It’s agreed that most jobs aren’t at imminent risk of being taken over by AI completely and their role will be more attuned to helping humans massively improve productivity and prompting human creativity. To read the full article visit the Fifty-Five website.
