March 11, 2019
One of the major shortcomings of the free version of Google Analytics is the limit on how much data to extract. Fortunately, there is a hack how to bypass the limits and extracting data for a full day at a time.
We’ve worked intensely on how to utilize web analytics data in Google Analytics for ML/AI-purposes. One of the many challenges is the limit on how much data you can extract from the free version of Google Analytics. This can off course be dealt with by upgrading to Google 360. However, this is rather costly.
Python Script Bypasses the Limits
Instead we developed a Python script that bypasses the limits and allows extraction of a preferred amount of data from Google Analytics. The script withdraws data from one day at a time, but within this timeframe there are not any restrictions.
What the script does is creating a loop and pulling data out in batches. The script adds the date, hour, and minute dimension and allows data to be split into 60 batches and neatly join the associated data frames together at the end.
The reason for using the minute dimension is because of how convenient it is to loop through for two reasons. First, it is a dimension were values are stored as numeric. Second, splitting data out by each minute during a day equally distributes the data out in 60 batches.
Conversely, an hour of the day would not equally distribute data out in 24 batches considering that website traffic often peaks around 4 pm which may cause a violation of the API limit.
Here's the Script:
The scripts shown above only show some of the features available with the Python API. Up to 9 dimensions can be pulled out in one request by adjusting the above scripts further. Furthermore, other filters can also be implemented and predefined by adjusting the script ever so slightly. Good luck extracting data from Google Analytics for your preferred purpose.
