Unpivot, Split Column, Pivot Bigquery/Postgresql
I have table like this: Then I want to change the table to this: I usually doing this in Power BI Query Editor but the file is too big so I try in another platform like Bigquery/Postgresql Thank you
how to get raw events from Firebase analytics using api without BigQuery?
I need to extract raw events from Firebase Analytics using python SDK. Actually, we can link a BigQuery to a firebase and access raw events through BigQuery. But it is not clear from the documentation is there any other ways…
Postgresql – Get the next (or previous) non-null value in multiple partitioned
Sample data below. I want to clean up data based on the next non-null value of the same id, based on row (actually a timestamp). I can't do lag, because in some cases there are consecutive nulls. I can't do…
Solution to Mongodb compass interior hyphen error?
I'm attempting to import a large CSV file into mongodb using mongodb compass. The data originally came from BigQuery via GDELT, then was dumped into 40+csv files. Over half of the files are not able to be imported as they…
How to install google-cloud-bigquery on python-alpine based docker?
I'm trying to build a docker with python 3 and google-cloud-bigquery with the following docker file: FROM python:3.10-alpine RUN pip3 install google-cloud-bigquery WORKDIR /home COPY *.py /home/ ENTRYPOINT ["python3", "-u", "myscript.py"] But getting errors on the pip3 install google-cloud-bigquery (too…
Redis – Google Cloud: How to deduplicate streaming data within a very long time frame?
I have a VM (or auto-scaling MIG, if it makes more sense), a pub-sub pull subscription as the input, and a pub-sub topic as the output. I want this VM to dedup messages. The input items are often repeated within…
SQL/Bigquery text classification – Artificial Intelligence
I need to implement a simple text classification using regex, and for this I though to apply a simple CASE WHEN statement, but rather than in case 1 condition is met, I want to iterate over all the CASEs for…
How to find combination of intersection from many tables? – SEO
I have a list of different channels that could potentially bring users to a website (organic, SEO, online marketing, etc.). I would like to find an efficient way to count daily active user that comes from the combination of these…
Bq command line error in compute engine VM instance CENTOS7
I'm running a VM instance (google compute engine) with CENTOS 7, everytime i run the bq command, i keep getting error. I supposed bq is by default in compute engine. [username@instance-1 ~]$ bq Error initializing bq client: service_account Traceback (most…
Create a Dataset in BigQuery using API
So forgive my ignorance, but I can't seem to work this out. I want to create a "table" in BigQuery, from an API call. I am thinking https://developer.companieshouse.gov.uk/api/docs/search/companies/companysearch.html#here I want to easily query the Companies House API, without writing oodles…