Notebooks & Programmatic Access¶

Notebooks¶

Quick Start¶

In order to use the BigQuery client libraries provided by Google, you first need to go through the following steps:

Select or create a Cloud Platform project.
Enable billing for your project.
Enable the Google Cloud BigQuery API (enabled by default for newly created projects).
Setup Authentication.

Authentication¶

Access to the Dimensions on BigQuery data sets is provided to the Google Account that was nominated during the signup process. In order to configure programmatic access for local development, the easiest way is to authenticate using the Google Cloud SDK.

$ gcloud auth application-default login

This will generate a JSON file that is used as the default application credentials for the account that was selected in the above login process. When using the default Client for each Google provided package (such as BigQuery) they should automatically authenticate using these default credentials.

$ gcloud auth application-default set-quota-project my-quota-project

The command above is required to set the default quota project that Google Cloud SDK and associated libraries will utilise when performing API requests. You must replace my-quota-project in the above example with your own project. Your credentials must have access to use the quota project (they need “serviceusage.services.use” at minimum). The project you specify will be used for managing the API access quota limits against the resources you utilise using these default credentials. Details of quota limits for BigQuery are available from here.

Install the BigQuery client¶

To install the Python variant of the BigQuery client, this can be accomplished using the package manager pip.

$ pip install --upgrade google-cloud-bigquery

Running Queries¶

from google.cloud import bigquery

BQ_PROJECT_ID = "<GCP project id from quick-start #1>"
client = bigquery.Client(project=BQ_PROJECT_ID)

query_job = client.query("""
  SELECT
    id,
    title.preferred as title,
    ARRAY_LENGTH(authors) as authors,
    CAST(altmetrics.score as INT64) as altmetric_score
  FROM
    `dimensions-ai.data_analytics.publications`
  WHERE
    year = 2020 AND 'grid.4991.5' in UNNEST(research_orgs)
  ORDER BY
    altmetrics.score desc
  LIMIT 5""")

results = query_job.result()  # Waits for job to complete.

for row in results:
  print("{} : {} : {}".format(row.id, row.authors, row.altmetric_score))

Google Colaboratory¶

Google Colaboratory provides an easy and free service allowing you to get started with notebooks. Using your Google Account you can create notebooks, execute BigQuery queries and share these with other Google Accounts quickly and easily.

The key features of Google Colaboratory are:

Zero configuration required.
Easy sharing
Supports easier Google Account authentication

Unattended Programmatic Access¶

Service Accounts¶

Google provides service accounts for unattended programmatic access. These are created on a per project basis and have an email address allocated to them just like normal user accounts. These accounts do not have a password associated with them, instead public/private keys in a JSON file are used to provide scripts with the ability to authenticate with GCP servers and utilise resources. These accounts are tied to your own GCP projects and are not something we can create on your behalf. Newly created service accounts do not have access to the BigQuery Dimensions datasets (they do not mirror or inherit the access rights from user accounts).

If you have a specific need for unattended programmatic access via a service account, you will need to contact us in order to organise BigQuery read access onto the Dimensions BigQuery tables. Please note however, unattended programmatic access via GCP service accounts is only available to specific customers, dependent on the type of subscription purchased.

Further details from Google regarding service accounts are available from: Cloud IAM: Service accounts.

We also provide documentation regarding the creation and recommended configuration of a service account, including tips and pointers regarding the setup of ODBC/JDBC connections using service accounts here: Creating & Configuring Service Accounts.