# Google Cloud

## Pre-requisites

This tutorial assumes that you have a Billing Account for your Google Cloud Projects in place. In the event this is not the case please follow the steps in [this tutorial](https://cloud.google.com/billing/docs/how-to/manage-billing-account) before proceeding.

## Setting up BigQuery

Exivity will leverage on the GCP BigQuery service. This will enable the export of detailed Google Cloud billing data to a BigQuery dataset, giving Exivity the capability to query the billing data in a more granular manner than with the standard file export (which is going to be [deprecated](https://cloud.google.com/billing/docs/how-to/export-data-bigquery) by Google).

{% hint style="info" %}
Before proceeding further, make sure sure your user has the **Billing Account Administrator** and the **BigQuery User** roles associated
{% endhint %}

First and foremost, Exivity will create a dataset containing the billing table which will be periodically queried. Please follow these instructions:

1. Go to the [BigQuery ](https://console.cloud.google.com/bigquery)service page.
2. Select the project that will contain your dataset in the project drop down.
3. Click **CREATE DATASET**.
   1. Enter a **Dataset ID**
   2. Select **Data location** (region)
   3. Set **Default table expiration** to *Never*
   4. Set **Encryption** to *Google-managed key*
4. Finally, click **Create dataset**
5. Note the *Project*, *BigQueryProject* and *Table*, these parameters will be used by Exivity on a later stage.

Once you have the dataset created in BigQuery you need to enable Cloud Billing export to BigQuery:

1. Sign in to the [Cloud Billing Console](https://console.cloud.google.com/billing), select *Organization* and main *Billing Account*.
2. Go to the **Billling Export** tab.
3. Click **Edit Settings** to enable the export.
   1. Select the project where you previously created the BigQuery dataset.
   2. Select the specific dataset from the **Billing export dataset** list.
   3. Click **Save**.

## Creating Exivity Service Account

Exivity requires a GCP *Service Account* with the **BigQuery User** role asscociated in order to retrieve the billing data. Please follow [this manual](https://cloud.google.com/iam/docs/creating-managing-service-accounts) to create a Service Account, [this manual](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) to associate a Private Key with the service account, and finally [this manual](https://cloud.google.com/iam/docs/granting-changing-revoking-access) to associate the role with the service account.

{% hint style="warning" %}
Make sure to write down the *Mail address / service account* and its associated *Private Key*, as these parameters will be required for the Exivity Data Extractor
{% endhint %}

## Configuring the Extractor

To create the Extractor in Exivity, browse to **Data Sources > Extractors** and click the *Create Extractor* button. This will try to connect to the Exivity Github account to obtain a list of available templates. For Google Cloud, please click *Google\_Cloud* from the list. Provide a name for the Extractor in the name field, and click the **Create** button.

Once you have created the Extractor, go to **Variables** tab and fill the parameters:

* **Hostname**:  Default endpoint for Google BigQuery with version
* **Private**: Provide the RSA private key PEM format 

{% hint style="danger" %}
**IMPORTANT** make sure to replace **\n** with **${NEWLINE}** in the **RSA Private key** using a text editor, before pasting the key in the Exivity Extractor field
{% endhint %}

* **Email**: Mail address / service account associated with the private key, obtained in the *Creating Exivity Service Account section.*
* **Project**: Main GCP Project name.
* **BigQuery project**: GCP BigQuery Billing Project, obtained in the *Setting up BigQuery section.*
* **BigQuery table**: GCP BigQuery Billing Table, obtained in the *Setting up BigQuery section.*

Finally, click on **Update**.

![Example of GCP Extractor Variables](https://3540922554-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LHEKskLK6aXinV75Knl%2Fsync%2F425eff5c97c7578bca277e33278ab3e4353f8f56.png?generation=1616416803517971\&alt=media)

Once you have filled in all details, go to the **Run** tab to execute the Extractor for a single day:

![](https://3540922554-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LHEKskLK6aXinV75Knl%2F-Lz1ckLhuu2xubIn-UpI%2F-Lz1fv-tCTdyxOhUHEcr%2Fimage.png?alt=media\&token=9a703a2f-d7d3-49f6-baed-96466c18a0c2)

The Extractor requires two parameters in *yyyMMdd* format:

* **from\_date** is the date for which you wish to start collecting consumption data.
* **to\_date** is the end date for which you wish to collect consumption data.

When you click the **Run Now** button, you should get a successful result.

## Configure a Transformer

Once you have successfully run your Google Cloud Extractor, you should be able to create a Transformer template via **Data Sources > Transformers** and click the **Create Transformer** button.

Select the Google Cloud Transformer that you just created, go to the **Run Tab** and run it for a single day as a test. Make sure that it is the same day as for which you extracted consumption data in the previous step.

![](https://3540922554-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LHEKskLK6aXinV75Knl%2F-Lz1ckLhuu2xubIn-UpI%2F-Lz1gpMQbLnBGyUGcmIv%2Fimage.png?alt=media\&token=7b0a42ec-5b21-462f-9fb7-828705190cb1)

When you click the **Run Now** button, you should get a successful result.

## Create your Report

Once you have run both your Extractor and Transformer successfully create a Report Definition via the menu option **Report Definition** via the menu **Reports > Definitions**:

![](https://3540922554-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LHEKskLK6aXinV75Knl%2F-Lz1ckLhuu2xubIn-UpI%2F-Lz1hS4CXZJAquvjKzyk%2Fimage.png?alt=media\&token=1efdf63a-48be-453d-a0a9-22b06d064cdc)

Select the column(s) by which you would like to break down the costs (you can start with only *project\_name* as a test). Once you have created the report, you should then click the **Prepare Report** button after first making sure you have selected a valid date selector shown when preparing the report.![](https://blobscdn.gitbook.com/v0/b/gitbook-28427.appspot.com/o/assets%2F-LHEKskLK6aXinV75Knl%2F-LMg6h46PYa2zRIM4aIw%2F-LMgS-sUs8uBd0jk5bFC%2FManual13.jpg?alt=media\&token=3fd1f372-60bc-48f3-a339-bbcc0f0d0086)

Once this is done you should be able to run any of **Accounts**, **Instances**, **Services** or **Invoices** report types located under the *Report* menu for the date range you prepared the report for.