Set up Google Cloud Platform

Some features of Tailer Platform rely on Google Cloud Platform's resources, or allow you to manipulate them. Some steps need to be performed as preparation.

🗄️ Create a project

Create a dedicated Google Cloud Platform project to use with Tailer Platform. Billing needs to be enabled.

To create a new project:

1. Go to the Manage resources page in the Cloud Console.

2. In the organization drop-down list at the top of the page, select the organization in which you want to create a project.

3. Click Create Project.

4. In the New Project window that appears, enter a project name and select a billing account as applicable.

5. If you want to add the project to a folder, enter the folder name in the Location box.

6. When you're finished entering new project details, click Create.

🔑 Enable APIs

The following Google Cloud APIs need to be enabled for the project you have created:

• Cloud Functions

• Identity and Access Management (IAM)

• Cloud Resource Manager

To enable an API:

1. Open the APIs & services page for your project in the Cloud Console.
2. Click the Enable APIs and Services button.

3. Click the API you want to enable. If you need help finding the API, use the search field.

4. In the page that displays information about the API, click Enable.

🔓 Grant roles to the App Engine default account

When you enable the Cloud Functions API, a service account is automatically created for your project. It should be named as follows:

[e-mail]

You need to grant roles to this service account so that it has permission to complete specific actions on some resources in your Cloud Platform project.

To grant roles to the service account:

1. Open the IAM & Admin page for your project in the Cloud Console.
2. Click the Edit button corresponding to [e-mail].
3. Add the Project > Editor and Service Account > Service Account Token Creator roles.

4. Click Save to apply the roles to the service account.

👥 Add the App Engine default account to the appropriate groups

The App Engine default account will need to access the following elements of Tailer Platform:

• Composer (Airflow): to trigger DAGs

• Firestore: to retrieve data operations

• Source Repositories: to retrieve Cloud Functions source code

The GCP project hosting the Composer and Firestore instances should already have groups with the appropriate permissions. You have to add the App Engine default account to these groups.

🆕 Create a generic service account

This generic service account will be used among other things for:

• Deploying configurations on behalf of an authorized user

• Moving files from one bucket to another

You will need a dedicated service account for each GCP project you will use Tailer with (for example if they contain a source or destination bucket used in a transfer operation). We recommend you only create one service account per project to avoid right administration becoming too complex.

To create the service account:

1. Open the Service Accounts page in the Cloud Console.
2. Click Select a project, choose your project, and click Open.

3. Click Create Service Account.

4. You can use YOUR-PROJECT-ID as a name for the service account.

5. Click Save.

You should now have a new service account named as follows:

[e-mail]

🔐 Generate JSON credentials

To generate JSON credentials:

1. In the Service Accounts page of the Cloud Console, find the row of the [e-mail] service account that you've just created.
2. In that row, click the More button, and then click Create key.
3. Select JSON as Key type and click Create.

When you create a key, your new public/private key pair is generated and downloaded to your machine. It serves as the only copy of the private key. You are responsible for storing the private key securely.

These credentials will need to be encrypted, so you can use them later in a data operation JSON configuration file.