API To Storage usage examples

Pub/Sub to GCS examples

Pub/Sub endpoint

Once the ATS configuration file has been deployed, a Pub/Sub endpoint will be available and ready to receive data.
The Pub/Sub URL will be constructed as follow:
https://pubsub.googleapis.com/v1/projects/[GCP_PROJECT_ID]/topics/ats-topic-[CONFIGURATION_ID]:publish
Considering the following value coming from your ATS configuration :
  • gcp_project_id (from "source" attribute: fd-io-jarvis-demo-dlk
  • configuration_id : 000099-ats-example-products
The resulting Pub/Sub endpoint will be:
https://pubsub.googleapis.com/v1/projectsfd-io-jarvis-demo-dlk/topics/ats-topic-000099-ats-example-products:publish

How to build a data payload

Pub/Sub expects a Base64 encoded JSON string.
API To Storage processor expects an array of objects build with the following schema:
{
"input_data": [
{ JSON Object 1 },
{ JSON Object 2 },
{ JSON Object 3 },
{ JSON Object 4 },
...
]
}
Example:
{
"input_data": [
{
"product_id": "123456789",
"label": "Some label ABC",
"description": "A specific description for product 123456789",
"new_attribute": "test"
},
{
"product_id": "987654321",
"label": "Some label YUI",
"description": "A specific description for product 987654321"
},
{
"product_id": "66668888",
"label": "Some label XYZ",
"description": "A specific description for product 66668888"
}
]
}
Then, the JSON string must be Base64 encoded. Using the previous example, the result will be:
eyAgICAiaW5wdXQtZGF0YSI6IFsgICAgICAgIHsgICAgICAgICAgICAicHJvZHVjdF9pZCI6ICIxMjM0NTY3ODkiLCAgICAgICAgICAgICJsYWJlbCI6ICJTb21lIGxhYmVsIEFCQyIsICAgICAgICAgICAgImRlc2NyaXB0aW9uIjogIkEgc3BlY2lmaWMgZGVzY3JpcHRpb24gZm9yIHByb2R1Y3QgMTIzNDU2Nzg5IiwgICAgICAgICAgICAibmV3X2F0dHJpYnV0ZSI6ICJ0ZXN0IiAgICAgICAgfSwgICAgICAgIHsgICAgICAgICAgICAicHJvZHVjdF9pZCI6ICI5ODc2NTQzMjEiLCAgICAgICAgICAgICJsYWJlbCI6ICJTb21lIGxhYmVsIFlVSSIsICAgICAgICAgICAgImRlc2NyaXB0aW9uIjogIkEgc3BlY2lmaWMgZGVzY3JpcHRpb24gZm9yIHByb2R1Y3QgOTg3NjU0MzIxIiAgICAgICAgfSwgICAgICAgIHsgICAgICAgICAgICAicHJvZHVjdF9pZCI6ICI2NjY2ODg4OCIsICAgICAgICAgICAgImxhYmVsIjogIlNvbWUgbGFiZWwgWFlaIiwgICAgICAgICAgICAiZGVzY3JpcHRpb24iOiAiQSBzcGVjaWZpYyBkZXNjcmlwdGlvbiBmb3IgcHJvZHVjdCA2NjY2ODg4OCIgICAgICAgIH0gICAgXX0=

Send data payload to Pub/Sub

Using CURL, we can now simply send the data payload to Pub/Sub:
curl --request POST \
--http2 \
--header "content-type:application/json" \
--header "Authorization: Bearer ${TOKEN}" \
--data '{"messages": [
{"attributes": {}, "data": "eyAgICAiaW5wdXQtZGF0YSI6IFsgICAgICAgIHsgICAgICAgICAgICAicHJvZHVjdF9pZCI6ICIxMjM0NTY3ODkiLCAgICAgICAgICAgICJsYWJlbCI6ICJTb21lIGxhYmVsIEFCQyIsICAgICAgICAgICAgImRlc2NyaXB0aW9uIjogIkEgc3BlY2lmaWMgZGVzY3JpcHRpb24gZm9yIHByb2R1Y3QgMTIzNDU2Nzg5IiwgICAgICAgICAgICAibmV3X2F0dHJpYnV0ZSI6ICJ0ZXN0IiAgICAgICAgfSwgICAgICAgIHsgICAgICAgICAgICAicHJvZHVjdF9pZCI6ICI5ODc2NTQzMjEiLCAgICAgICAgICAgICJsYWJlbCI6ICJTb21lIGxhYmVsIFlVSSIsICAgICAgICAgICAgImRlc2NyaXB0aW9uIjogIkEgc3BlY2lmaWMgZGVzY3JpcHRpb24gZm9yIHByb2R1Y3QgOTg3NjU0MzIxIiAgICAgICAgfSwgICAgICAgIHsgICAgICAgICAgICAicHJvZHVjdF9pZCI6ICI2NjY2ODg4OCIsICAgICAgICAgICAgImxhYmVsIjogIlNvbWUgbGFiZWwgWFlaIiwgICAgICAgICAgICAiZGVzY3JpcHRpb24iOiAiQSBzcGVjaWZpYyBkZXNjcmlwdGlvbiBmb3IgcHJvZHVjdCA2NjY2ODg4OCIgICAgICAgIH0gICAgXX0="}]}' \
"https://pubsub.googleapis.com/v1/projectsfd-io-jarvis-demo-dlk/topics/ats-topic-000099-ats-example-products:publish"
Note that the Pub/Sub endpoint is not publicly callable. You need to specify a JWT in the HTTP Authorization header to go though.
The user MUST have the Pub/Sub Publisher role to the according GCP Project to have access.
This is a link to GCP documentation on how to generate a JWT from a service account credential file : https://cloud.google.com/endpoints/docs/openapi/service-account-authentication