Modern Data Stack requires setup and maintenance and Snowflake goes a long way to lowering the bar.

Our Powered By Snowflake partner Keboola offers Data Platform as a Service, helping users further accelerate data projects by delivering a ready-to-use managed platform. It includes all the technologies a data team needs. Therefore enabling everyone to capitalize on Snowflake's capabilities and acting as an avenue to the incredible modern no-maintenance and no-code solution.

This Quickstart will guide you to build a complete data pipeline from ingestion, transformation, orchestration to presentation in a Streamlit application.

Prerequisites

What You'll discover

Template What You'll Need

What You'll Build

Template

Step 1

  1. Login to Snowflake Snowsight.
  2. Switch role to ACCOUNTADMIN.
  3. Click Admin > Partner Connect.
  4. Search for Keboola. kebPC
  5. Click Keboola tile.

Step 2

  1. See objects that will be created in Snowflake.
  2. Click Connect.
    PartnerConnect

Step 3

  1. Click Activate.
    PartnerConnect

Troubleshooting

You must have a verified email in your profile. Otherwise you'll get the following message:

PartnerConnect

You also MUST have a first name and last name in your profile:

Open your profile:

PartnerConnect

Add email, first and last name:

PartnerConnect

Define your Keboola password

After you have clicked "Activate" from the previous section, you'll land on Keboola's Welcome page to define your password:

PartnerConnect

Enable full features

Congratulation, you have now a Keboola account!

We need to enable all features for this lab. It's a simple process.

Click on "Enable All Features" link:

PartnerConnect

Copy the SQL code from the Keboola UI:

PartnerConnect

In Snowflake Snowsight, open a new worksheet, paste the code and select "Run All", be sure all rows are executed!

PartnerConnect

Go back to Keboola, you should see the following:

PartnerConnect

Here we'll prepare everything for Keboola to write into Snowflake account, remember Partner Connect flow already created a DB (PC_KEBOOLA_DB) and a warehouse (PC_KEBOOLA_WH):

Login to Snowflake trial account

PartnerConnect

At the left bottom of the screen, hover the account and click on "Copy account URL".

Paste the content in safe place to reuse later.

We need first to activate the Data Apps feature in Keboola:

In the top right corner, select your user and navigate to "Settings"

Template Creation

Activate the Data Apps Feature:

Template Creation

Let's move to Keboola platform, after you have created your Keboola trial account, login to the platform and navigate to the "Templates" tabs.

In the searchbar, type "RFM"

Template Be sure you select the correct template "Shopify RFM Analysis with Streamlit".

Template Creation

Click the green button "+ USE TEMPLATE"

Template Creation

You'll see more details on the template, click the top right green button "+ USE TEMPLATE"

Template Creation

Enter a name and click " -> NEXT STEP"

For this lab, we'll use dummy Shopify data so we don't need to configure the Shopify Component.

The template configuration constists in setting-up connection information to each component needed to run the flow.

For our lab, we'll need to setup only Snowflake.

Template Creation

We need to enter Snowflake information in the "Snowflake Destination" step of the flow:

Click on "Edit Configuration"

Template Creation

Add the information we created during step 4 (Get Snowflake Account URL)

Template Creation

Template Hostname is the URL you copied in Snowflake WITHOUT "https://"

Template Port is "443"

Template Username is your Snowflake user name

Template Password is your Snowflake password

Template Database Name is "PC_KEBOOLA_DB"

Template Schema Name is "PUBLIC"

Template Warehouse is "PC_KEBOOLA_WH"

Save the template:

Template

Get Keboola token:

From the Streamlit application, we can write back data thanks to Keboola API.

We need first to get an API token. Follow the steps to generate this token.

Navigate to token page in Keboola:

Template Creation

Template Generate the token, do not forget to set "Full Access" and to copy the value, paste into a text file!

Template Creation

Get Snowflake account name:

Template At the left bottom of the screen, hover the account and click on "Copy account URL" paste into a text file!

Template Creation

Navigate to the Flow page: Template

Select the Flow: Template

Scroll down in the Flow and select the "RFM Analysis Data App", click on "Edit Configuration":

Template

Change the values according to your Snowflake account and Keboola API token:

Template user and password are your SNOWFLAKE user and password

Template account is the snowflake URL you have pasted in your text file

Template keboola_key is the keboola API token you have pasted in your text file

Template Creation

The template generated a Keboola Flow. A flow is a sequence of actions.

Navigate to the Flow page: Template

Select the Flow: Template

Run the Flow:

Template

Monitor the run:

Template

The inital run can take about 10mn, subsequent will take only few mn.

Success!

Template

Open your Snowflake web tab and check the PC_KEBOOLA_DB content.

You should see the tables created:

Template

We have seen Keboola Shopify template in action. We have now a complete set of tables in Snowflake with Shopify sales data including an RFM segmentation, ready to use!

RFM stands for Recency, Frequency, and Monetary value, each corresponding to some key customer charasteritics. These RFM metrics are important indicators of a customer's behavior because frequency and monetary value affects a customer's lifetime value, and recency affects retention, a measure of engagement

In this section, we'll leverage this segmentation in the Streamlit application, automatically created from the template.

This application will :

Navigate to the Data Apps section

Template Creation

Open the Data App

Template Creation

If your Snowflake information are correct, after clicking "Connect" you should see:

Template Creation

RFM Segmentation Overview

You see here the generated segmentation and the number of customers assigned to.

Template Creation

Simulate discount

You can select the segment(s) you want to assign discount:

Template Creation

You can then adjust discount level and expected revenue increase.

Template Creation

This will calculate the impact on the total revenue

Template Creation

Generated list of targeted customers

Scrolling down the page, you'll find an always adjusted list of customers (based on selected segments) and the level of discount.

This list can be used to trigger a marketing campaign:

Template Creation

Write the table back into Keboola

Finally press the "UPLOAD" button:

Template Creation

Wait until the upload is finished:

Template Creation

Check the table in Keboola:

Template Creation

YOU'RE DONE WITH THE LAB !! 🍾

You choose the wrong template

Delete the associated Flow

Template Creation

Delete the Components

Template Creation

Delete the Storage

Template Creation

Same player, start again :-)

Your Snowflake credentials are wrong

Navigate to the Components tab, click on the Snowflake Data Destination:

Template Creation

Select "Database Credentials" on the right:

Template Creation

Udpate and test your credentials:

Template Creation

Your DB name, schema, warehouse name are wrong

Navigate to the Components tab, click on the Snowflake Data Destination:

Template Creation

Select "Database Credentials" on the right:

Template Creation

Check the DB, Schema, Warehouse:

Template Creation

Your Data Application secrets are wrong

Navigate to the Data Apps section:

Template

Open the configuration:

Template

Correct the secrets:

Template Creation

Congratulations! You've successfully built a complete data pipeline from ingestion, transformation, orchestration, to presentation in a Streamlit application!

What You Learned

Related Resources