Automating processes and saving 10+ hours per week with Keboola

Keboola is the core foundational piece of the data stack that we prescribe to our clients.

Key takeaways:

• We are an end-to-end data consultancy in Minneapolis and Phoenix that streamlines data operations as a service using Snowflake and Keboola.
• We saved one of our manufacturing client 10+ hours a week by automating their end-to-end reporting with Keboola.
• Instead of spending weeks re-architecting the infrastructure of their online education client, they achieved the same results in hours by using Keboola.
• Inspired by the countless success stories, we have launched a Data Concierge service. This removes the barriers of entry into the world of data for SMBs and mid-sized businesses, empowering them to become truly data-driven companies - all thanks to Keboola.

Our partner, Keboola talked to our very own, Daniel Rothamel, Cloud Data Delivery Engineer at North Labs, to better understand how the joint solution of Snowflake, Keboola, and North Labs helps their customers to get more value from their data.

First, let’s look at how our team uses Keboola to empower their clients.

Use cases from the industry frontline

Use case #1: Saving valuable time in manufacturing

Our team was asked to align data sources into the Snowflake data warehouse for a manufacturing company in North America. The company needed a single source of truth for all of their reporting.

The main challenge was that the company used ERP systems with non-standard data organization and non-trivial data extraction specifications.

One of their plant managers was spending more than 10 hours a week manually creating Excel spreadsheets which showed their customers expected production versus actual production.

Every week, the plant manager had to complete the following steps:

• Log into the ERP system
• Pull the week's production numbers
• Compile information in a spreadsheet
• Analyze the data in the spreadsheet
• Distribute to customers to validate it

He was doing all of this manually.

To help him out, North Labs used Keboola to automate the process end to end.

Keboola was used to ingest data from the ERP systems into Snowflake, build transformations, and feed the cleaned data into their data visualization software. This automated the entire end-to-end pipeline to produce reports without any additional work.

Now, all the plant manager needs to do is go to a dashboard and pull those numbers automatically.

Use case #2: Sales and marketing automation for online education

We helped an online education company build a data foundation to understand how marketing and sales are driving growth.

The challenge? Before we were called in to help, the online education company built its own data ecosystem. It was a Zapier spaghetti mess.

With more than 200 zaps integrating data across dozens of sources, it was hard to understand data provenance and semantics. The client’s data team was “handcuffed to the source systems” and had to follow the structure of the data sources before every analysis.

We started by untangling the existing DataOps into a more observable, scalable, and maintainable set of ETL pipelines in Keboola.

By piping the clean data into Snowflake - a central source of truth for marketing and sales information - We validated the data before exposing it to analysts and made data consumption more accessible and self-serviceable.

The impact?

“Without Keboola, it would have taken weeks and likely months to get their data ingested because of their source systems. With Keboola, our engineers were ingesting data within hours.”

Use case #3: Using telemetry data to optimize DataOps

For every data pipeline you build and every data process you engineer, you need to be monitoring and optimizing.

And when you’re North Labs, those DataOps processes accumulate with every client.

We had two goals they wanted to chase:

1. Deliver health-of-pipeline and consumption reports to help customers understand how their DataOps is performing.
2. Analyze and discover their own efficiencies, so they can scale their services from existing clients to new clients.

To achieve these, they needed telemetry data to help them build those all-important health, consumption, and efficiency reports.

Luckily, Keboola exposes telemetry data out of the box, enabling our team to build the telemetry infrastructure.

“Other tools have that data, but you can't really access it. You can see it, but you can't play with it. With Keboola we get to play with it, and we get to tweak it.”

So, how do these use cases translate to our services?

The Data Concierge offering

We noticed that many organizations on the market have data, know that it’s valuable, and recognize that they should be getting better insights. The thing that they lack is the internal resources needed to set up the DataOps processes and turn data into insights.

This is where our Data Concierge offering comes in.

Data Concierge offers North Labs as an extension of the client’s in-house team. We take care of the data ingestion, cleaning, modeling, and other DataOps engineering, so that “the company's team can focus on driving value for the business because that's really where their time and effort is best spent”.

Central to this offering is Keboola:

“Keboola is central to our Data Concierge offering. It has so many features that help you take advantage of your data. Everything is very customer-centric and easy to use. Keboola is a really incredible tool for centralizing and unifying your DataOps.”

Now, let’s go back to the beginning and learn how North Labs discovered Keboola and why they were selected over other alternatives on the market.

4 reasons why we chose Keboola

The story began with solving a problem.

“We had a customer that had a specific source system that we needed to connect to for the ETL pipeline. And we couldn't find a connector anywhere. We were trying to figure out how we were going to build a connector, and avoid additional engineering costs as well as awkward conversations with the client. In looking through the Snowflake partners, we found Keboola.”

Keboola entered our radar by offering an out-of-the-box Generic Extractor, which helps to speed up data extraction from any API-like source.

As North Labs explored Keboola, they realized the platform fully aligns with their organizational needs:

• Centered around Snowflake. Both Keboola and North Labs center their services and platform offerings around Snowflake to empower users with cloud-native features, such as compute-on-demand and practically infinite scalability and elasticity.
• Value alignment. Both companies see the data landscape as being full of potential. Their roles involve delivering the best possible experience for clients and unlocking the full power of their data.
• Extensive support. North Labs uses the self-service Keboola Data Academy to get its engineers up to speed on Keboola’s platforms and features. When things get complex, Keboola’s team jumps on a call with Daniel to smooth out the sharp data edges. As Daniel said, “the support from you guys has been nothing short of amazing”.
• Feature-rich offering. From CLI that streamlines engineering work to the automated no-code platform, Keboola offers features that help North Labs develop solutions that their clients can consume.

Schedule a call with our sales team for more information about Keboola and our Data Concierge service!