Loading data from SAP using SAP Datasphere with S3 Bucket

Preface

This scenario covers the loading data from SAP ERP - be it SAP ECC or SAP S/4.

It assumes you are using SAP Datasphere for replicating the needed tables from SAP to an S3 bucket.

Overview

You will use SAP Datasphere for replicating all the needed tables from SAP into an S3 bucket, which will in-turn be read and ingested by Datricks.

Here is a diagram showing a possible setup:

For more information about setting up a replication flow from SAP to S3 please review this SAP blog:

https://community.sap.com/t5/technology-blogs-by-sap/sap-datasphere-replication-flow-from-s-4hana-to-amazon-s3/ba-p/13616320

Prerequisites

Depending on the Processes you would like Datricks to analyze, you'll need to replicate a different set of tables.

See the attached spreadsheet with an initial table list:

Table list (per Process)

Make sure you have an S3 bucket available, and you know the following details:

Where the bucket is hosted (AWS or Minio)

The S3 bucket path, Access key ID, Secret Access Key

For more information about S3, please refer to the following amazon S3 user guide.

After you've have successfully set up replication flow from SAP to the S3 bucket for all the required tables, you're ready to start loading the data into Datricks.

Loading the data - Set up a new loading area (source system) in Datricks

Step 1:

Go to your tenant's URL /welcome 

i.e. :   <mytenant>.datricks.ai/welcome

Select "Upload files from SAP"

Step 2 - Provide S3 Bucket details

Give a name for the new data source (i.e SAP-ID8)

In addition, fill in the connection details to the S3 bucket, and test the connection

Step 3 - Select business processes

We recommended you don't disable the default Scenarios (enabled by default) - they ingest data that is useful for all other scenarios.

Add the required scenario(s) by enabling them:  Procure-to-Pay, Order-to-Cash, etc

On the bottom of the screen, give a name to this loading area (i.e. Audit of SAP-ID8)

Step 4 - Confirm and start the load

Review the summary and if you are ready to go, hit "Run data load"

Step 5 - Load is starting

The data load has started.

Once the data is loaded successfully completed, the insights and processes will become visible in the system, and you'll be able to start reviewing them.