Solutions For Snowflake

Snowflake, fed compliant SAP data at terabyte scale.

Lima Backbone lands SAP and Oracle history in your Snowflake account as native tables — raw schemas as loaded, a conformed schema across ERPs — with every table reconciled, then keeps it current with change-data capture minutes behind the source. Extraction runs on Lima's capacity through interfaces SAP keeps open, so your Snowflake credits stay for analytics and there is no additional SAP licence line.

Two-week Terabyte Proof-of-Scale on your largest tables. Partner Connect listing planned for 2027.
SNOWFLAKE · YOUR ACCOUNT · dark border = what Lima creates and fillsYour Snowflake accounta role for Lima's loads · a warehousesized for loadsyour keys, your network policyDatabase LIMA_SAPone database per source estateschemas by layerHorizon catalogtags · masking · access policiesLima's lineage recorded beside itSchema RAW_ECC · RAW_ORACLEtables as loaded: MARA, VBAP, BSEG …productive client only · raw values keptNUMBER(p,s) · DATE · TIMESTAMP_NTZSchema CONFORMEDITEM · SITE · SALES_ORDER … across ERPsdecimals shifted · keys trimmed · textsjoinedclustered by period and company codeSchema HISTORYclosed periods · retired systemsretention tiers · legal holdLima writesbulk: internal stage + COPY INTO,parallelchange data: MERGE by full key, deletesappliedYour warehouses for analyticsyour credits, your queriesLima's compute did the extractionThe Lima fabric reads livequestions · agents · planning loadsjoined with SAP in place · one accesspolicyBulk loads use an internal stage and COPY INTO in parallel; change data is merged by full key; every table is reconciled before the next layer reads it
Snowflake landing model: raw schemas per ERP, a conformed schema across them, history tiers; Lima writes through stages and merges, your warehouses query, the fabric reads live.
~30 min
220M rows · 615 tables · ~30 GB on 64 vCPU
80 GB/day
moved in production on the engine
1m 48s
change-data lag behind SAP after the initial load
Raw + converted
every shifted amount and trimmed key keeps its raw value beside it
product
COPY + MERGE
bulk through internal stages, change data merged by full key
product
0
extra SAP licence lines for getting the data out
product
Step 1 · What lands where in Snowflake

Raw as loaded, conformed across ERPs, history in tiers

Everything lands in your account, in native Snowflake tables you own. The raw layer keeps SAP and Oracle as they are; the conformed layer is where the decimals are shifted, the keys trimmed and the ERPs joined into one item, site and order.

LayerWhat lands thereHow Lima writes itStatus
RAW_ECC · RAW_S4 · RAW_ORACLEEvery extracted table as loaded, productive client only, raw values kept, typed correctly (NUMBER with scale, DATE, TIMESTAMP_NTZ)Internal stage + COPY INTO in parallel by partition; MERGE by full key for change dataShipped
CONFORMEDItem, site, customer, order and accounting tables across ERPs with decimals shifted, keys trimmed, texts and code descriptions joined, clustered by period and company codeBuilt from the raw layer by Lima operators; reconciled against raw totalsShipped
HISTORYClosed periods and retired systems under retention rules and legal holdOffload and decommissioning runsShipped
Horizon catalog and policiesTags, masking and access policies on the landed tables; Lima's lineage recorded beside themApplied on write; Lima's catalog holds the SAP-to-table traceShipped
Iceberg tables on your storageOpen table format written to your object storage for engines beside SnowflakeOn the published standards scheduleScheduled · standards plan
Statuses match the published standards conformance page; the Iceberg line is dated there.
Step 2 · What makes ERP data awkward to land

Six things every SAP and Oracle table brings with it, and what they do to a lakehouse

These are the reasons a straight copy of ERP tables produces wrong totals. Each has a cause in how the ERP stores data and an effect an analyst sees.

The client column and multi-org tables counts wrong or a unit missingEvery SAP table carries a client column (MANDT) and most hold rows for several clients — test, quality, productive. Oracle E-Business Suite keeps several operating units in the same tables (the _ALL tables, keyed by ORG_ID). Land them unfiltered and every count is wrong; filter them wrongly and a whole business unit is missing.
Keys with leading zeros, and keys made of several columns joins fail silentlyAn SAP material number is stored as an 18-character field padded with zeros, so 4711 is '000000000000004711'; a document line is identified by company code, document number, fiscal year and line number together. Trim the zeros in one table and keep them in another, and joins fail silently; miss one key column and rows overwrite each other.
Dates and times stored as text the wrong day, or no dateSAP dates are eight-character strings ('20260912'), with '00000000' meaning no date, and times are six-character strings in the system's time zone. Loaded as text they cannot be compared or bucketed; loaded naively as dates they fail on the zero date or land in the wrong day after time-zone conversion.
Amounts with hidden decimals, quantities with a unit next door totals confidently wrongSAP stores every amount with two decimals regardless of currency; for currencies with none, such as JPY, the stored value 12.34 means 1,234 yen, and the correction lives in a separate table (TCURX). Quantities carry their unit in a neighbouring column. Sum a column without the currency shift or the unit and the total is confidently wrong.
Wide tables, and cluster tables in ECC the history left behindAccounting line items (BSEG) have more than 300 columns and, in ECC, sit in a cluster table that cannot be read like an ordinary table; pricing conditions and change-document items are stored the same way. Tools that read only transparent tables leave the most valuable history behind.
Few change timestamps, and rows that disappear a copy that driftsMany ECC tables record only a change date, or nothing; rows in schedule lines, conditions and some master tables are deleted outright. Keeping a lakehouse current needs change documents, change pointers or triggers to find what changed, and a way to apply deletes. Oracle E-Business Suite keeps a last-update timestamp on most tables, and still deletes rows.
Snowflake-specific: types, clustering and the two write paths

Three things are specific to landing in Snowflake. Types: SAP amounts need NUMBER with the right scale and dates need DATE and TIMESTAMP_NTZ, or comparisons and bucketing fail. Clustering: a billion-row accounting table needs a clustering key on period and company code, or every month-end query scans the whole table. Two write paths: bulk history is fastest through an internal stage and COPY INTO in parallel; change data needs MERGE by the full key with deletes applied — using one path for both makes the migration slow or the daily changes wrong.

Step 3 · How Lima Backbone handles each one

One row, transformed on the way, with the raw value kept beside it

The figure follows one accounting line from SAP into Snowflake. The table maps all six problems and the Snowflake-specific ones.

One accounting line, as SAP stores itMANDT 100 · BUKRS 1000 · BELNR 4900012345GJAHR 2024 · BUZEI 001MATNR 000000000000004711WRBTR 12.34 · WAERS JPYBUDAT 20240315 · AUGDT 00000000(ECC: read from the BSEG cluster)What Lima does to itproductive client onlykeys kept and a trimmed key addeddates → real dates, zero date → nullamount × 100 for JPY (TCURX)unit kept beside every quantitytime zone → UTCThe same line in Snowflakebukrs 1000 · belnr 4900012345 · gjahr 2024 ·buzei 1matnr 4711 (matnr_raw kept)amount 1234.00 · currency JPYposting_date 2024-03-15 · clearing_date nullloaded 2026-09-12T02:14:08Z · source client100Checks on the waytotals per company code and periodkey unique · decimals entry presentThe raw value is kept beside the converted one, so an auditor can trace any number back to what SAP stored
ProblemWhat Lima doesThe capability behind itWhat you see
1 · Client and orgFilters to the productive client (and the operating units you name) on every table, and records the client in the landed rowConnector settings per source; lineage records the filterRow counts per client before and after the filter
2 · KeysKeeps every key column, adds a trimmed business key beside the raw one, and enforces uniqueness on the full key before the writeKey-aware transformation operators; uniqueness rulesKey coverage per table; duplicates listed with owners
3 · Dates and timesConverts eight-character dates and six-character times to real date and timestamp columns, turns the zero date into null, and converts system time to UTC with the source's time zoneType-aware SAP operatorsConversion counts per column; anomalies listed
4 · Amounts and unitsApplies the currency decimals table to every amount, keeps the unit beside every quantity, and checks totals per company code and period against the ERPCurrency and unit rules; control totalsTotals per company code and period, raw and shifted, against the ERP
5 · Wide and cluster tablesReads ECC cluster tables through a named function module that unpacks them, and lands wide tables with their full column set, partitioned by periodNamed SAP read paths; partitioned parallel loadsColumn coverage per table; partitions with counts
6 · Change capture and deletesFinds changes through change documents, change pointers, timestamps or triggers depending on the table, applies deletes, and replays late rows from the watermarkChange-data capture with per-table methods; ordered replayMethod per table; lag and deletes applied per run
Snowflake · types and clusteringCreates each table with NUMBER(p,s), DATE and TIMESTAMP_NTZ from the SAP dictionary types, and sets clustering keys on period and company code for the large fact tablesTarget-model catalogue for SnowflakeColumn types per table; clustering keys stated
Snowflake · write pathsBulk history through internal stages and COPY INTO in parallel by partition; change data through MERGE by full key with deletes appliedTwo write operators, chosen per runRows per minute per path; MERGE counts (inserted, updated, deleted)
Step 4 · The checks that run on every table

Nine checks a migration and every change-data run pass through

Every check is a rule with an owner. A failed partition reruns from its checkpoint; a failed row is held with the rule and the row; the report stays open until every table reads Reconciled.

CheckKindExampleWhen it fails
Row count per table and per partition equals the source, for the productive client onlyCompletenessBSEG fiscal 2024: 132 monthly partitions, each countedPartition reruns from its checkpoint; report stays open until they match
Amount control totals per company code and period match the ERP's own totals after the decimal shiftConsistencySum of BSEG amounts for company code 1000, period 2024-03, in JPY and EURException to finance with both figures
Quantity control totals per unit match the sourceConsistencyMSEG quantities per unit of measure, plant 1000Exception to the data owner
Every key is unique in the target and every key column is presentQualityCompany code, document, year and line together on accounting linesLoad refused for the partition; owner notified
Zero dates become null; no date lies outside the valid rangeQuality'00000000' in 1.4 million rows of one tableConverted and counted; anomalies listed
Every currency used has a decimals entry; every unit pair used has a conversionCompleteness9 rows in a currency with no entry in TCURXRows held; exception to finance
Rows deleted in the source are removed in the targetConsistencyWeekly full-key compare on schedule lines and conditionsDeletes applied; count reported
In the conformed layer, every child has its parentConsistencyEvery sales-order item has its header; every line has its materialException to the data owner
Change events applied in order; lag below the agreed thresholdConsistencyWatermark per table; lag shown on the dashboardRun pauses and resumes from the checkpoint
Column types and scale preserved: no truncation of amounts or keysQualityNUMBER(15,2) for amounts; VARCHAR(18) for material numbersLoad refused for the table; owner notified
Kinds: completeness (is everything there), consistency (do related numbers agree), quality (is each value valid).
Step 5 · Watching the data flow, correcting it on the way

One screen from the ERP table to the lakehouse table

Migration · BSEG · fiscal 2024 · 132 monthly partitions · 128 vCPULINEAGE CAPTURED FROM THIS RUN · EVERY STEP COUNTEDRead BSEG (cluster)SAP ECC · named functionmodule1,204,118,930 rowsClient · keys · datesclient 100 · full key ·zero dates1,204,118,930Decimal shift · TCURXamount × 100 for JPY,KRW, HUF9 rows heldStage + COPY INTORAW_ECC.BSEG · 132partitions131 done · 1 runningReconcile per periodcounts · totals percompany code131 of 132 reconciledEXCEPTION · RULE F-006Every currency used has an entry in the currency decimals table (TCURX)9 rows in currency XAU held · 1,204,118,921 landedOwner: Priya Sharma · finance data owner · due 16:00 todayFix and replay from hereShow the rowsLINEAGE FOR ONE NUMBER · RECORDED FROM THE RUNCONFORMED.ACCOUNTING_LINE.amount 1,234.00 JPY ← decimal shift (× 100) ← RAW_ECC.BSEG.WRBTR 12.34, document 4900012345 line 1, SAP ECC client100SHOWCASE ESTATE · the same screen for every load · a fix is applied where the row stopped and the replay starts from that step

Integration suites such as Informatica sell data integration, data quality, the catalog and lineage as separate services on one consumption meter, and a team stitches the four together for each load. Lima runs the four as one surface. The screen above is the same for every load into Snowflake: each step from the SAP or Oracle table to the Snowflake table shows its row count; the rule that stopped a row shows the row and the person who owns the fix; the fix is applied where the row stopped and the replay starts from that step. Column-level lineage is recorded from the run itself, so the trace from a number in Snowflake back to the ERP table is current every morning without a scanning project.

Rules in plain language

Checks a planner can read

A rule such as "confirmed date is on or after order date" is written in words, previewed on live rows, versioned, and owned by a named person.

Exceptions with owners

Nothing disappears

A row that fails a rule becomes an exception with the row, the rule, an owner and a deadline. The load carries on with the valid rows and says what it held.

Lineage from the run

Always current

Every step records what it read and wrote at column level, so the trace from a planning number back to the ERP field is captured as a by-product of running.

Governed planning master data

Parameters with owners

Values the ERP never held — hierarchies, changeover matrices, demand-stream rules — live in Lima with an owner and an effective date.

Acknowledged write-back

The loop closes

Results return to the ERP as transactions the ERP confirms; the confirmation is stored and reconciled against what was sent.

Deadlines on the pipeline

The planning run waits for nothing

Each load carries its window and deadline; a tripped rule pauses the run and the clock, and the run resumes from its checkpoint after the fix.

Step 6 · Keep it current, then use it

Change data keeps it current; the fabric reads it live

After the initial load, Lima keeps the raw and conformed layers minutes behind the source, and the same data serves questions, agents and planning loads without another copy.

Change data, merged by key

Inserts, updates and deletes found per table by the method that table allows, merged into the raw layer, then re-conformed; late rows replayed from the watermark; 1 minute 48 seconds behind SAP on the showcase estate.

Replication and sync →

Your warehouses do the querying

Extraction and transformation ran on Lima's cluster; your Snowflake credits go to the analytics and models you run on the conformed layer.

Questions across systems

ChatLima answers a question that joins Snowflake history with live SAP stock in one query, with one access policy applied to both.

Planning loads from the lakehouse

Demand history in Snowflake feeds the planning system on the calendar through the same reconciled pipeline.

Planning integration →
The Terabyte Proof-of-Scale, step by step

Fourteen days on the showcase estate

How the proof-of-scale runs into Snowflake on the showcase estate's largest tables. Figures are from the showcase estate.

Day 1
scope
The largest tables chosenAccounting lines (1.2 billion rows), material documents, sales-order items and the material master from SAP ECC, plus Oracle order lines; a Snowflake database, role and load warehouse created in your account.
Day 2
profile
Counted before anything moves2,480 non-empty tables, 3.9 TB; BSEG partitioned into 132 monthly slices; 14 currencies in use, all with decimals entries except one precious-metal currency.
Days 3–6
initial load
In parallel, on 128 vCPUPartitions loaded side by side through internal stages and COPY INTO; two partitions rerun from their checkpoints after a source timeout; every table reconciled by count and by amount totals per company code and period.
Day 7
change data
Minutes behindChange capture switched on per table — change documents for master data, timestamps where S/4HANA has them, triggers elsewhere; deletes applied; lag settles at under two minutes.
Day 9
cost
Cost per terabyte, statedThe report shows terabytes moved, rows per minute per path, reconciliation variance (zero after the nine held rows were fixed) and cost per terabyte beside the metered alternative running today.
Day 11
use
Read liveThe fabric reads the conformed layer beside live SAP stock; a planning load draws demand history from Snowflake on the calendar.
Day 14
hand-over
Report signedReconciliation report, method per table, and the exception register handed over; the migration extends to the rest of the estate at the same tier.
Partner status

Stated as it is

ProgramWhat it gives youStatus
Snowflake Partner ConnectOne-click provisioned trial as the SAP-aware ingestion layer beside SnowflakePlanned · 2027, after connector pages and docs
Snowflake partner listingRecognised technology partner statusPlanned
Statuses are stated as they are and dated when they change.
Lifecycle

Migration and replication use four stages at volume; Return is there the day you need planning write-back.

One engine underneath every page in this family. It reads from any source and writes to any target, runs batch loads or change-data streams, handles a nightly delta or a multi-terabyte migration, and is licensed by capacity, so the bill stays the same however many rows move. Lima's fabric answers questions on data in place; Lima Backbone moves, transforms and returns data on a deadline. When to federate and when to pipeline · The Backbone hub.

Terms on this page

Plain definitions

Internal stageSnowflake's landing area for files before COPY INTO loads them into a table; the fastest path for bulk history.
MERGEA write that inserts new rows, updates changed ones and deletes removed ones by key — the path for change data.
Clustering keyThe columns Snowflake uses to organise a large table so queries on those columns scan less.
Cluster table (ECC)An SAP storage form where several logical tables are packed into one physical table; readable only by unpacking it.
Currency decimals (TCURX)SAP's table of currencies whose stored amounts must be shifted because they use other than two decimals.
WatermarkThe last change position a pipeline has processed; late rows are replayed from it.
Questions we hear

Asked before the first load.

Snowflake has its own ingestion tooling. Where does Lima fit?

Lima is the SAP-aware layer: it reads SAP through interfaces SAP keeps open, fixes the six problems above on the way, reconciles every table, and runs the change-data stream. Other sources can keep their existing path; Snowflake keeps doing the querying.

Does Lima consume Snowflake credits to load?

Loads use a warehouse you size for loads; extraction and transformation run on Lima's cluster. Your analytics credits go to the queries you run.

Can we keep the raw SAP values?

Yes. The raw layer keeps every table as loaded, and the conformed layer keeps the raw value beside every converted one, so any number traces back to what SAP stored.

What about the ODP RFC deadline?

Lima's methods sit outside the interface SAP restricted in Note 3255746, so the December 2026 change has no effect on this path.

How current is the copy?

Change-data capture ran 1 minute 48 seconds behind SAP in the showcase estate. Bulk refreshes run on the schedule you set.

Run the Terabyte Proof-of-Scale

Two weeks on your largest tables into Snowflake: terabytes moved, rows per minute, reconciliation variance, and cost per terabyte beside the metered alternative you run today.