Fourteen days on the showcase estate
How the proof-of-scale runs into Snowflake on the showcase estate's largest tables. Figures are from the showcase estate.
scope
profile
initial load
change data
cost
use
hand-over
Lima Backbone lands SAP and Oracle history in your Snowflake account as native tables — raw schemas as loaded, a conformed schema across ERPs — with every table reconciled, then keeps it current with change-data capture minutes behind the source. Extraction runs on Lima's capacity through interfaces SAP keeps open, so your Snowflake credits stay for analytics and there is no additional SAP licence line.
Everything lands in your account, in native Snowflake tables you own. The raw layer keeps SAP and Oracle as they are; the conformed layer is where the decimals are shifted, the keys trimmed and the ERPs joined into one item, site and order.
| Layer | What lands there | How Lima writes it | Status |
|---|---|---|---|
| RAW_ECC · RAW_S4 · RAW_ORACLE | Every extracted table as loaded, productive client only, raw values kept, typed correctly (NUMBER with scale, DATE, TIMESTAMP_NTZ) | Internal stage + COPY INTO in parallel by partition; MERGE by full key for change data | Shipped |
| CONFORMED | Item, site, customer, order and accounting tables across ERPs with decimals shifted, keys trimmed, texts and code descriptions joined, clustered by period and company code | Built from the raw layer by Lima operators; reconciled against raw totals | Shipped |
| HISTORY | Closed periods and retired systems under retention rules and legal hold | Offload and decommissioning runs | Shipped |
| Horizon catalog and policies | Tags, masking and access policies on the landed tables; Lima's lineage recorded beside them | Applied on write; Lima's catalog holds the SAP-to-table trace | Shipped |
| Iceberg tables on your storage | Open table format written to your object storage for engines beside Snowflake | On the published standards schedule | Scheduled · standards plan |
These are the reasons a straight copy of ERP tables produces wrong totals. Each has a cause in how the ERP stores data and an effect an analyst sees.
Three things are specific to landing in Snowflake. Types: SAP amounts need NUMBER with the right scale and dates need DATE and TIMESTAMP_NTZ, or comparisons and bucketing fail. Clustering: a billion-row accounting table needs a clustering key on period and company code, or every month-end query scans the whole table. Two write paths: bulk history is fastest through an internal stage and COPY INTO in parallel; change data needs MERGE by the full key with deletes applied — using one path for both makes the migration slow or the daily changes wrong.
The figure follows one accounting line from SAP into Snowflake. The table maps all six problems and the Snowflake-specific ones.
| Problem | What Lima does | The capability behind it | What you see |
|---|---|---|---|
| 1 · Client and org | Filters to the productive client (and the operating units you name) on every table, and records the client in the landed row | Connector settings per source; lineage records the filter | Row counts per client before and after the filter |
| 2 · Keys | Keeps every key column, adds a trimmed business key beside the raw one, and enforces uniqueness on the full key before the write | Key-aware transformation operators; uniqueness rules | Key coverage per table; duplicates listed with owners |
| 3 · Dates and times | Converts eight-character dates and six-character times to real date and timestamp columns, turns the zero date into null, and converts system time to UTC with the source's time zone | Type-aware SAP operators | Conversion counts per column; anomalies listed |
| 4 · Amounts and units | Applies the currency decimals table to every amount, keeps the unit beside every quantity, and checks totals per company code and period against the ERP | Currency and unit rules; control totals | Totals per company code and period, raw and shifted, against the ERP |
| 5 · Wide and cluster tables | Reads ECC cluster tables through a named function module that unpacks them, and lands wide tables with their full column set, partitioned by period | Named SAP read paths; partitioned parallel loads | Column coverage per table; partitions with counts |
| 6 · Change capture and deletes | Finds changes through change documents, change pointers, timestamps or triggers depending on the table, applies deletes, and replays late rows from the watermark | Change-data capture with per-table methods; ordered replay | Method per table; lag and deletes applied per run |
| Snowflake · types and clustering | Creates each table with NUMBER(p,s), DATE and TIMESTAMP_NTZ from the SAP dictionary types, and sets clustering keys on period and company code for the large fact tables | Target-model catalogue for Snowflake | Column types per table; clustering keys stated |
| Snowflake · write paths | Bulk history through internal stages and COPY INTO in parallel by partition; change data through MERGE by full key with deletes applied | Two write operators, chosen per run | Rows per minute per path; MERGE counts (inserted, updated, deleted) |
Every check is a rule with an owner. A failed partition reruns from its checkpoint; a failed row is held with the rule and the row; the report stays open until every table reads Reconciled.
| Check | Kind | Example | When it fails |
|---|---|---|---|
| Row count per table and per partition equals the source, for the productive client only | Completeness | BSEG fiscal 2024: 132 monthly partitions, each counted | Partition reruns from its checkpoint; report stays open until they match |
| Amount control totals per company code and period match the ERP's own totals after the decimal shift | Consistency | Sum of BSEG amounts for company code 1000, period 2024-03, in JPY and EUR | Exception to finance with both figures |
| Quantity control totals per unit match the source | Consistency | MSEG quantities per unit of measure, plant 1000 | Exception to the data owner |
| Every key is unique in the target and every key column is present | Quality | Company code, document, year and line together on accounting lines | Load refused for the partition; owner notified |
| Zero dates become null; no date lies outside the valid range | Quality | '00000000' in 1.4 million rows of one table | Converted and counted; anomalies listed |
| Every currency used has a decimals entry; every unit pair used has a conversion | Completeness | 9 rows in a currency with no entry in TCURX | Rows held; exception to finance |
| Rows deleted in the source are removed in the target | Consistency | Weekly full-key compare on schedule lines and conditions | Deletes applied; count reported |
| In the conformed layer, every child has its parent | Consistency | Every sales-order item has its header; every line has its material | Exception to the data owner |
| Change events applied in order; lag below the agreed threshold | Consistency | Watermark per table; lag shown on the dashboard | Run pauses and resumes from the checkpoint |
| Column types and scale preserved: no truncation of amounts or keys | Quality | NUMBER(15,2) for amounts; VARCHAR(18) for material numbers | Load refused for the table; owner notified |
Integration suites such as Informatica sell data integration, data quality, the catalog and lineage as separate services on one consumption meter, and a team stitches the four together for each load. Lima runs the four as one surface. The screen above is the same for every load into Snowflake: each step from the SAP or Oracle table to the Snowflake table shows its row count; the rule that stopped a row shows the row and the person who owns the fix; the fix is applied where the row stopped and the replay starts from that step. Column-level lineage is recorded from the run itself, so the trace from a number in Snowflake back to the ERP table is current every morning without a scanning project.
A rule such as "confirmed date is on or after order date" is written in words, previewed on live rows, versioned, and owned by a named person.
A row that fails a rule becomes an exception with the row, the rule, an owner and a deadline. The load carries on with the valid rows and says what it held.
Every step records what it read and wrote at column level, so the trace from a planning number back to the ERP field is captured as a by-product of running.
Values the ERP never held — hierarchies, changeover matrices, demand-stream rules — live in Lima with an owner and an effective date.
Results return to the ERP as transactions the ERP confirms; the confirmation is stored and reconciled against what was sent.
Each load carries its window and deadline; a tripped rule pauses the run and the clock, and the run resumes from its checkpoint after the fix.
After the initial load, Lima keeps the raw and conformed layers minutes behind the source, and the same data serves questions, agents and planning loads without another copy.
Inserts, updates and deletes found per table by the method that table allows, merged into the raw layer, then re-conformed; late rows replayed from the watermark; 1 minute 48 seconds behind SAP on the showcase estate.
Replication and sync →Extraction and transformation ran on Lima's cluster; your Snowflake credits go to the analytics and models you run on the conformed layer.
ChatLima answers a question that joins Snowflake history with live SAP stock in one query, with one access policy applied to both.
Demand history in Snowflake feeds the planning system on the calendar through the same reconciled pipeline.
Planning integration →How the proof-of-scale runs into Snowflake on the showcase estate's largest tables. Figures are from the showcase estate.
| Program | What it gives you | Status |
|---|---|---|
| Snowflake Partner Connect | One-click provisioned trial as the SAP-aware ingestion layer beside Snowflake | Planned · 2027, after connector pages and docs |
| Snowflake partner listing | Recognised technology partner status | Planned |
One engine underneath every page in this family. It reads from any source and writes to any target, runs batch loads or change-data streams, handles a nightly delta or a multi-terabyte migration, and is licensed by capacity, so the bill stays the same however many rows move. Lima's fabric answers questions on data in place; Lima Backbone moves, transforms and returns data on a deadline. When to federate and when to pipeline · The Backbone hub.
Lima is the SAP-aware layer: it reads SAP through interfaces SAP keeps open, fixes the six problems above on the way, reconciles every table, and runs the change-data stream. Other sources can keep their existing path; Snowflake keeps doing the querying.
Loads use a warehouse you size for loads; extraction and transformation run on Lima's cluster. Your analytics credits go to the queries you run.
Yes. The raw layer keeps every table as loaded, and the conformed layer keeps the raw value beside every converted one, so any number traces back to what SAP stored.
Lima's methods sit outside the interface SAP restricted in Note 3255746, so the December 2026 change has no effect on this path.
Change-data capture ran 1 minute 48 seconds behind SAP in the showcase estate. Bulk refreshes run on the schedule you set.
Two weeks on your largest tables into Snowflake: terabytes moved, rows per minute, reconciliation variance, and cost per terabyte beside the metered alternative you run today.