How does Data Lake Sync handle deleted records?

Overview

In the Infor Data Lake, records are never physically removed. Instead, each record includes a delete flag that indicates when a record has been deleted in the source system. Data Lake Sync preserves this behavior by carrying the same flag into the customer’s target database. This approach is known as a soft delete.

What is a Soft Delete?

A soft delete means that when a record is deleted in the source system, it remains present in the database but is marked as deleted. No data is physically removed. The record can still be used for historical tracking, audit trails, and change-capture logic. This differs from a hard delete, where the data is permanently removed.

Why We Use Soft Deletes

Soft deletes have several key advantages:

• Auditability: Deleted data remains available for traceability and recovery.

• Change Detection: Because the record still exists, downstream integrations can easily detect that it has been deleted.

• Consistency: Ensures that all consuming systems receive a clear “deleted” signal rather than silently losing data.

• Flexibility: Enables business logic, triggers, or analytics processes to respond to deletions in a controlled way.

Customer Responsibilities

It is important for customers to handle soft deletes properly within their own reporting and integrations. If not managed, deleted records will continue to appear in queries and dashboards.

The typical approach is to expose only active data through a view. For example:

CREATE VIEW active_items AS
SELECT *
FROM csi_item_mst
WHERE deleted = FALSE;

By using such a view, reporting and analytics tools automatically exclude deleted records while preserving the underlying audit history.

Soft deletes ensure that no data is lost while keeping your analytics consistent, traceable, and reliable.