Data Lake Sync Architecture

Data Lake Sync is a powerful tool that automates the movement of enterprise data from the Infor platform to your external systems. What makes it efficient and secure isn’t just the technology it connects to — it’s how it connects, transforms, and delivers data without ever storing it locally. This article breaks down how Data Lake Sync operates behind the scenes.

Pulling Data from Infor APIs

At its core, Data Lake Sync leverages Infor’s API Gateway to access the latest enterprise data. It supports two key API types:

• Compass APIs

• Object APIs

By supporting both Compass and Object APIs, Data Lake Sync gives you additional flexibility.

Intelligent Schema Generation & Sync Logic

Data Lake Sync doesn’t just move data — it structures it. Using automated schema discovery and transformation logic, Data Lake Sync:

• Determines the structure of incoming data

• Generates a corresponding schema in the destination database

• Maps and transforms data types and fields as needed

• Handles inserts, updates, and soft deletions to keep your target tables in sync

This means you don’t need to manually define schemas — the system understands the source structure and replicates it in a way that’s optimized for querying and reporting.

Streamed, Not Stored

Unlike traditional ETL tools, Data Lake Sync does not store your data during transfer. It functions as a batched pipeline:

1. Data is retrieved from the Infor API Gateway.

2. It is processed, transformed and mapped.

3. It is immediately written to the destination system.

Secure Credential Storage

All credentials used by Data Lake Sync to connect to external destination systems, such as databases or data stores,  are stored securely using AWS Secrets Manager.

This ensures that:

• Credentials are never hard-coded or exposed in configuration files or logs.

• Access to secrets is strictly controlled using fine-grained IAM policies.

• Secrets are encrypted at rest using AWS Key Management Service (KMS).

• Credentials can be rotated automatically if supported by the destination system.

When a sync job runs, Data Lake Sync retrieves the required credentials securely from Secrets Manager at runtime. This design keeps sensitive information protected while still enabling automated, unattended data movement to your target systems.

Encryption in Transit

Security is built into every step:

• From Infor APIs: Data is retrieved using encrypted connections through the Infor API Gateway, ensuring confidentiality and integrity.

• To Destination Storage: Data is transmitted over secure, encrypted channels (TLS/SSL) to your destination database or storage platform.

This means your data is encrypted in transit at every stage, minimizing exposure and aligning with enterprise-grade security standards.

Seamless Data Availability

Whether your use case involves syncing to a SQL data warehouse, Azure blob storage, S3 or SFTP, Data Lake Sync provides a scalable, secure, and low-maintenance pipeline. There’s no need to manage intermediate staging areas, and schema changes at the source are intelligently handled and reflected in your destination system.

Summary