User and Department Sync Strategy in Practice: A Lightweight Landing from CRM to Integration Platform

What this strategy solves

In a privately deployed environment where CRM and ERP coexist, the question of "who belongs to which department" is confirmed by business and IT almost every day. In a real project at a manufacturing enterprise, CRM was treated as the single source of truth for sales staff, while ERP was treated as the single source of truth for the organizational structure and permissions. Once the user tables on both sides are misaligned, reimbursements, approvals, and data permissions downstream will all drift. The core purpose of this strategy is to regularly pull the CRM-side user master data and its associated departments into the Qeasy data integration platform for normalization, and then hand it over to subsequent base data strategies for distribution downstream.

Data flow and field mapping

This strategy is a lightweight landing of "source → middle layer". The source is the CRM user query interface, the middle layer is the Qeasy data integration platform, and the target is a "write null operation" placeholder interface, waiting to be consumed by subsequent strategies.

The key field mapping is as follows:

Both the department name and the department code are intentionally retained. One of the common response patterns we see at the customer site is "centralized management of code mapping": first land the name and code together, and then use a separate mapping table downstream to avoid repeated maintenance in each strategy.

How to configure it on Qeasy

The metadata of the source strategy is a typical query type: the interface path is /v1/user/list, GET request, pagination is automatically taken over by Qeasy, the document number field is set to nickname, the document ID field is set to user_id, the document existence check is turned off (idCheck=false), the write model is not built, and the response fields are defined according to the table above.

The target side first attaches a "write null operation" WebAPI node, method POST, with an empty request body. This serves two purposes: first, to land the data steadily on the Qeasy integration platform; second, to leave a stable subscription point for the subsequent "department → ERP organization" and "user → ERP operator" strategies, so as to avoid putting pressure on downstream systems at the very beginning.

There are three configuration points worth elaborating on:

First, the crontab on the source side is recommended to be scheduled during the early morning off-peak period, for example 3 2 * * *; the target placeholder strategy can be scheduled a little later, for example 23 2 * * *, to leave enough landing window for the source side. This reflects the "header and body in phases" approach in scheduling time.

Second, since Qeasy data integration platform does not write back to the source system for query-type strategies by default, there is no need to worry about reverse synchronization here.

Third, when all response fields are of type string, make sure to explicitly declare the primary key field on the middle layer; otherwise, downstream subscriptions will not be able to obtain a stable basis for deduplication.

Implementation steps

We usually advance in three phases:

Phase 1: incremental starting point initialization. Trigger a full pull once, and pull the current snapshot of CRM users and departments into the Qeasy integration platform in one go. This step is completed in the strategy through "manual execution + full mode", and then switch back to incremental after the run.

Phase 2: scheduling frequency landing. The source strategy is executed in the early morning every day, depending on the final state of CRM on the previous day; the target placeholder strategy is executed some time after the source strategy completes, taking over the landing. Strategies that actually distribute to ERP later will take the middle layer data as the starting point and run at their own pace.

Phase 3: monitoring and reconciliation. The operation logs built into the Qeasy data integration platform can show the number of records fetched each time and the failure details; at the customer site, a simple reconciliation table is usually added to compare the daily CRM user count with the middle layer landing count, and an alarm is triggered when the difference exceeds a threshold. This is a common combination of "incremental and full dual tracks": incremental runs are performed daily, and a full backstop is performed weekly or monthly.

Pitfalls and lessons learned

1. Don't look only at department_name for the department field. In a real project, the customer only pulled the name. When the downstream tried to do the organizational mapping, it was found that there was a department with the same name in each business line, and the codes were completely different. In the end, fields had to be traced back and added. The safe approach is to land both the name and the code together.

2. Don't use user_id as the email. A typical mistake is to directly treat user_id as the downstream ERP operator code. Once CRM adjusts the ID rules, all downstream matching will fail. The middle layer only does transportation, not semantic merging.

3. Don't do write operations in a query strategy. The strategy type is QUERY_ONLY, and the target side uses a "write null operation" placeholder, which does not actually write to a third-party system. A common pitfall is that someone casually puts a write script into the source strategy, resulting in coupling between query and write, making troubleshooting difficult later.

4. Time zone and day boundaries. If the CRM fields carry local time zone timestamps, when the downstream slices by calendar day, "one day missing" or "one day extra" will occur. It is recommended to uniformly use the server time zone on Qeasy, and clearly state it in the documentation.

5. Pagination and deduplication. When the interface does not explicitly specify pagination fields, Qeasy defaults to judging based on the length of the response array; however, if the interface occasionally returns duplicate user_id, downstream subscriptions will process it repeatedly. The safe approach is to perform an explicit primary key deduplication on the middle layer.

Applicable and non-applicable scenarios

This strategy applies to: a privately deployed environment where CRM is the single source of truth for user master data, and ERP needs to consume this source in a "source → middle layer" lightweight landing scenario. It does not apply to: scenarios where the CRM and ERP user sources have already been unified on one side and no middle layer normalization is required, as well as scenarios that require extremely high real-time performance (such as second-level synchronization of permission changes).