Practical Guide to Scheduled Cleanup of Queues and Logs with Qeasy DeleteExceptData

What This Strategy Solves (Scenario and Value)

In a supply chain integration project for a retail enterprise, the Qeasy platform generates a large volume of queue records and API logs every day. After three months, operations reported slower platform responses. Investigation revealed that historical data occupied most of the storage. We use the "delete all queues and logs older than three days" strategy for periodic slimming, clearing out traces older than three days to keep the platform stable in the long run.

Data Flow and Field Mapping (Source → Middleware → Target)

This strategy is essentially an O&M action of the platform itself. It does not involve external business systems, but it still follows the "source request → platform processing → target write-empty-operation" chain.

The source API is DeleteExceptData, type WebAPI, effect QUERY, method POST, used to query and delete data older than three days. The target is Write empty operation, effect EXECUTE, carrying no request or response fields.

How to Configure on Qeasy

We usually set it up in the Qeasy data integration platform as follows:

1. Create a new strategy, selecting a category such as "System Maintenance" or similar O&M type;
2. For the source, choose the Qeasy platform's own DeleteExceptData interface, keep the default POST + QUERY behavior, enable idCheck and autoFillResponse;
3. Fill in target_1 with all solution identifiers that need cleanup; multiple can be passed at once;
4. Attach a "write empty operation" action at the target, which serves to make the strategy form a complete source → target chain;
5. Configure the schedule crontab: source at 3 1 * * * (01:03 AM), target at 23 2 * * * (02:23 AM), staggered execution;
6. Keep buildModel off, since this strategy does not need to build complex data models.

One common pattern we have seen on customer sites: centralized code-mapping management. Compile all solution identifiers into an internal checklist, read uniformly from a configuration center, avoid hardcoding scattered across multiple strategies, so that adding or removing solutions later only requires a single change.

Implementation Steps (Phased Scheduling)

• Incremental start: First run a gray release on a small set of solutions, limiting target_1 to one or two non-critical solutions, and observe for 2-3 days to confirm the cleanup is stable;
• Full trigger: After the gray release passes, expand target_1 to all solutions that need governance;
• Scheduling frequency: Running once a day in the early morning is enough. Source runs at 01:03, target wraps up at 02:23, with a buffer in between;
• Inspection mechanism: Check the platform storage curve weekly to confirm the cleanup takes effect; if the curve does not drop, check whether target_1 is missing any solution identifiers.

Pitfall Review

1. A common pitfall: running the strategy without filling target_1. The interface strictly enforces is_required; if the required field is empty, it errors out directly and the strategy will not enter the cleanup flow. The safe practice is to dry-run once in a test environment before going live.
2. A typical mistake is making the cleanup frequency too aggressive. For example, once every hour. With the three-day window being frequently rewritten, almost no historical logs are available for troubleshooting. We usually recommend keeping three days; during major change windows, this can be temporarily relaxed to seven days.
3. A wrong effect on the source causes weird behavior. DeleteExceptData should be configured as QUERY, not EXECUTE; otherwise the platform may treat it as a write action and skip the deletion logic.
4. The target "write empty operation" cannot actually be omitted. Qeasy's strategy orchestration requires both source and target to be configured, otherwise the scheduler will not trigger as expected.
5. crontab timezone pitfalls. The platform defaults to the server timezone for parsing, so a "1 AM in the early morning" mentioned verbally by cross-region teams often differs by a few hours. The safe practice is to specify the timezone in the strategy description and check the logs after the first execution.

Applicable and Non-Applicable Scenarios

Applicable: Qeasy platform running for more than a month, with noticeable queue and log accumulation, and storage or query performance degradation, on public cloud deployments. Not applicable: projects in major change windows, intensive troubleshooting periods, or those required by regulations to retain longer operational traces.