Skip to main content

​
What is a Data Mine?

Data Mining is a way to start automations from data that lives in your data warehouse and is exposed in Elementum through CloudLinks. You define a Data Mine on a Table that reads that data: Elementum runs your criteria on a schedule, tracks whether each row matches or does not match, and fires automations when a row transitions between those states (for example, from unmatched to matched, or the reverse). A Data Mine does not run arbitrary one-off queries for reporting—it watches the same logical rows over time and pairs that behavior with state management so you are not notified on every scan when nothing has changed for a given row. When your criteria match a row, that row is ON (matched); when they do not, it is OFF (unmatched). Automations run on transitions between those states, not on every evaluation where the state is unchanged. Key components:
  • Data source — Rows from a Table backed by a CloudLink to your data warehouse (for example Snowflake or BigQuery, depending on your connection).
  • Matching criteria — Filters that define which rows count as matched.
  • State management — Per-row tracking of matched vs. unmatched across runs.
  • Automations — Workflows you attach via the Data Mine trigger.
Important: A Data Mine alone does not complete work in your process. Connect it to automations that take the next steps (notifications, updates, handoff to an agent, and so on). Decide those steps before you finalize the Data Mine.

​
When to Implement Data Mines

Use Data Mining when you need scheduled, condition-based reactions to warehouse data: thresholds, SLAs, inventory levels, open claims, and similar rules. It fits batch-style monitoring (for example hourly or daily), which is also how many CloudLink-backed tables are refreshed. For very fast reactions when a record changes inside an Elementum app, record triggers and other automation triggers may be a better fit—see Automations and Change monitoring. Outcomes you can expect:
  • Automations start when a condition becomes true (or false) for a row, not on every run.
  • Teams spend less time scanning reports to find exceptions.
  • You can chain work: notify, assign, update records, or call Agents when your automation design includes them.
Data Mining connects warehouse-backed table data to automations. CloudLinks provide access to that data; Data Mines define what to watch and when to run; automations define what happens next.

​
How Data Mines Operate

  1. Table — Use a Table whose source is CloudLink data you want to monitor (see Tables).
  2. Data Mine — In that Table, create a Data Mine (see Create a Data Mine), set identifying columns, matching criteria, and a schedule.
  3. Automation — Create an automation with a Data Mine trigger and choose whether it runs when data meets or no longer meets your criteria (see Scheduled triggers — Data Mine).

​
Understand state management: ON and OFF

Instead of firing an automation on every scheduled run for every matching row, Elementum tracks state for each row (using your identifying columns). Automations tied to the Data Mine run when state changes in a way you configured. How state transitions map to events: 
Record state: OFF → ON  = Automation can run for "Data Mine Triggered" (condition newly met)
Record state: ON → ON   = No transition; no new trigger for that row
Record state: ON → OFF  = Automation can run for "Data Mine Cleared" (condition no longer met)
Record state: OFF → OFF = No transition; no new trigger for that row
This supports workflows such as: alert when a case first crosses a threshold, stay quiet while it stays above the threshold, then run a different path when it drops back below.
Scenario: Claims over $10,000 while status is open need immediate attention.Data Mine setup:
  • Source: Claims table (CloudLink)
  • Criteria: claim_amount > 10000 AND status = 'open'
  • Schedule: Every 15 minutes
State behavior:
  1. New high-value open claim (OFF → ON): Automation runs (for example assign adjuster, notify a queue).
  2. Claim still matches (ON → ON): No additional automation from this transition.
  3. Claim closed or amount no longer in range (ON → OFF): Automation runs for clearance steps (for example update a dashboard record, notify that the exception cleared).
Scenario: Reorder when stock falls below a threshold for active SKUs.Data Mine setup:
  • Source: Inventory table (CloudLink)
  • Criteria: stock_level < reorder_threshold AND status = 'active'
  • Schedule: Hourly
State behavior:
  1. OFF → ON: Start reorder or approval automation.
  2. ON → ON: No duplicate reorder signal for the same row until state changes.
  3. ON → OFF: Run follow-up when stock is back above the threshold (for example notify sales).
Scenario: Tickets open past the response window without a first response.Data Mine setup:
  • Source: Support tickets table (CloudLink)
  • Criteria: Example pattern: open tickets where the clock has passed your SLA and first_response_date is still null—adjust fields to match your schema.
  • Schedule: Every 30 minutes
State behavior:
  1. OFF → ON: Escalation or reassignment automation.
  2. ON → ON: No repeated escalation for the same breach state.
  3. ON → OFF: Automation when the ticket is responded to or no longer breaches.

​
Working with Agents

Data Mine triggers can start automations whose actions include Agents: for example summarizing the matched row, classifying it, or drafting a reply. Agent behavior depends on how you configure prompts, tools, and steps in that automation—not on automatic “learning” stored inside the Data Mine itself. For details, see Agent tools, deployment, and integrations and the AI overview.

​
Types of Data Mining

When you create a Data Mine, you choose a type that matches how you want to detect rows of interest. The product may show one or more of the following.
Best for: Clear business rules and known patterns.Example: Alert when any order exceeds $5,000.
Criteria: order_total > 5000
Actions: Route to approval workflow, notify finance team
You define explicit filters (comparisons, ranges, null checks, time windows) that rows must satisfy. Most guided documentation (including the UI steps below) uses this type.
Available Data Mine types depend on your workspace and product configuration. If an option is not shown in the UI, use the types your environment supports and follow the on-screen steps for that type.

​
Create a Data Mine in the UI

These steps align with guided flows elsewhere in the docs (for example file and document monitoring). Exact labels may vary slightly by workspace.
  1. Open the Table that uses your CloudLink data.
  2. Go to Data Mining → Create Data Mine → Logic-Based Rules Mining.
  3. Identifying columns — Choose stable columns so the same business row is recognized across runs (IDs, composite keys). See Define identifying columns.
  4. Matching criteria — Set filters for rows that should count as matched. See Build matching criteria.
  5. Name and schedule — Name the Data Mine and set how often it runs.
Then build an automation:
  1. Go to Automations → Create Automation.
  2. Add a Data Mine trigger and select this Data Mine.
  3. Set the trigger to Trigger when data meets requirement and/or Trigger when data no longer meets requirement, depending on whether you need ON or OFF transition handling. Details and option names are in Automation triggers reference — Data Mine.

​
Best Practices for Data Mine Creation

Before creating a Data Mine, decide:
  • What should happen the first time a row matches (OFF → ON)?
  • What should happen when it stops matching (ON → OFF), if anything?
  • Who or what receives the work (users, records, Agents)?
  • Which fields from the row the automation needs?
Choose a CloudLink-backed Table where:
  • Data is sufficiently current for your process.
  • You have permission to read the objects you reference.
  • Refresh timing matches how often you need to evaluate conditions (see Change monitoring).
Purpose: Identifying columns let the system recognize the same logical row across Data Mine runs so state (OFF/ON) is stable.Practices:
  • Prefer stable unique identifiers (IDs, UUIDs).
  • Add business keys when they help disambiguate (order number, case ID).
  • Avoid using columns that change every run unless they are part of your identity rule.
Example (support tickets):
Identifying columns:
- ticket_id (primary identifier)
- customer_id (context)
- created_date (if required for your keying strategy)
Simple conditions:
WHERE priority = 'High'
WHERE amount > 1000
WHERE status IN ('pending', 'review')
Grouped logic:
WHERE (
    (priority = 'High' AND amount > 500)
    OR
    (priority = 'Critical' AND amount > 100)
)
AND status = 'active'
Time-based conditions (adjust functions to your warehouse SQL dialect):
WHERE created_date > NOW() - INTERVAL '7 days'
WHERE last_updated < NOW() - INTERVAL '24 hours'
Scheduling: Match frequency to the process—tighter schedules (for example 15–30 minutes) when latency matters; hourly or daily when batch refresh and cost matter more.Volume: Each Data Mine can process up to 100,000 rows per run (see Field updates). For performance and predictable run times, keep the matching set as small as practical with selective criteria; very large match sets may need longer runs or a different split of rules.
  • Start with one clear business rule per Data Mine when possible.
  • Name Data Mines and automations so owners know what is monitored.
  • Document assumptions (SLA definition, thresholds, timezone).
  • Revisit criteria when business rules change; archive Data Mines you no longer use.
  1. Use VIEW MATCHING DATA (or the equivalent in your Table) to confirm which rows match.
  2. Start with a narrow criteria or time window if possible.
  3. Confirm automations fire on OFF → ON and ON → OFF as intended.
  4. If the automation uses an Agent, run end-to-end tests with realistic rows.

​
Advanced patterns

Cascade: One automation updates data that a second Data Mine or process depends on—design order and idempotency carefully. Threshold ladders: Separate Data Mines or criteria bands for different severities (for example response time tiers). Trends over time: Compare snapshots or use conditions that encode “worse than prior period” if your table holds the needed history.

​
Troubleshooting

  • Confirm CloudLink and Table access.
  • Validate criteria against sample rows.
  • Check schedule and that a state transition occurred (not ON → ON).
  • Review the Data Mine trigger configuration on the automation.
  • Tighten criteria.
  • Run less often if business rules allow.
  • Confirm you are not relying on repeated ON → ON when you need only the first match.
  • Reduce the matched row count with stricter filters.
  • Align schedule with warehouse refresh and off-peak windows where applicable.
  • See Data best practices for limits and change strategies.
  • Review agent configuration and prompts in the automation.
  • Confirm the trigger passes the fields the agent needs.

​
Measuring results

Track what matters for your process: time to act on exceptions, number of manual touches avoided, and error rates. A simple cost view is: 
ROI ≈ (time saved × loaded cost rate × frequency) − setup and maintenance effort
Use it only with numbers your organization accepts; the value is in consistent measurement, not the formula itself. For automation design, see Automation System. For trigger options and variables, see Automation triggers reference.