> ## Documentation Index
> Fetch the complete documentation index at: https://docs.elementum.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Management

> Guidelines for organizing, sharing, and managing data in Elementum

Maximize your data's value with proven strategies for organization, sharing, and management. These guidelines cover table and view choices, identifiers, integrity, change monitoring, and loading patterns. For related options by use case, see [Data structure options](#data-structure-options).

<Info>
  **New to data integration?** Start with the [CloudLink setup guide](/administration/connect-snowflake-to-elementum) to establish your data connection first.
</Info>

## Data Sharing Fundamentals

### Tables vs Views: The Foundation Decision

Choose between tables and views based on whether you need writes, how you filter columns, and performance. For a concise matrix of structure types by scenario, see [Data structure options](#data-structure-options) below.

<Tabs>
  <Tab title="Use Tables When">
    #### Write Operations Required

    Use a table when data is updated through automations or user input:

    * User form submissions
    * Automation-driven status updates
    * Real-time data modifications
    * Transactional processing

    #### Performance is Critical

    Use a table when queries run often and latency matters:

    * Dashboard data sources
    * Real-time reporting
    * Frequently accessed reference data
    * Performance-sensitive workflows
  </Tab>

  <Tab title="Use Views When">
    #### Data Subset Needed

    Use a view when you need to share only certain columns or filtered rows:

    * Security-sensitive data with column restrictions
    * Department-specific data views
    * Filtered datasets based on business rules
    * Simplified data presentations

    #### Read-Only Access Sufficient

    Use a view when no writes are required:

    * Reporting and analytics
    * Reference data lookups
    * Audit and compliance views
    * Data exploration and discovery
  </Tab>

  <Tab title="Special Considerations">
    #### Data exchange tables

    When you need both read and write behavior and the presentation benefits of a view, use a view for display and a separate exchange table for updates.

    **Example:** A customer view for display plus a `customer_updates` table for changes.
  </Tab>
</Tabs>

<Note>
  To create automations based on record changes (updates, new records, etc.) or track business entities, create an [Element](/getting-started/fundamentals/core-concepts#data--elements) with your data instead.
</Note>

## Unique Identifiers

Each row in your data must have a unique identifier. If one does not exist, create it using:

* UUID functions
* Concatenated fields
* Incremental number assignment

Always verify uniqueness by running a validation query before linking.

## Table Integrity Guidelines

<AccordionGroup>
  <Accordion title="Critical Changes">
    These changes cause "No Rows" errors:

    * Renaming tables
    * Renaming shared columns
  </Accordion>

  <Accordion title="Changes Requiring Review">
    Consult with Elementum before:

    * Deleting columns
    * Repurposing columns
    * Using Create/Replace table commands
  </Accordion>

  <Accordion title="Low-Impact Changes">
    * Adding new columns (requires Elementum admin to update layouts)
  </Accordion>
</AccordionGroup>

## Data Structure Options

| Use Case             | Recommended Option             | Example                       |
| -------------------- | ------------------------------ | ----------------------------- |
| Read-only access     | Regular table/view             | Data Mining                   |
| Write access needed  | Hybrid table                   | Reclaim License Data Exchange |
| Performance critical | Materialized view/hybrid table | -                             |
| Multi-table queries  | Dynamic table                  | -                             |

For conceptual guidance on tables versus views, see [Tables vs Views](#tables-vs-views-the-foundation-decision) above.

## Change Monitoring

Choose your monitoring approach based on frequency:

<AccordionGroup>
  <Accordion title="Known Frequency">
    * Use data mining for hourly, daily, weekly, or monthly updates
    * Row limits per data mine apply; see [Field updates](#field-updates)
  </Accordion>

  <Accordion title="Real-time Needs">
    * Use record created/updated triggers
    * Enable `CHANGE_TRACKING` for automated updates (configuration details in [Change tracking configuration](#change-tracking-configuration))
  </Accordion>

  <Accordion title="Time-based Updates">
    * Use time-based triggers for known update schedules
    * Elementum detects changes based on resource scheduler or data mine schedule
  </Accordion>
</AccordionGroup>

## Field Updates

### Automation Options

* Use create triggers for non-null initial values
* Use data mines for batch updates (**100,000 row limit** per data mine)
* Enable change tracking for real-time element updates (see [Change tracking configuration](#change-tracking-configuration))

### Calculated Fields

Use for related item data:

```sql theme={null}
STRING_AGG_UNIQUE(HANDLE."Field", ', ')  -- For text
MAX(HANDLE."Field")                      -- For numbers
```

## Change Tracking Configuration

### Capabilities

* Tables: Select, Update, Insert, Delete (with permissions)
* Views: Select only

### Implementation Notes

* Enables automation triggers from tables and Elements
* Supports table modifications for joins and data mines
* Works best when update frequency is known so scheduling stays efficient

## Data Loading Strategies

### Incremental Loading

* Works directly with `CHANGE_TRACKING`
* Efficient for regular updates

### Flush and Fill

* Requires duplicate checking
* Use search actions before create/update operations

## Update Behavior Matrix (Snowflake)

The following describes trigger behavior when using Snowflake with change tracking:

| CHANGE\_TRACKING      | Elementum Triggers | Snowflake Triggers |
| --------------------- | ------------------ | ------------------ |
| Enabled before config | Immediate          | Based on scheduler |
| Enabled after config  | Immediate          | Won't fire         |
| Disabled              | Immediate          | Won't fire         |

## Next Steps

Use these guides to connect your warehouse, align table design with how Elementum uses data, and operationalize monitoring and loads.

<CardGroup cols={2}>
  <Card title="Connect Snowflake" icon="snowflake" href="/administration/connect-snowflake-to-elementum">
    Run the setup script and connect your Snowflake account to Elementum
  </Card>

  <Card title="Data mining" icon="download" href="/data/data-mining">
    Schedule batch pulls, respect row limits, and align mines with your change strategy
  </Card>

  <Card title="CloudLink Overview" icon="link" href="/administration/cloudlink-overview">
    Concepts, authentication, platform schema rules, and the security model
  </Card>

  <Card title="Tables" icon="table-2" href="/data/tables">
    Work with table objects, layouts, and how data surfaces in the workspace
  </Card>

  <Card title="Snowflake table types" icon="layers" href="/administration/snowflake-table-types">
    Choose Standard vs Hybrid tables that match your read, write, and performance needs
  </Card>

  <Card title="Snowflake warehouses" icon="server" href="/administration/snowflake-warehouses">
    Size and configure your Snowflake warehouse for Elementum workloads
  </Card>
</CardGroup>