> ## Documentation Index
> Fetch the complete documentation index at: https://moengage-sarvam.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Databricks Exports

> Export campaign interaction events from MoEngage to Databricks using a direct export or OpenSharing (zero-copy sharing). Configure either method for scalable event analytics.

# Introduction

[Databricks](https://www.databricks.com/resources/ebook/maximize-your-organizations-potential-data-and-ai?scid=7018Y000001Fi1AQAS\&utm_medium=paid+search\&utm_source=google\&utm_campaign=15418435374\&utm_adgroup=130717555576\&utm_content=ebook\&utm_offer=maximize-your-organizations-potential-data-and-ai\&utm_ad=703594935116\&utm_term=databricks\&gad_source=1\&gclid=CjwKCAjwvr--BhB5EiwAd5YbXpGh4ugWZyyOwJrqbfr-c5cZhP9HVrGMKQmBuZRlVcC_BRLco-zKVxoCmZkQAvD_BwE) is a platform that allows organizations to store, analyze, and process large volumes of structured and semi-structured data in a highly scalable and efficient manner. With its unique architecture, organizations can consolidate their data, perform quick analytics, and gain valuable data-driven insights accessible to all users.

# MoEngage `<>` Databricks

With this integration, you can export data from campaign interaction events to your Databricks tables.

## Delivery Methods

MoEngage delivers event data to Databricks in two ways:

* **OpenSharing (Zero-Copy Sharing)**: Your workspace queries MoEngage-hosted data in place through the Delta Sharing protocol. This is the recommended method.
* **Direct export**: MoEngage writes events into a table in your own schema. Use this method when your workspace does not have Unity Catalog enabled.

Both methods produce the same table schema and refresh hourly. The following table compares them:

|                                | OpenSharing (Zero-Copy Sharing) <br />**Recommended**                                | Direct Export                                         |
| ------------------------------ | ------------------------------------------------------------------------------------ | ----------------------------------------------------- |
| **How data arrives**           | Your workspace queries MoEngage-hosted data live through the Delta Sharing protocol. | MoEngage writes events into a table in your schema.   |
| **Credentials MoEngage needs** | None. You provide a sharing identifier, which is not a secret.                       | `WRITE` access to your Databricks catalog and schema. |
| **IP allowlisting**            | Not required.                                                                        | Required if your system enforces it.                  |
| **Where data is stored**       | In a dedicated MoEngage-hosted catalog that you mount.                               | In your own catalog and schema.                       |
| **Requires Unity Catalog**     | Yes.                                                                                 | No.                                                   |
| **Setup time**                 | Up to seven business days after you request enablement.                              | Self-serve on the App Marketplace.                    |

<Note>
  The two methods use separate storage layers. Records already delivered through a direct export do not appear in an OpenSharing catalog, and vice versa. To move between methods, contact your MoEngage Customer Success Manager (CSM).
</Note>

# Integration

## Prerequisites

The prerequisites depend on the delivery method you choose. Click the tab for your method:

<Tabs>
  <Tab title="OpenSharing (Zero-Copy Sharing)">
    * Ensure your Databricks workspace has Unity Catalog enabled.
    * Retrieve your sharing identifier, formatted as `<cloud>:<region>:<metastore-id>`. For more information, refer to [Step 1: Request Enablement](/user-guide/data/exports/events/data-warehouse/databricks-exports#set-up-your-export).
  </Tab>

  <Tab title="Direct Export">
    * Ensure you have a Databricks account with relevant permissions to create/edit Databricks instances, roles, and policies. It is optional to create a new user; you must have access to create a new user in your Databricks database.
    * If your system requires IP whitelisting, you must whitelist MoEngage IPs. For more information, refer [here](/user-guide/settings/account/security/ip-whitelisting-in-moengage).
    * Ensure you have a [Databricks Connection](/partner-guide/data-and-analytics/data-warehouses/databricks) on the App marketplace.
    * Exports to Databricks are part of the Streams add-on. Contact your dedicated MoEngage CSM (customer success manager) to enable them for your account.
  </Tab>
</Tabs>

## Set Up Your Export

<Info>
  * For the full list of events and attributes, refer to the [Data Export Glossary](/user-guide/data/exports/events/data-export-glossary).
  * The event\_time of the exported events is always in UTC.
  * MoEngage does not support exporting historical data. Data flows from the point you enable the export.
</Info>

The setup steps differ by delivery method. Click the tab for the method you are using:

<Tabs>
  <Tab title="OpenSharing (Zero-Copy Sharing)">
    With OpenSharing, your workspace queries MoEngage-hosted data directly through the Delta Sharing protocol instead of receiving file-based exports. You do not exchange credentials or write tokens, and your queries always read the current data. You query the shared catalog with your existing SQL editors, notebooks, and BI dashboards through standard Unity Catalog workflows.

    The following diagram shows how data moves between MoEngage and your workspace:

    <img src="https://mintcdn.com/moengage-sarvam/mE_gdwPZ0a259M_S/images/databricks-openshare-architecture.png?fit=max&auto=format&n=mE_gdwPZ0a259M_S&q=85&s=fef799f5e848b1e5236dfa92e63c4474" alt="Architecture diagram showing MoEngage provisioning an isolated catalog, share, and recipient, connected to your Databricks workspace over a secure Delta Share that you mount as a read-only catalog and query live" width="2644" height="1623" data-path="images/databricks-openshare-architecture.png" />

    **Step 1: Request Enablement**

    To request OpenSharing, perform the following steps:

    1. In your Databricks workspace sidebar, click **Catalog**.

    2. Click **Share**, and then click **OpenSharing**.

           <img src="https://mintcdn.com/moengage-sarvam/mE_gdwPZ0a259M_S/images/databricks-openshare-share-menu.png?fit=max&auto=format&n=mE_gdwPZ0a259M_S&q=85&s=e281e84d3fc697ea4a8a8c2a7ebbf75c" alt="Databricks Catalog page with the Share menu open, showing the OpenSharing option" width="1814" height="383" data-path="images/databricks-openshare-share-menu.png" />

    3. On the **OpenSharing** page, click **Sharing ID** to copy your sharing identifier.

           <img src="https://mintcdn.com/moengage-sarvam/mE_gdwPZ0a259M_S/images/databricks-openshare-sharing-id.png?fit=max&auto=format&n=mE_gdwPZ0a259M_S&q=85&s=c95e170c53ad07e09e43caf707d43e2d" alt="Databricks OpenSharing page with the Sharing ID button highlighted" width="1814" height="394" data-path="images/databricks-openshare-sharing-id.png" />

    Send the sharing identifier to your MoEngage CSM.

    **Step 2: Accept the Share**

    MoEngage usually provisions the share within seven business days. Your CSM then sends you the name of your data share.

    To accept the share, perform the following steps:

    1. In your Databricks workspace, click **Catalog** > **OpenSharing**, and then click the **Shared with me** tab.

           <img src="https://mintcdn.com/moengage-sarvam/mE_gdwPZ0a259M_S/images/databricks-openshare-shared-with-me.png?fit=max&auto=format&n=mE_gdwPZ0a259M_S&q=85&s=bbfd8cc474242747968130c6219cb99f" alt="Databricks OpenSharing page with the Shared with me tab highlighted, listing the available providers" width="1736" height="452" data-path="images/databricks-openshare-shared-with-me.png" />

    2. Click the MoEngage provider that matches the name your CSM sent you.

    3. Next to the share, click **Mount to catalog**.

           <img src="https://mintcdn.com/moengage-sarvam/mE_gdwPZ0a259M_S/images/databricks-openshare-mount-to-catalog.png?fit=max&auto=format&n=mE_gdwPZ0a259M_S&q=85&s=458218225554d9654d4b5530ff33a592" alt="Databricks provider page showing the MoEngage share with the Mount to catalog button highlighted" width="1736" height="561" data-path="images/databricks-openshare-mount-to-catalog.png" />

    4. Enter a catalog name, for example, `moengage_shared_data`, and then confirm it.

    **Step 3: Configure Your Export**

    To configure the export in MoEngage, perform the following steps:

    1. On the left navigation menu in the MoEngage dashboard, click **App marketplace**.
    2. On the **App marketplace** page, search for **Databricks**, and then click the **Databricks** tile.
    3. Click the **Integrate** tab.

    Create the export as described in [Add a New Databricks Export on the MoEngage App Marketplace](/user-guide/data/exports/events/data-warehouse/databricks-exports#add-a-new-databricks-export-on-the-moengage-app-marketplace). In the **Export into schema** list, select the shared catalog's schema. The list shows the same value your CSM shared with you.

    OpenSharing provisions the connection for you, so MoEngage does not prompt you for a connection, service principal credentials, or write access.

    **Step 4: Query Your Data**

    The first rows can take up to 30 minutes to appear in your shared catalog. To verify the data, perform the following steps:

    1. Run the following query:

       ```sql theme={null}
       SELECT * FROM <your_catalog_name>.<schema>.<table_name>;
       ```

       Replace the placeholders with your catalog, schema, and table names.

    2. Click **Catalog Explorer**.

           <img src="https://mintcdn.com/moengage-sarvam/mE_gdwPZ0a259M_S/images/databricks-openshare-catalog-explorer.png?fit=max&auto=format&n=mE_gdwPZ0a259M_S&q=85&s=dc3cb959986584647d7fbd22c971cd21" alt="Databricks Catalog Explorer showing the mounted moengage_shared_data catalog under Shares received, with its schemas and tables" width="1808" height="1009" data-path="images/databricks-openshare-catalog-explorer.png" />

    3. Click a table, and then click **Sample Data**.

    Catalog Explorer lists the shared catalog under **Shares received**, along with its schemas and tables. The **Sample Data** view previews records and attribute definitions.

    If you want MoEngage to walk you through the schema, request a technical schema review from your CSM.

    **Step 5: Manage Access Within Your Organization**

    Because the shared catalog sits in your own infrastructure, you control access through Unity Catalog. Assign the `USE CATALOG` and `SELECT` privileges to the users or groups that need the data.

    MoEngage has no write access to the catalog, and either party can end the connection at any time. Tell your CSM about any changes to your workspace setup or sharing identifier, because these changes can interrupt delivery.
  </Tab>

  <Tab title="Direct Export">
    **Step 1: Grant Your User's Role the Required Permissions**

    MoEngage needs `WRITE` access to your Databricks database to store data. You can grant these permissions to an existing database user or create a new database user for MoEngage.

    Use the following code to configure write access for data storage in your Databricks database:

    ```text wrap theme={null}
    GRANT USE CATALOG ON CATALOG `<CATALOG_NAME>` TO `<EMAIL_ID_OF_THE_USER>`;
    GRANT USE SCHEMA, CREATE TABLE ON SCHEMA `<CATALOG_NAME>`.`<SCHEMA_NAME>` TO `<EMAIL_ID_OF_THE_USER>`;
    ```

    Change the following values:

    * `<CATALOG_NAME>` - The name of your catalog.
    * `<SCHEMA_NAME>` - The name of your database or schema.
    * `<EMAIL_ID_OF_THE_USER>` - The email ID of the user who created the token.

    **Step 2: Add the Export on the App Marketplace**

    Continue with [Add a New Databricks Export on the MoEngage App Marketplace](/user-guide/data/exports/events/data-warehouse/databricks-exports#add-a-new-databricks-export-on-the-moengage-app-marketplace), below.
  </Tab>
</Tabs>

### Add a New Databricks Export on the MoEngage App Marketplace

Both delivery methods use this step. With OpenSharing, the connection already exists, so you only choose the schema, events, and attributes.

To add a new Databricks export, perform the following steps:

1. On the left navigation menu in the MoEngage dashboard, click **App marketplace**.

2. On the **App marketplace** page, search for **Databricks**.

   <img src="https://mintcdn.com/moengage-sarvam/mE_gdwPZ0a259M_S/images/databricks.png?fit=max&auto=format&n=mE_gdwPZ0a259M_S&q=85&s=20fdecdc3144569aa798324eee3ccbd6" alt="MoEngage App Marketplace search results showing the Databricks tile" width="2624" height="1346" data-path="images/databricks.png" />

3. Click the **Databricks** tile.

4. On the **Databricks** page, click the **Integrate** tab, and then click **+ Add Export**.

5. Enter the following details:

   | Field                     | Required | Description                                                                                                                                                                                  |
   | ------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
   | **Export name**           | Yes      | Type a name for the Databricks export.                                                                                                                                                       |
   | **Databricks connection** | Yes      | Select your Databricks connection.                                                                                                                                                           |
   | **Export into schema**    | Yes      | Select the schema that MoEngage uses to create new tables and export your events into.                                                                                                       |
   | **User properties**       | Optional | Select the user properties (user properties and device properties) you want to export with your events.                                                                                      |
   | **Select events**         | Yes      | Select the events you want to export to Databricks. You can choose to export all events, including existing and new ones coming into MoEngage, or manually select specific events to export. |

   <img src="https://mintcdn.com/moengage-sarvam/mE_gdwPZ0a259M_S/images/databricks-add-export-form.png?fit=max&auto=format&n=mE_gdwPZ0a259M_S&q=85&s=bd922f8af3eebaea5565193cc1fb7ef4" alt="Setup exports to Databricks form showing the export name, Databricks connection, export into schema, user properties, and event selection fields, with the Save button highlighted" width="862" height="836" data-path="images/databricks-add-export-form.png" />

6. Click **Save**.

## Export Process and Frequency

Both delivery methods refresh hourly, and neither requires you to run manual sync jobs.

For a direct export, MoEngage creates a destination table in your schema after you save the export, and then appends new event data at hourly intervals:

* **Initial action**: MoEngage creates the table when the first export is triggered.
* **Recurring action**: MoEngage appends data to the same table every hour.

For OpenSharing, MoEngage refreshes the shared datasets every hour. Your queries read the current state of the data, so there is no table for MoEngage to write into on your side.

## Sample Table Structure

MoEngage creates a single table using the following format:\
`moe_events_<table_creation_timestamp>`.

For a direct export, this table is created in your schema. For OpenSharing, it appears in the shared catalog you mounted.

**Note**: This table is created per connection. All hourly exports and different event types configured under this connection are consolidated and appended to this specific table.

The overall structure of the table remains fixed for all event types. Any new user property or event attribute that is exported is included in the user\_attributes or event\_attributes column, respectively. All events are consolidated in a single table containing necessary information, such as event names and event times, for easy analysis:

| Field                                     | Type      | NULL     |
| ----------------------------------------- | --------- | -------- |
| app\_name                                 | VARCHAR   | NOT NULL |
| event\_name                               | VARCHAR   | NOT NULL |
| event\_code                               | VARCHAR   | NOT NULL |
| event\_uuid                               | VARCHAR   | NOT NULL |
| event\_time                               | TIMESTAMP | NOT NULL |
| event\_type                               | VARCHAR   | NOT NULL |
| event\_source                             | VARCHAR   | NOT NULL |
| uid                                       | VARCHAR   | NULL     |
| user\_attributes\_moengage\_user\_id      | VARCHAR   | NULL     |
| device\_attributes\_moengage\_device\_id  | VARCHAR   | NULL     |
| push\_id                                  | VARCHAR   | NULL     |
| email\_address                            | VARCHAR   | NULL     |
| mobile\_number                            | VARCHAR   | NULL     |
| event\_attributes\_app\_version           | VARCHAR   | NULL     |
| event\_attributes\_sdk\_version           | VARCHAR   | NULL     |
| event\_attributes\_platform               | VARCHAR   | NULL     |
| event\_attributes\_campaign\_id           | VARCHAR   | NULL     |
| event\_attributes\_campaign\_name         | VARCHAR   | NULL     |
| event\_attributes\_campaign\_type         | VARCHAR   | NULL     |
| event\_attributes\_campaign\_channel      | VARCHAR   | NULL     |
| event\_attributes\_readable\_campaign\_id | VARCHAR   | NULL     |
| event\_attributes\_parent\_campaign\_id   | VARCHAR   | NULL     |
| event\_attributes\_parent\_flow\_id       | VARCHAR   | NULL     |
| event\_attributes\_parent\_flow\_name     | VARCHAR   | NULL     |
| event\_attributes\_variation\_id          | VARCHAR   | NULL     |
| event\_attributes\_locale\_id             | VARCHAR   | NULL     |
| event\_attributes\_locale\_name           | VARCHAR   | NULL     |
| event\_attributes\_url                    | VARCHAR   | NULL     |
| event\_attributes\_timestamp              | TIMESTAMP | NULL     |
| event\_attributes\_first\_session         | BOOLEAN   | NULL     |
| event\_attributes\_logged\_in\_status     | VARCHAR   | NULL     |
| event\_attributes                         | VARIANT   | NULL     |
| user\_attributes                          | VARIANT   | NULL     |
| device\_attributes                        | VARIANT   | NULL     |

The schema is the same for both delivery methods. Custom column naming is not supported.

# Frequently Asked Questions

<Accordion title="How soon will I start getting the data?">
  It typically takes up to 30 minutes for the automated data exports to be set up for your account and for you to start seeing data there.
</Accordion>

<Accordion title="Can I export additional events at a later time?">
  Yes, if you need to export new events at a later time, you can always edit your configuration on the App Marketplace and get the new events added to your exports.
</Accordion>

<Accordion title="Can I export historical data using Automated Data Exports?">
  As of now, you cannot export historical data using Automated Data Exports. Once configured, you will start seeing data for each event from the time you enabled your exports. This applies to both direct exports and OpenSharing.
</Accordion>

<Accordion title="Is there any pricing involved for Automated Data Exports?">
  If your billing plan does not cover this already and you need to get this enabled at a later time, you would need to work with your Customer Success Manager to get this included in your billing plan. Based on the amount of data that MoEngage will place in your Data warehouse/Partner/API on a monthly basis, this could lead to additional charges.
</Accordion>

<Accordion title="Does Data Exports guarantee the latest user attributes?">
  No, Data Exports is primarily built to export your Events in near real-time and user attributes in MoEngage are updated asynchronously so it is currently not possible to guarantee the latest values of user attributes in the exports.
</Accordion>

<Accordion title="Can I move from a direct export to OpenSharing?">
  Yes. Contact your MoEngage CSM to change delivery methods. The two methods use the same schema but separate storage layers, so records already delivered through your direct export do not appear in the OpenSharing catalog.
</Accordion>

<Accordion title="Can I rename the columns or tables in the OpenSharing catalog?">
  No. The schema is standardized and custom naming is not supported. You choose which events and attributes to export during configuration, as described in [Step 3: Configure Your Export](/user-guide/data/exports/events/data-warehouse/databricks-exports#set-up-your-export).
</Accordion>

<Accordion title="Who do I contact after the OpenSharing connection is live?">
  Your MoEngage CSM is the primary contact for maintaining the connection, resolving technical questions, and adjusting your data requirements.
</Accordion>