Skip to main content

Data Import

Importing CSV data files into DataX

Updated this week

The import feature provides you with the capabilities to upload CSV data files from systems that are not integrated with your DataX instance.

You can access the Import feature by selecting Lakehouse in the navigation menu.

In order to access the Import feature, users must have the permission Import Management enabled.

Importing Data Files

Once on the Import page, select the "+ New Import" button, in the upper right hand corner, you will be redirected to the New Entity Import page where you will be able to upload your data files.

Step 1: Upload Files

  1. On the New Entity Import page,

    1. Enter a name for the Import (required)

    2. Description of the Import (optional, but recommended)

    3. Tags for the Import (optional, but recommended)

  2. Select the Advance Options and make any necessary adjustments regarding the formatting of the data in your file.

    • descriptions are available within the tooltips next to each of the Advanced Settings options.

  3. In the File Upload section, drag & drop or browse for the files that will be uploaded

  4. Click the Upload Files button

The files will begin to upload. Once the files are finished uploading, they will automatically be scanned to identify column data types and identify any potential issues within the file.

Note:

  • Depending on the size of your file, the time for the uploading and scanning process will vary. While your file uploads, you do not need to keep the page opened.

  • If an error was identify, the Import Entity will have a status of Error with Latest Scan. Review the error and make any necessary adjustments to your file.

  • If you are unable to successfully address the errors, please reach out to Support.

Step 2: Review Columns

Once your file is finished scanning and there are no errors, your Imported Entity will have the status of Ready to Deploy.

Select the Entity from the list of Imports, to Review the Columns that were identified in the file. Make any adjustments as needed to the

  • Column Name

  • Descriptions

  • And any constraints

    • Nullable, if the column is allowed to be null

    • PII, Personally Identifiable Information

    • Primary Key, uniquely identifies each row

Select the Update Import button, to save any adjustments made.

Then select Step 3: Sample Rows.

Step 3: Sample Rows

  1. After selecting Sample Rows, sample data will generate.

  2. Review the Sample Data, if needed make any adjustments to the Columns in Step 2.

  3. Once ready, click the Deploy button.

The process of deploying an entity will begin. Once the entity is finished being deployed it will be available in the Catalog, within the Import folder.

Note:

  • Depending on the size of the entity, the amount of time to finish deploying will vary. While your entity is in the process of deploying, you do not need to stay on the page.

Editing an Imported Entity

Once an Entity has been imported in, you will only be able to edit the

  • Description of the entity

  • Tags on the entity

  • Description of the columns

To make any edits to the imported entity

  1. Navigate to the Catalog

  2. select the imported Entity

  3. then select Manage >Edit

Make the necessary adjustments needed, then click Publish & Deploy.

Note

  • Since the imported data is meant to be a one time data import, you are not able to import additional data into the entity once it has been deployed. To add additional data, you will need to create a new imported entity.

Did this answer your question?