Data Import and Export

Momen database provides efficient data import and export capabilities, supporting bulk data import from external sources and export of database data to standard format files for data migration and backup.

🚀

Supports importing up to approximately 10 million rows of data, with an import speed of up to 100,000 rows per minute.

Data Import Process

Prepare Data Files

📥Sample files for this article

Table file requirements:

UTF-8 encoded Excel files (.xlsx only) or CSV files
First row must be headers
Empty cells should not contain data or spaces
Must contain at least one data record

Template tables can be downloaded from the bottom left of the upload interface.

excel_sample

For importing images, videos, and files, prepare an additional zip package:

Create image, video, and file folders locally to store images, videos, and other files respectively
Compress these three folders directly into a zip file (do not place them in another folder before compression)
In the CSV or Excel file, multimedia column values should be written as “folder_name/filename”, e.g., image/cover0.png, file/api.pdf, video/intro.mp4

database

Upload Files

Click the “Import” button in the bottom left corner of Momen database to enter the data upload interface. Upload the prepared tables and multimedia files (optional). Multiple tables can be uploaded (supports mixed CSV and Excel), but only one multimedia zip file is allowed.

entry

upload_file

Select Import Type

Choose “New” or “Update”. “New” means directly inserting data into the database, while “Update” updates existing records based on primary key (id).

Configure Mapping

Mapping consists of:

Table Mapping: Maps uploaded tables to database tables. System tables (account tables, payment tables, etc.) do not support import. Each sheet in Excel and each CSV file corresponds to one table.
Field Mapping: Maps columns from uploaded tables to database table fields. Columns that don’t need to be imported can be ignored using the “Ignore” option on the right side of each row.

mapping

Related Field Mapping

If other tables are related to the current table, related table mapping needs to be configured. The principle is: after establishing a relationship, the current table stores the ID of the related table. Therefore, during import, a column in the table needs to be matched with a field in the related table (such as username, email, etc.) to find the corresponding unique ID.

For example, if there’s a one-to-many relationship between the “account” and “post”, the “author” column in the “post” stores the ID from the “account”. During import, you can use the author name from the Excel table to find the matching “username” in the “account” to get the corresponding ID.

To avoid ID lookup failure due to duplicate related fields (if there are multiple users named “Arche” in the Account table, it won’t know which ID to use), ensure that the “Username” in the “Account Table” cannot be duplicated by configuring a unique constraint.

unique_constraint

After configuring the unique constraint, when mapping related tables, map the “Author” column from the Excel table to the “Username” field in the Account table. If the constraint includes multiple fields, configure matching for multiple fields.

relation_mapping

Multimedia Field Mapping

To import multimedia files (images, videos, and files), the corresponding column values in the import table should be written as “folder_name/filename”, such as “image/cover0.png”, “video/intro.mp4”, “file/api.pdf”.

During import, the system will automatically extract these files from the uploaded zip package and store them in the database.

Geographic Coordinate Field Mapping

To import coordinates, first select the coordinate type (WGS84, GCJ-02, or BD-09), then configure latitude and longitude mapping separately. Therefore, the import table needs two columns to store latitude and longitude.

geo_mapping

Data Conflict Handling

When the import mode is “New” and data fields have unique constraints, data conflicts occur if duplicate values appear in the imported data. To handle such situations, the system provides conflict data handling methods:

First, select the constraint:

Any Unique Constraint: Triggers conflict handling if any field with a unique constraint conflicts.
Specific Unique Constraint: Select a specific unique constraint as the basis for conflict judgment.

The system provides three conflict handling methods:

Skip: Ignore conflicting data and continue importing non-conflicting data.
Update: Use the new imported data to overwrite existing conflicting data in the database.
Abort: Immediately stop the entire import process and record detailed error information. This option is only available when “Any” is selected.

constraint

Trigger Configuration

If the data table has configured triggers (such as new, update, delete), you can choose whether to trigger them during import. Import triggers use a whitelist mechanism. During import:

New triggers added are not added to the whitelist and won’t be triggered
Modified triggers still in the whitelist will trigger the modified version
Deleted triggers are removed from the whitelist and will no longer be triggered

Preview Data

After configuration, you can preview the import data. The system displays the first 10 rows of the table for users to confirm data format and content. Multimedia file previews only show filenames, not specific content.

preview

Execute Import and View Results

After confirmation, click the “Import” button to start the import process. The import process will occupy some database space, which will be automatically released upon completion.

After import, the system will indicate success or failure. Detailed import results, including successful entries, can be viewed in “History”. If some data was skipped or failed, the system provides a downloadable log file (retained for 7 days) with detailed records of skipped data and error reasons.

To ensure data integrity and consistency, if any data fails to import, none of the data from this import will be written to the database.

import_result

log

Data Export Process

Configure Export Options

Export Format: CSV or Excel (xlsx)

When exporting both the current table and related tables, CSV format will generate multiple independent CSV files, while Excel format will generate a single file containing multiple workbooks.

Export Multimedia Files:

When enabled, the system will package all related multimedia files (images, videos, files) into a ZIP archive for export. The archive contains IMAGE, VIDEO, and FILE folders for storing corresponding media content.

Select Data Range

Select Data Table: Supports all system tables, custom tables, and custom views
Configure Filter Conditions: Set filtering, sorting, and deduplication conditions to precisely filter export data
Select Export Fields: Freely choose fields from current and related tables

exporting

Execute Export and View Results

After configuration, click the “Export” button to start the export. Export files can be viewed and downloaded in the “History” records, with files retained for 7 days.

Import Error Handling

Import logs provide detailed error information, including filename, worksheet name, specific row numbers, and error descriptions, facilitating precise location and troubleshooting. Log line numbers correspond to actual table line numbers, with table data starting from row 2 (row 1 being the header), and logs also reporting from row 2.

Error Message	Cause	Solution
NO_MATCHING_MEDIA	Corresponding multimedia file not found, possibly due to non-standard directory structure or incorrect file-data matching.	1. Check if multimedia file directory structure is standard; 2. Verify if filenames in table are standard; 3. Check if table filenames match actual files.
unique_constraint_violation	Import data conflicts with unique constraints.	1. Check and remove duplicate data; 2. Choose to skip this unique constraint in import configuration.
MISMATCHED_DATA_TYPE	Data type mismatch.	Please modify table data to match the specified data type.
INTERNAL_ERROR	Various possible causes, such as non-UTF-8 encoded zip package.	Recommend using tools like Bandizip for compression.