ETL Configurations – ZoneReporting Knowledge Base

Below are details to help users better understand and manage all aspects of the Zone Data Platform ETL (Extract, Transform, Load) pipeline to ensure data is accurate, timely, and aligned with your business needs. This includes:

Configure and monitor alerting and notifications to proactively identify and respond to ETL issues or delays.
Manage file processing workflows to control how data is ingested, staged, and transformed from source systems.
Oversee ETL job execution and performance, ensuring tasks run as expected and system resources are optimized.

Workflow Category	Workflow	Workflow Description	Configuration Editor Section	Field	Field Description	Sample Values
System Configuration	General Configuration	Stores global settings using key-value pairs. Used to configure core system behavior.	Configuration	Customer: Name	The name of the customer or client organization.
System Configuration	General Configuration	Stores global settings using key-value pairs. Used to configure core system behavior.	Configuration	Customer: TimeZone	The time zone in which the customer operates; used for scheduling, logging, and data consistency.
System Configuration	General Configuration	Stores global settings using key-value pairs. Used to configure core system behavior.	Configuration	Database: Language	The default language setting for database operations. This includes Date formats, System messages, Names of days and months, first day of the week. To see a list you can execute: SELECT * FROM sys.syslanguages;	British, English, Spanish, German
System Configuration	General Configuration	Stores global settings using key-value pairs. Used to configure core system behavior.	Configuration	Email: Cc	Email address(es) to be used for system-generated emails. If using multiple addresses, separate them with a semicolon (;) — for example: testemail1@zoneandco.com;testemail2@zoneandco.com.	testemail1@zoneandco.com;testemail2@zoneandco.com
System Configuration	General Configuration	Stores global settings using key-value pairs. Used to configure core system behavior.	Configuration	Email: To	Email address(es) to be used for system-generated emails. If using multiple addresses, separate them with a semicolon (;) — for example: testemail1@zoneandco.com;testemail2@zoneandco.com.	testemail1@zoneandco.com;testemail2@zoneandco.com
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Configuration	Flows: 0:RunAtStartup	Specifies whether this flow (index 0) should run automatically when the system starts. In Flows screen the first flow (index 0) is usually the Load DW flow
ETL Process & File Handling	ETL Process Flow	Defines the ETL pipeline, a sequence of FlowTasks that execute data processing logic.	Configuration	ParameterSearch: BlobStorageFolder	Blob storage folder to search for Parameters.
ETL Process & File Handling	ETL Process Flow	Defines the ETL pipeline, a sequence of FlowTasks that execute data processing logic.	Configuration	ParameterSearch: FileName	File name that has parameters. This is a special file used by TacticalConnect to help define the criteria for every saved search that's been configured. Unlike regular files that are loaded directly into the Data Warehouse (DW), the Parameter Search file is handled a bit differently: It’s stored in a separate folder in Blob Storage defined in this parameter: ParameterSearch: BlobStorageFolder This file plays a key role in a special process that fills in missing values for the FilePart column. Why is this important? There’s a rare but important edge case: sometimes, records that have been deleted in NetSuite don’t get properly removed from the Data Warehouse. This file and its process help catch and correct those issues — making sure your data stays accurate and reliable over time.
BI & Analytics Integration	Power BI Model Refresh	Refreshes Power BI datasets and models as part of a reporting pipeline.	Configuration	PowerBI:Dataset-Id	The unique identifier for the Power BI dataset used for reporting.
BI & Analytics Integration	Power BI Model Refresh	Refreshes Power BI datasets and models as part of a reporting pipeline.	Configuration	PowerBI:Workspace-Id	The identifier of the Power BI workspace where the dataset resides.
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionHeader_History:HistoryTableName	Table that stores historical records for transaction headers.	[dbo].[vw_TransactionHeader_History]
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionHeader_History:LastUpdatedColumn	Column used to track the last update to related transactions.	[DW_DateModified]
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionHeader_History:RangeColumn	Specifies the column used to track what data has changed and when, for incremental data loading. Common Value: IR_Date (Item Receipt Date) Why IR_Date? Because it's a reliable, sequential timestamp for new or updated transaction data related to inventory movement.	key: TransactionHeader_History:RangeColumn value: [IR_Date]
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionHeader_History:StoreInLastUnit	Defines the time granularity or windowing unit used to track the last extracted data point. Helps group and manage historical data cleanly for business reporting periods. Typical Value: "Quarters" (but could also be "Months", "Days", etc.). If set to Quarters The system stores and compares the last extracted point at the quarter level. For example, if the latest data falls in Q1 2025, that’s the value it retains.	Quarters
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionHeader_History:StoreInLastValue	Captures the last known value of the RangeColumn based on the selected StoreInLastUnit. These are internal values that increment per quarter and help track "last seen" data for incremental logic. How it works: On the next load, the pipeline will only pull records where IR_Date falls in a quarter greater than 100.	80, 100
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionHeader_History:TableName	Main table for current transaction line items.	[dbo].[vw_TransactionHeader_History]
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionLines_History:HistoryTableName	Historical table for transaction line items.	[dbo].[vw_TransactionLines_History]
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionLines_History:LastUpdatedColumn	Column used to track the last update timestamp for lines.	[DW_DateModified]
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionLines_History:RangeColumn	Specifies the column used to track what data has changed and when, for incremental data loading. Common Value: IR_Date (Item Receipt Date) Why IR_Date? Because it's a reliable, sequential timestamp for new or updated transaction data related to inventory movement.	[IR_Date]
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionLines_History:StoreInLastUnit	Defines the time granularity or windowing unit used to track the last extracted data point. Helps group and manage historical data cleanly for business reporting periods. Typical Value: "Quarters" (but could also be "Months", "Days", etc.). If set to Quarters The system stores and compares the last extracted point at the quarter level. For example, if the latest data falls in Q1 2025, that’s the value it retains.	Quarters
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionLines_History:StoreInLastValue	Captures the last known value of the RangeColumn based on the selected StoreInLastUnit. These are internal values that increment per quarter and help track "last seen" data for incremental logic. How it works: On the next load, the pipeline will only pull records where IR_Date falls in a quarter greater than 100.	80, 100
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionLines_History:TableName	Main table for current transaction line items.	[dbo].[vw_TransactionLines]
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionRelated_History:HistoryTableName	Historical table for transaction line items.	[dbo].[vw_TransactionRelated_History]
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionRelated_History:LastUpdatedColumn	Column used to track the last update timestamp for lines.	[DW_DateModified]
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionRelated_History:RangeColumn	Specifies the column used to track what data has changed and when, for incremental data loading. Common Value: IR_Date (Item Receipt Date) Why IR_Date? Because it's a reliable, sequential timestamp for new or updated transaction data related to inventory movement.	[IR_Date]
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionRelated_History:StoreInLastUnit	Defines the time granularity or windowing unit used to track the last extracted data point. Helps group and manage historical data cleanly for business reporting periods. Typical Value: "Quarters" (but could also be "Months", "Days", etc.). If set to Quarters The system stores and compares the last extracted point at the quarter level. For example, if the latest data falls in Q1 2025, that’s the value it retains.	Quarters
ETL Process & File Handling	Update Processing Logic	Determines which rows get updated, inserted, or ignored.	Configuration	TransactionRelated_History:StoreInLastValue	Captures the last known value of the RangeColumn based on the selected StoreInLastUnit. These are internal values that increment per quarter and help track "last seen" data for incremental logic. How it works: On the next load, the pipeline will only pull records where IR_Date falls in a quarter greater than 100.	80, 100
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTask	Replicate to Customer Database	Process made up of 1 to many DBTaskSteps	Replicate to Customer Database
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTaskSchedule	Replicate to Customer Database - Every 30 minutes	Enables recurring data replication every 30 minutes to the customer’s database. This will map to the DBTask by the DBTaskId
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTaskSteps	dbTaskId	The identifier linking this step to the DBTask. Multiple steps can belong to the same task.
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTaskSteps	dbTaskStepOrder	The sequence in which this step should run within the overall task. Lower numbers run first.
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTaskSteps	dbTaskStepTypeId	Indicates the type of task step (e.g., stored procedure or external code). See DBTaskStepType below for values.
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTaskSteps	isEnabled	A flag to determine if the step is active (true) or temporarily disabled (false). Disabled steps are skipped during execution.
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTaskSteps	dbTaskStepName	A human-readable name for the task step, useful for identifying it in logs or interfaces.	Copy Database - Phase 1
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTaskSteps	dbTaskStepCommand	The command that the step executes — such as the name of a stored procedure or an external script reference.	exec Zone.usp_UpdateCopyDBStep @SourceDBServer = ''sqllTest'', @SourceDBName = ''NetSuite'', @TargetDBServer = ''sqllTest-repl'', @TargetDBName = ''NetSuite_TEST'', @MaxSeconds = 1200, @DBTaskRunID = @DBTaskRunID, @DBTaskStepOrder = @DBTaskStepOrder, @Edition = ''Standard'', @ServiceObjective = ''S0''
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTaskSteps	dbTaskStepServer	The server or environment where the task step should be executed (e.g., prod-sql01).	sqltestservername-repl
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTaskSteps	sqlLoginUsername	The SQL login used to authenticate and run the task step, if required.
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTaskSteps	sqlLoginKeyVaultSecretName	The name of the Key Vault secret that stores the password or token for secure login — used instead of storing credentials in plain text.
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTaskSteps	dbTaskStepDatabase	The target database where the task step is executed	ZoneDW
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTaskStepType	SP	Stored Procedure. Executes a stored procedure directly within the database. This is typically used for data transformations, staging, or loading.
Replication	Replicated SQL - Task Scheduling	Defines when ETL replicated SQL flows should execute.	DBTaskStepType	NonDBCode	Code managed outside the database. Refers to a step that runs external code (e.g., a Python or PowerShell script), often used for orchestration, API calls, or non-SQL-based data manipulation.
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	id	The identifier for the file configuration.
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	filenamePattern	This setting defines the pattern used to identify files that should be processed with this configuration. The pattern supports wildcards and special delimiters to help you match filenames precisely. Supported Symbols % — Wildcard that matches any number of characters, including none. ! — Escape character used to indicate that the following symbol (usually _) should be treated literally, not as a wildcard. Pattern Rules To ensure reliable file matching, follow these conventions: Always start with % This accounts for dynamic parts of the filename like timestamps that may appear at the beginning. Escape all underscores (_) using ! In the pattern system, _ is treated as a single-character wildcard by default. To match an actual underscore in the filename, prefix it with ! (e.g., !_ matches _ literally). Always end with % (before the file extension) This ensures that any additional characters in the filename (such as version tags or system-generated suffixes) are accounted for. Example Pattern: %!_customsearch!_zr!_full!_sub!_items%.csv Matches filename: 20250429.22.15.02.378_customsearch_zr_full_sub_items.csv This pattern: Ignores the timestamp at the beginning Precisely matches the string _customsearch_zr_full_sub_items Ignores anything extra after that up to the .csv extension	%!_customsearch!_%!_full!_coa%.csv
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	stageTableName	The schema is always stg, and the table name should match the production table.	[stg].[COA]
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	productionTableName	The destination table where data is loaded after validation and transformation. The schema is dbo, and the table name matches the actual dataset.	[dbo].[COA]
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	fileType	Delete – Removes records from the Data Warehouse. Used when records have been deleted in the source system. Delta – Captures only new, changed, or deleted records from the last two days. Minimizes processing by only syncing what's changed. Full – Loads the entire dataset without partitioning. Ideal for initial loads or recovery when deltas have been missed. Partial – Loads only a portion of the dataset based on a partition field (commonly a date). Used to manage large files efficiently. Snapshot – Captures the dataset as it existed at a point in time. Useful for auditing or historical comparisons.	Delete Delta Full Partial Snapshot
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	fieldTerminator	The character used to separate fields in each row of the file. If left blank we default to using a comma.	, for CSVs or \t for tab-delimited files.
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	rowTerminator	The character(s) that indicate the end of a row.	0x0a (LF) or 0x0d0a (CRLF)
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	uniqueColumnName	The column (or set of columns) used to uniquely identify each record. This is used for deduplication and merging logic. Can also have multiple fields to determine uniqueness separated by a pipe [Field1]\|[Field2]	[Transaction Internal ID] or: [Field1]\|[Field2]
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	formatFileName	The name of the format file (if using one), typically for parsing fixed-width or complex delimited files.	DelimitedFormat.xml and within this file it will have the format specs.
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	makeSourceDistinct	Boolean flag (true/false) that enforces deduplication at the source level before loading to staging.
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	useColumnNameMatching	Boolean flag indicating whether to map columns from the file to the staging table based on column names rather than column order. The items, inventory_levels, transaction_lines, transaction_related, projects, demand_plan tables use this function. transaction_lines uses the function. Without the function then the column name from the file must exactly match the column name from the table. Using a function lets you make changes between the two. For example, the transaction_lines file outputs a field named * that we map to a column in the table named Header.
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	columnNameTransformFunction	A function or rule applied to incoming column names (e.g., to standardize casing or replace spaces with underscores) before mapping them to database columns. Only used if UseColumnNameMatching is enabled (is optional). [stg].[udf_TransactionLinesHeader] where the function is defined in sql to do the mapping.
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	filenameAntiPattern	A pattern used to exclude files from being processed, even if they match the filenamePattern.	%!_temp.csv
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	doSoftDelete	Boolean flag that, if enabled, marks records as deleted in the production table instead of physically removing them. If enabled, the table must have a Primary Key
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	isRfc4180Compliant	Boolean flag that indicates whether the file adheres to RFC 4180 CSV formatting rules (e.g., properly escaped quotes, consistent delimiters). More reference here: https://www.ietf.org/rfc/rfc4180.txt The default state for a new FileConfiguration item should be TRUE When the field is set to FALSE, then the FieldTerminator field becomes mandatory
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	lastRow	Used to define a condition or pattern that identifies the final data row in the file, useful for ignoring trailing footers or summary lines. The input is a negative number so -1 will not include the last row and -10 will not include the last 10 rows.	-1
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FileConfiguration	fieldQuote	The character used to enclose fields, typically for strings containing delimiters.	" or '
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Flows	id	Unique identifier for the flow.
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Flows	name	Descriptive name of the flow.	Weekly Maintenance, Load DW
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Flows	schedule	Defines when the flow should run. This is in CRON format. You can use various online resources to translate this into human readable time until this is embedded in the product, https://crontab.guru/	0 10 * * *
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	FlowTasks	id	Unique identifier for the task.
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	FlowTasks	name	Task name used for identification in logs or UI.
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	FlowTasks	flowId	Links this task to the parent Flow.
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	FlowTasks	type	The type of task being executed	ArchiveFiles ProcessFiles ProcessModel SetDatabase ProcessJobs Wait RunAdfPipeline Delay LogMessage
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	FlowTasks	ordinal	Order of execution within the flow.
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	FlowTasks	continueOnError	Whether the flow should continue if this task fails.
ETL Process & File Handling	Archive Files	Moves or archives processed files from staging to archive locations.	FlowTasks.type.ArchiveFiles	Stage	Folder where active files are located.	stage-for-sql
ETL Process & File Handling	Archive Files	Moves or archives processed files from staging to archive locations.	FlowTasks.type.ArchiveFiles	Archive	Target folder for archived files.	archive
ETL Process & File Handling	Archive Files	Moves or archives processed files from staging to archive locations.	FlowTasks.type.ArchiveFiles	Error	Folder for failed or errored files.	archive-error
ETL Process & File Handling	Copy Files	Copies source files to designated destinations for processing.	FlowTasks.type.CopyFiles	SourceContainer	The source container (e.g., blob storage folder) to read files from.	zone-data-in
ETL Process & File Handling	Copy Files	Copies source files to designated destinations for processing.	FlowTasks.type.CopyFiles	DestinationContainer	The destination to write files to.	latest-files
ETL Process & File Handling	Copy Files	Copies source files to designated destinations for processing.	FlowTasks.type.CopyFiles	Files	File pattern or specific files this task should handle. The default is to use customsearch.	customsearch, bi, plans , mgmt , ics, salesforce
ETL Process & File Handling	Copy Files	Copies source files to designated destinations for processing.	FlowTasks.type.CopyFiles	MoveFiles	Whether to delete from source.
ETL Process & File Handling	Copy Files	Copies source files to designated destinations for processing.	FlowTasks.type.CopyFiles	Throttle	Performance tuning options.
ETL Process & File Handling	Copy Files	Copies source files to designated destinations for processing.	FlowTasks.type.CopyFiles	SourceConnectionKey	Secure connection references.
ETL Process & File Handling	Copy Files	Copies source files to designated destinations for processing.	FlowTasks.type.CopyFiles	DestinationConnectionKey	Secure connection references.
ETL Process & File Handling	Copy Files	Copies source files to designated destinations for processing.	FlowTasks.type.CopyFiles	DirectCopy	Performance tuning options.
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FlowTasks.type.ProcessFiles	Data	Primary file data input.
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FlowTasks.type.ProcessFiles	Stage	Staging folder used during processing.
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FlowTasks.type.ProcessFiles	Custom	Reference to any custom logic or transformation rules.	custom-searches
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FlowTasks.type.ProcessFiles	Files	File pattern or specific files this task should handle. The default is to use customsearch.	customsearch, bi, plans , mgmt , ics, salesforce
ETL Process & File Handling	Process Files	Processes raw files and prepares them for structured loading.	FlowTasks.type.ProcessFiles	TimestampFormat	Format used to parse time-based fields in the file.
BI & Analytics Integration	Power BI Processing	Performs Power BI-related tasks such as publishing or scheduling refreshes.	FlowTasks.type.ProcessModel	WorkspaceId	Power BI workspace identifier.
BI & Analytics Integration	Power BI Processing	Performs Power BI-related tasks such as publishing or scheduling refreshes.	FlowTasks.type.ProcessModel	DatasetId	The Power BI dataset to be refreshed.
BI & Analytics Integration	Power BI Processing	Performs Power BI-related tasks such as publishing or scheduling refreshes.	FlowTasks.type.ProcessModel	NotifyAfter	When to notify (e.g., after success/failure).
BI & Analytics Integration	Power BI Processing	Performs Power BI-related tasks such as publishing or scheduling refreshes.	FlowTasks.type.ProcessModel	Type	Type of model operation.
BI & Analytics Integration	Power BI Processing	Performs Power BI-related tasks such as publishing or scheduling refreshes.	FlowTasks.type.ProcessModel	MaxParallelism	Execution behavior tuning.
BI & Analytics Integration	Power BI Processing	Performs Power BI-related tasks such as publishing or scheduling refreshes.	FlowTasks.type.ProcessModel	RetryCount	Execution behavior tuning.
Cloud & Infrastructure Tuning	Cloud Database Optimizations	Database Service Tier	FlowTasks.type.setDatabaseTier	ServiceTier	Desired tier.
Cloud & Infrastructure Tuning	Cloud Database Optimizations	Database Service Tier	FlowTasks.type.setDatabaseTier	ComputeSize	Compute resources to allocate.
ETL Process & File Handling	ETL Process Flow	Defines the ETL pipeline, a sequence of FlowTasks that execute data processing logic.	FlowTasks.type.ProcessJobs	Timeout	Max time to allow job execution before it is canceled.
BI & Analytics Integration	Power BI Processing	Power BI Model Refresh	FlowTasks.type.WaitForModelRefresh	WorkspaceId	Power BI workspace identifier.
BI & Analytics Integration	Power BI Processing	Power BI Model Refresh	FlowTasks.type.WaitForModelRefresh	DatasetId	The Power BI dataset to be refreshed.
ETL Process & File Handling	ETL Process	Triggers a pipeline in Azure Data Factory.	FlowTasks.type.RunAdfPipeline	DataFactoryResourceId	Reference to the ADF instance.
ETL Process & File Handling	ETL Process	Triggers a pipeline in Azure Data Factory.	FlowTasks.type.RunAdfPipeline	Pipeline	The pipeline name to run.
ETL Process & File Handling	ETL Process	Pauses the flow for a defined period of time.	FlowTasks.type.Delay	Timespan	How long to wait (e.g., 00:05:00 for 5 minutes).
ETL Process & File Handling	ETL Process Flow	Defines the ETL pipeline, a sequence of FlowTasks that execute data processing logic.	FlowTasks.type.LogMessage	Message	Text to log.
ETL Process & File Handling	ETL Process Flow	Defines the ETL pipeline, a sequence of FlowTasks that execute data processing logic.	FlowTasks.type.LogMessage	Level	Log level (e.g., Info, Warning, Error).
Data Source Mapping	Source-to-Table Mapping	Maps a source record type (e.g., from NetSuite or another system) to its corresponding table name in the Data Warehouse.	RecordTypeTableMap	id	Unique identifier for the mapping record.
Data Source Mapping	Source-to-Table Mapping	Maps a source record type (e.g., from NetSuite or another system) to its corresponding table name in the Data Warehouse.	RecordTypeTableMap	recordType	The name of the source record type.	Case, Project, Time
Data Source Mapping	Source-to-Table Mapping	Maps a source record type (e.g., from NetSuite or another system) to its corresponding table name in the Data Warehouse.	RecordTypeTableMap	tableName	The target table in the Data Warehouse where this record type’s data is stored.	case, projects, time
Data Source Mapping	Source-to-Table Mapping	Maps a source record type (e.g., from NetSuite or another system) to its corresponding table name in the Data Warehouse.	RecordTypeTableMap	configurationCode	Optional code used to identify Client or Framework mapping	Client
Replication	Replicated SQL	Defines which source objects (tables/views) are actively replicated from a source system into the Data Warehouse.	ReplicatedObjects	id	Unique identifier for the replication definition.
Replication	Replicated SQL	Defines which source objects (tables/views) are actively replicated from a source system into the Data Warehouse.	ReplicatedObjects	schemaName	The schema of the source object	dbo
Replication	Replicated SQL	Defines which source objects (tables/views) are actively replicated from a source system into the Data Warehouse.	ReplicatedObjects	objectName	Name of the object to be replicated.	dbo.vw_COA
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Schedules	id	Unique identifier for the schedule.
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Schedules	scheduleName	A human-readable name for the schedule.	Hourly, Daily Once @ 0800, Every 7 days Once @ 0100
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Schedules	enabled	Boolean flag indicating if the schedule is active.
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Schedules	freqTypeId	Foreign key to ScheduleFreqType to define the recurrence pattern	4 which maps to Daily
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Schedules	frequencyInterval	Main interval between executions	1
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Schedules	freqSubdayTypeId	Foreign key to ScheduleFreqSubdayType to define sub-day execution	Seconds, Minutes, Hours
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Schedules	frequencySubdayInterval	Frequency for the subday type.	1
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Schedules	frequencyRecurrenceFactor	Used for more complex patterns.
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Schedules	ActiveStartDate	Date the schedule starts running.
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Schedules	ActiveEndDate	Date the schedule stops running.
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Schedules	ActiveStartTime	Time of day the schedule can start.
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	Schedules	ActiveEndTime	Time of day the schedule should stop.
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	ScheduleFreqSubdayType	id	Unique identifier. Reference table for types of sub-day frequency intervals (e.g., hourly, every minute).
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	ScheduleFreqSubdayType	freqSubdayTypeName	Short name for the subday type. Reference table for types of sub-day frequency intervals (e.g., hourly, every minute).	Seconds, Minutes, Hours
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	ScheduleFreqSubdayType	freqSubdayTypeDesc	More detailed description of the subday type. Reference table for types of sub-day frequency intervals (e.g., hourly, every minute).
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	ScheduleFreqType	id	Unique identifier. Reference table for high-level frequency types (e.g., daily, weekly, monthly).
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	ScheduleFreqType	freqTypeName	Name of the frequency type (e.g., Daily, Weekly). Reference table for high-level frequency types (e.g., daily, weekly, monthly).	Daily
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	ScheduleFreqType	freqTypeDesc	Detailed explanation of what this type means. Reference table for high-level frequency types (e.g., daily, weekly, monthly).	Daily
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	UpdateOrder	id	Unique identifier for the entry. Specifies the order in which stored procedures should run, often used for ordered data refresh or processing tasks.
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	UpdateOrder	order	Execution order value — lower numbers run first. Specifies the order in which stored procedures should run, often used for ordered data refresh or processing tasks.	100001, 100002, 100003
ETL Process & File Handling	Task Scheduling	Defines when ETL flows should execute.	UpdateOrder	procedureName	The stored procedure name to execute in the given order. Specifies the order in which stored procedures should run, often used for ordered data refresh or processing tasks.	[ZoneDW].[usp_TransactionLines_History] [ZoneDW].[usp_TransactionRelated_History] [ZoneDW].[usp_TransactionHeader_History]
Data Source Mapping	Transaction Type Abbreviations	Stores abbreviations for transaction types used in URL analytics.	URLAbbreviations	id	Unique identifier.
Data Source Mapping	Transaction Type Abbreviations	Stores abbreviations for transaction types used in URL analytics.	URLAbbreviations	transactionType	The name of the transaction type.	Opportunity, Sales Order
Data Source Mapping	Transaction Type Abbreviations	Stores abbreviations for transaction types used in URL analytics.	URLAbbreviations	abbreviation	The shortened or abbreviated name used in URLs.	opprtntylist, salesord