Opening our ETLService.config file, we can see the structure of how, what and when the data is imported.
The main config file is broken down into a few key areas. Today we’ll focus on the schedules, and with a third party tool we can minimize the areas we don’t need to focus on, such as the:
- Config Sections
- File System
In the DatabaseFiles tag, we have a subsection for the tables we’re creating.
These tables detail how they capture the data (See the Source tags), the Schedule, and the Fields they create (the columns in the table)
We can ignore the first and last sections, so let’s focus on the schedule.
The Schedule Type of Explicit remains the same, with the other value being None, however we can alter the days the schedule runs on by nominating the day tag in this field.
The field highlighted shows the seven days of the week. You can trim this to suit your requirements, but for the schedule to run on weekdays, you can remove the values of “sa” and “su”.
We can set a time for this table to update by changing the 24 hour time for the start attribute. Typically, we load data very early in the morning so it doesn’t affect the load on the target server, and typically after the maintenance cycles on the server.
The additional configuration we can perform includes changing the table to run multiple times per day. You can still set a start time, but it’s possible to add an end time for the ETL to stop, and an interval attribute in minutes for it to repeat the ETL update. The below example starts at 1 am, runs every 30 minutes, and stops at 4 am.
When you’ve made your change to that table, you will need to repeat this task for any other table in the ETLService.config file. Meaning, if every table needs to run one hour later than normal, you need to move every schedule for every table forward by an hour.