Loading data into your Intuitive for PaperCut solution V3
1 Introduction
Prior to proceeding with the following steps, you should have already installed your Intuitive Dashboards for PaperCut solution. This section will take you through the steps to load data into the dashboards for the first time.
2 Solution Architecture
The Intuitive for PaperCut MF solution takes advantage of the data integrations features of PaperCut MF which is available in all versions since V20.0. The dashboard solution is deployed onto a Windows server which can be a physical, virtual or hosted platform. The Intuitive solution may also be co-hosted with your PaperCut MF application.
The PaperCut MF Data Integrations feature produces daily data files detailing
- Print usage
- New or updated users
- New or updated printers
- New or updated accounts
- New of updated servers
The data files are processed by the Intuitive ETL application (Extract Transform Load) and prepared in a SQLite database for use by the Intuitive Dashboards application. The entire data load process is automated and refreshes the data overnight. Data is automatically collected and updated overnight and used to populate the dashboards. For existing PaperCut MF installations, the solution allows for the import of historical data to populate the dashboards.
The dashboard solution maintains a multiyear record of the print, copy, scan and fax activity in a local SQLite database. Database views are created within the dashboard application known in Intuitive as Datafeeds. Each Datafeed performs a specific piece of analysis and are used to create the Datasets which underpin the visual components and dashboards.
Users access the dashboards by connecting to the dashboard server via any network connected device with a HTML5 compliant browser (Usually a PC or tablet).
In order to access the dashboards, users first need to pass authentication. Intuitive for PaperCut ships with its own internal user authentication and permission system. Users can also be attached from a Microsoft Active Directory (‘AD’) or other LDAP compliant repository. This enables users to authenticate to the dashboards using their network username and password.
3 PaperCut Data Integrations
When the Intuitive for PaperCut solution is installed, it automatically creates a file called Intuitivebi.properties . PaperCut will look for this file and start producing data exports either:-
- When the service is re-started
- When the next maintenance window runs
- If the option for "Load Integrations" is manually selected on the data-integrations page
When first run, the data integrations can be set to export historical data as specified. You can also specify how long that data files should be kept. This setting does not affect how long the Intuitive solution keeps its data. For this reason we recommend you keep it as default.
All data files are exported as Comma Separated Value (CSV) files. The exported CSV’s (including historical data export) are exported under data.export.directory defined in the
configuration file above.
- The files are saved into a date based folder hierarchy mapping to numerical year and
numerical month (ie January = 01) with a file name format TYPE-YYYY-MM-DD.csv - environment-values.csv (contains environment variables defined in PaperCut MF
environment config keys. The same file will be overwritten every night in order to support
changes to values - The printers file contains an added column for PRINTER_GROUP which contains a
comma-delimited string of group names (if they're available) - The printer_usage_log is exported in real-time
- User, printer, accounts, servers and environment datasets are exported nightly during maintenance (ie
yesterday's data is exported)
Once the initial historical export is complete, you will find a file called '.intuitivebi-historical-export-done' in the data-integrations directory. Removing this file will cause PaperCut to reproduce the data files.
4 Intuitive ETL
In order to keep a historical copy of the PaperCut data, Intuitive uses an ETL (Extract Transform and Load) service to retrieve the data from the CSV files and aggregate the results.
The ETL service is configured to run at 02:00 a.m. On a standard PaperCut install, the maintenance routines are set to run at 00:55 a.m. Should the maintenance window on the customers PaperCut server be set to any other time than the default of 12:55am, you will need to reconfigure the ETL schedules. For more information on changing the ETL schedules, please see the following knowledge base article here.
On installation, it is possible to run the ETL manually to force the data to load immediately. In order to do this, locate and run the ETLService.Console at the following location (default).
C:\Program Files\Intuitive Business Intelligence\Intuitive Dashboards\Tools\ETL Service
Running the ETL Service Console will open a console window so that you can monitor the loading of the data files and check for any reported errors. If successful, the result should look as follows with the final entry reading 'SLEEP - Wake up at 02:00:00'
You should now close the console window and open the Services window. Start the Intuitive Dashboards ETL Service and the Intuitive Dashboards Service.
