Module 1 of Intuitive Solution Expertise
In this module we’ll cover how we get data from SAFEQ Cloud into Intuitive, including the API calls we generate, how often they’re run and the general architecture of the Intuitive for SAFEQ Cloud solution.
Let’s first take a look at the architecture diagram. We’ll keep returning to this diagram, as it’s a useful indicator of where we are in the data process.
Scanning to the bottom left, we have the customers’ SAFEQ Cloud tenancy. This sits outside of the Intuitive software, and could be located anywhere in the SAFEQ Cloud locations.
With any of the other print management solutions, we typically have some kind of physical transfer either by ODBC or through CSV files. As we access data from SAFEQ Cloud via their API, it’s a convenient method of getting that same print data, but through a series of JSON files.
Those JSON’s contain up to a week’s worth of data in a single request, but can be limited to 500 records at one time, which effectively is 500 print-related jobs. This means that if a single request for a week contains more than 500 jobs, it can be paginated, so the next request is the same weeks-worth of data, but for the next batch of 500 jobs until the request has been fulfilled.
For example, if we made a request for the week of 09/09/2024 to 16/09/2024, but there were 1234 records/jobs, that request would be paginated into three separate JSON’s. Each JSON contains a code, or a nextPageToken to reference so you can correctly locate the next page of results.
These API requests are built into the software itself, meaning it won’t be visible to anyone outside of the Intuitive development team.
On the subject of the API calls, there are quite a few requests we make to fully flesh out the dashboards. They’re performed each morning around 2am (depending of implementation) and the order of the calls are listed down below. We’re not going to go into the detail, but we’ve summarized what each call is doing:
- Register Endpoint
- Register Intuitive as a location that can make requests inside of SAFEQ Cloud, and creates a device code.
- Obtain / Refresh Device Token
- The device code is used to generate / refresh a device token which used in conjunction with the next request
- Get / Refresh User Token
- Generates / refreshes a user token from the bearer device token. The user token is used for the next set of API requests for getting the detailed SAFEQ Cloud tenancy information.
- Get Documents History
- Contains the job-level detail i.e. who printed what and where
- Get Account Containers
- Contains the unique account containers attached to that customer
- Get Output Ports
- Contains the list of devices
- Get Input Ports
- Contains the list of the print queues
- Get Locations
- Locations of the geographical group objects, such as servers, input / output ports
- Get Groups
- Contains the user group memberships
The documents’ history table / API call works incrementally. On the first ingest of data, by default we’ll request up to 730 days’ worth of data. If the customer has less than two years’ worth of data, that’s no problem, as we just request up to that amount.
Thereafter, the API calls will check the last available date in our data, and request from that date thereafter. For example, if we run for the first time on the 09/09/2024 at 14:36pm, we’ll request data as far back as the 09/09/2022. With that two years’ of data, tomorrow’s incremental data load will request data from the 09/09/2024 at 14:36pm to the 10/09/2024 at ~2am.
The other API calls, from Get Account Containers down to Get Groups work slightly differently. We only make those individual requests when we don’t already have that data in the SQLite database. The main instigator for making these requests lies in the documents history call. If a user is performing a job and it references a device, location or group that does not already exist in the table for those calls, it will request that data as part of the nightly ingests.
There may be more data in those associated calls, but we are only requesting data that is associated to those job-level activities. Additionally, it means that we build up a dictionary over time of the customers’ estate so we can accurately show the account containers, locations etc as a snapshot of how it looked at the time, and what is actually being used by the customer.
Let’s take a look at our architecture diagram again.
We’ve gone over the kinds of calls we’re making, how often we make them, and what we retrieve back. This is all collated into a SQLite database by the ETL Service Container which, as you can see in the diagram, has now moved it into the Intuitive section.
The Intuitive infrastructure is held in an Azure UK environment, with a single ‘pod’ for the SAFEQ Cloud customer.
The SQLite database of the print data is held within Azure files. It’s held with the dashboard metadata which is the database for the software itself. It contains details such as the users who are able to sign in, what roles they have, what groups they have access to, where components are located onscreen etc.
The Dashboard Service container uses both of these databases for different purposes, but ultimately to facilitate the usage of the dashboards via the User Interface.
The next modules will cover the connection, datafeeds, datasets and so on, but with the SQLite database inside of Azure files, we are able to execute queries against it directly without the need for ODBC via the Microsoft.Data.Sqlite library. ODBC being the traditional method for on-premise applications, but is no longer relevant in the cloud / Intuitive V7 applications.
This Dashboard Service container handles the authentication of the software by setting users to either use the internal Intuitive accounts, your traditional username and password but with 2FA, or alternatively, a method for using OAUTH2 for external authentication providers such as Office 365, Okta and Gmail.
Customers then access the dashboards via a publicly accessible URL. The traditional URL structure is under HTTPS, then cloud.weareintuitive.com / the integration name. For example, ACME Corp may have the below URL structure:
https://cloud.weareintuitive.com/acme