LogoLogo
3.6.9
3.6.9
  • Introduction
  • Getting started
    • Installation
      • Prerequisites
        • Server requirements
      • On-premises
        • Single-node
          • Directory structure
        • Multi-node
      • Azure Market Place
      • AWS Market Place
    • Tutorials
      • Amazon AWS CUR
      • Amazon AWS CUR (Athena)
      • Azure Stack
      • Azure EA
      • Azure CSP
      • Google Cloud
      • VMware vCloud
      • VMware vCenter
    • How-to guides
      • How to configure receiving a monthly billing report
      • How to automatically trigger a monthly billing report
      • How to update your license
      • How to store contract information with an Account in a report
      • How to automatically send workflow errors as webhooks to a monitoring system
    • Concepts
      • User interface
      • Services
    • Releases
      • Upgrading to version 3
      • Known issues
      • Announcements
      • Archive
  • Reports
    • Accounts
    • Services
    • Instances
    • Summary
    • Budget
  • Services
    • Manage
    • Rates
      • Tiered Services
        • Aggregation Levels and the Account Hierarchy
    • Adjustments
    • Subscriptions
  • ACCOUNTS
    • Budget management
  • Data pipelines
    • Extract
      • Configuration
      • Extractor templates
      • Script basics
      • Parslets
      • Subroutines
        • check_dateformat
        • check_dateargument
        • format_date
        • validate_response
      • Language
        • aws_sign_string
        • basename
        • buffer
        • csv
        • clear
        • decimal_to_ipv4
        • discard
        • encode
        • encrypt
        • environment
        • escape
        • exit_loop
        • foreach
        • generate_jwt
        • get_last_day_of
        • gosub
        • gunzip
        • hash
        • http
        • if
        • ipv4_to_decimal
        • json
        • loglevel
        • loop
        • lowercase
        • match
        • pause
        • print
        • return
        • save
        • set
        • subroutine
        • terminate
        • unzip
        • uppercase
        • uri
        • var
    • Transform
      • Configuration
      • Transformer templates
      • Transform Preview
      • Language
        • aggregate
        • append
        • calculate
        • capitalise
        • convert
        • copy
        • correlate
        • create
        • default
        • delete
        • dequote
        • environment
        • event_to_usage
        • export
        • finish
        • Functions
        • if
        • import
        • include
        • lowercase
        • normalise
        • option
        • rename
        • replace
        • round
        • services
        • set
        • sort
        • split
        • terminate
        • timecolumns
        • timerender
        • timestamp
        • update_service
        • uppercase
        • var
        • where
    • Datasets
    • Lookups
    • Metadata
    • Reports
    • Workflows
  • Administration
    • User management
      • Users
      • Groups
    • Notifications
      • Budget Notifications
      • Report notifications
      • Workflow notifications
    • Settings
      • Global Variables
      • White Labeling
  • Advanced
    • Integrate
      • GUI automation
        • Examples
      • API docs
      • Single sign-on
        • Claims-based identity provisioning: users, Account access and user groups
        • Azure-AD
        • Auth0
        • OKTA
        • OneLogin
        • ADFS
        • LDAP
    • Digging deeper
      • Authentication flows
      • Transformer datadate
      • Dataset lifecycle
      • Config.json
      • Databases
  • Security
    • Security
    • Authentication
      • Token
      • LDAP
      • SAML2
    • Password reset
    • Password policy
    • Announcements
  • Troubleshooting
    • Logs
  • Terms & Conditions
  • Privacy Policy
Powered by GitBook
On this page
  • PostgreSQL
  • Windows Services
  • Default TCP ports
  • Default Security Settings
  • Transformer changes
  • Extractor changes
  • Extractor & Transformer Schedules
  • GUI changes
  • Changes to data processing for reports
  • API changes
  • Normalised date/time data in responses
  • Changed functionality
  • Remove deprecated functionality

Was this helpful?

Export as PDF
  1. Getting started
  2. Releases

Upgrading to version 3

Please ensure to read and understand all subjects that are mentioned here. Implement the suggested changes (where applicable) before upgrading to version 3 to avoid unexpected behavior.

PreviousReleasesNextKnown issues

Last updated 3 years ago

Was this helpful?

Upgrading to a v3.x.x will require upgrading to version v2.10.x first. The installer will verify this and displays a warning when this requirement is not satisfied.

PostgreSQL

The biggest single change is the use of a new database engine powering all application states, audit logs and processed report data. Upgrading to this new database engine is transparent and the installer will take care of installing the database server as part of the regular installation process. After or during the upgrade, it is possible to leverage an .

Windows Services

In version 2.x.x there were only two Exivity Services installed:

With Exivity version 3.0.0 up to version 3.4.3, assuming all components are installed on a single host system, there was a total of 4 different services:

As of version 3.5.0, the following Exivity services are to be considered when all components are deployed on a single host, there will be 7 different services:

In case you were using a service account in Exivity version 2.x.x for the Exivity Scheduling Service and the Exivity Web Service, you will have to reconfigure this service account for both services, as well as the Exivity API Service. In most cases, the Exivity Database Service may continue to run un the Local System account.

In case your current Exivity version 2.x.x installation runs inside an Active/Passive Windows Cluster, you will need to re-register the Cluster Roles for the Exivity Scheduling Service. Additionally, a new Cluster Role should be created for the Exivity Database Service, in case you decide to not use an external database host.

Default TCP ports

In v2.x.x, the default port for the Exivity GUI was 8001 and 8002 for the Proximity API. Both services were already available through port 443 (the default port used for HTTPS traffic, which means clients don't have to explicitly specify the port) and in v3.x.x this will be used by default:

Default Security Settings

As of Exivity version 3, more strict security settings are applied by default. These can be found under Administration > Settings. One important item which should be considered when upgrading a multi-node environment is the use of CORS. It is required to list all possible front end UI nodes in the CORS origins field:

Transformer changes

@FILE_EXISTS and @FILE_EMPTY

Previously, it would only check for files in the system and exported folders within the Exivity home directory, and if a specified path + filename did not start with system/ or exported/ then these would be prepended automatically before the check was done.

This behaviour has been changed in the following ways in v3.x.x:

  • Any path + filename within the Exivity home directory can now be checked

  • path + filenames are now accepted and treated as being relative to the Exivity home directory

  • The folders system/ and exported/ are no longer automatically prepended

Consider a file somefile.csv in the %EXIVITY_HOME_PATH%/exported folder. Previously with version 2.x.x a user could check for the existence of this file using the following example:

In version 3 it is required to include the entire path relative to the %EXIVITY_HOME_PATH% :

This change may require modifications to existing Transformer scripts. This is because the system and exported will no longer be automatically prepended.

Extractor changes

HTTP server certificate validation

The default behaviour of the HTTP subsystem in USE was changed to fully validate server SSL certificates, which may cause some USE scripts to fail. This typically applies to data Extractors, which are connecting to on-premises data sources that use self-signed SSL certificates. In version 2.x.x, the default behavior was to ignore these certificate errors. You can identify these errors in your Extractor logs:

The above will apply to all of your Data Extractors where you are connecting to (most likely internal) data sources which are using self-signed certificates. Make sure to apply this change before upgrading to v3 to avoid any data extraction errors.

Extractor & Transformer Schedules

GUI changes

There are some minor changes in the GUI that are not backwards-compatible:

  • We removed the Excel export options from the reports. They were using the CSV format under the hood (i.e. they never actually produced valid Excel worksheets). In the future, we plan to implement proper Excel export formats for the reports, including a full summary report Excel export.

Changes to data processing for reports

Due to changes to the processing of reports, when making changes to either services, rates or adjustments, associated reports should be prepared. In v2.x.x this was already required when making changes to services or rates. Since v3.x.x this will be also required when making changes to adjustments. We've planned further improvements to make this more transparent (i.e. handling the preparation of reports automatically in the background).

API changes

Normalised date/time data in responses

Some endpoints were returning dates and timestamps in different formats. This has been normalised in such a way that all responses use the same serialization for dates and timestamps:

  • A date is always represented as ISO-8601 string: "yyyy-mm-dd", e.g. "2020-01-29"

  • A date/time is always represented as ISO-8601 string in UTC: "yyyy-mm-ddThh:mm:ssZ", e.g. "2020-01-29T11:26:52Z". Note that the Z suffix denotes the UTC time standard.

This affects the responses (attributes) for the following group of API endpoints:

  • /v1/audit

    • created_at

  • /v1/budgetrevisions

    • effective_from

  • /v1/dsets

    • earliest_rdf

    • latest_rdf

    • rdf_detail.created

    • rdf_detail.updated

  • /v1/extractors

    • last_modified

  • /v1/log

    • created

    • lines.date

  • /v1/reports

    • data_status.first_date

    • data_status.last_date

    • data_status.status.date

  • /v1/services

    • created_at

    • updated_at

  • /v1/transformers

    • last_modified

  • /v1/workflows

    • created_at

    • updated_at

  • /v1/workflowschedules

    • start_time

    • next_run

Changed functionality

/v1/reports/{id}/run endpoint

  • The JSON format in the response is simplified. See examples below.

This is an example response from v2.x.x:

{
   "report": [
      {
         "meta": {
            // ...
         },
         "error": false,
         "data": [
            {
               // ...
            }
         ]
      }
   ]
}

This is an example response from v3.x.x:

{
   "report": [
      {
         // ...
      }
   ]
}

Remove deprecated functionality

/v1/usergroups endpoint

Removed deprecated permission aliases.

  • upload_files (use manage_files instead)

  • manage_configuration (use manage_settings instead)

  • manage_system (use manage_settings instead)

/v1/reports/{id}/run endpoint

Functionality deprecated in v2.x.x has been removed. If you were relying on any of the following functionality, please use the suggested replacement instead:

  • The pdf/invoice option for the format query parameter has been removed. Please use pdf/summary instead.

  • The invoice_options query parameter has been removed in favour of summary_options.

/v1/configuration endpoint

  • Configuration keys prefixed with INVOICE_ are replaced by respective keys prefixed with SUMMARY_.

/v1/workflowsteplogs endpoint

Also relevant to other endpoints including workflowsteplogs .

  • Removed the last_log attribute. Include the last log by specifying the query parameters include=steplogs&related[steplogs][limit]=1&related[steplogs][sort]=-start_ts.

  • Removed the timestamp attribute. Use start_timestamp instead.

  • Removed the message attribute. Use a combination of status and output instead.

/v1/file endpoint

Only applicable to POST requests to this endpoint.

  • The filename in the response from this endpoint will no longer include the import/ prefix to better align for other requests in this endpoint. See the example below:

{
-    "filename": "/import/generic/2020/01/31_uploaded_001.txt"
+    "filename": "/generic/2020/01/31_uploaded_001.txt"
}

/v1/usergroups endpoint

The following permissions have been removed in favour of their new counterparts:

  • UPLOAD_FILES has become MANAGE_FILES

  • MANAGE_CONFIGURATION has become MANAGE_SETTINGS

  • MANAGE_SYSTEM has become MANAGE_SETTINGS

This is achieved by shipping a web proxy configuration for Nginx, which routes all requests starting with /v1/ to port 8002 and all other requests to port 8001. The recommended configuration is to not expose port 8001 to the public and only accept incoming traffic on port 443. Port 8002 may still be opened to external hosts, typically in a configuration where the Web and API/Backend components are deployed . In such a scenario it is advisable to allow communication from the .

Multiple hosts including https:// may be added while separating each URL using a , (comma) symbol. Wildcards may also be used as part of the hostname to match multiple URLs in one go such as: https://*.cors.exivity.io. An overview of all current security policies can be found .

The and functions in Transcript have been modified in a manner that may require changes to scripts that use them.

It is highly recommended to use valid SSL certificates, properly signed by a trusted CA (Certificate Authority). However, it is still possible to switch certificate validation off by specifying in an Extractor script before executing an HTTP request.

In version 2.x.x it was still possible to schedule an Extractor or Transformer from the editor screen. This feature is removed from version 3.x.x. In case you still have schedules that are configured through this interface, you should Unschedule these and create an appropriate instead.

Removed functionality that would take a custom API port from the #port=xxx location hash parameter on the login screen. Specifying a custom API port (and hostname) is still possible by on your system.

here
@FILE_EXISTS
@FILE_EMPTY
Workflow
external database server
set http_secure no
Web node towards the API/Backend node on port 8002
setting up a config.json file
as separate hosts
Windows Services with Exivity version 2.x.x
Windows Services with Exivity version 3.0.0 and =< 3.4.3
Windows Services with Exivity version 3.5.0 and higher
Default using port 443 for both UI and API
Administering security settings
v2: FILE_EXISTS statement auto-prepends exported / system
v3: Path must be relative to %EXIVITY_HOME_PATH%
Identifying Certificate Errors in the Extractor logs
Example v2.x.x Schedule which will be removed in version 3.x.x