A copy of the Exivity installer can be obtained on our website. Installing or upgrading to the latest release is a straightforward process, refer to the on-premises article for more information.
Migration to PostgreSQL within the Exivity architecture As of version 3.0.0 Exivity stores, all global configuration data and reports results in PostgreSQL as opposed to SQLite. This is a mandatory and breaking change. Please refer to the version 3 upgrade guide for more information.
As of version 3.2.1, quantity adjustments are applied first and before any charge related adjustments. This may potentially affect your billing records. If you were depending on mixing charge and quantity adjustments, please reach out to support@exivity.com for guidance.
As of version 3.5.0, Exivity uses the RabbitMQ message broker software for inter-component communication. By default, RabbitMQ will be installed automatically on the Exivity host. Alternatively, you may leverage a remote/dedicated RabbitMQ instance. Make sure to verify the system requirements in the installation guide before upgrading.
You may register with the following JSON feed to obtain information about new releases.
December 01, 2021
Bug fixes
Fixed an issue when installing RabbitMQ locally When using the interactive installer and installing RabbitMQ locally, the installer would store invalid values for the RabbitMQ configuration. This has now been resolved.
Fixed an issue when system returns from hibernation If a Windows system that is running the "Exivity Backend Service" would return from hibernation mode, the service (merlin.exe) would not recover and remain in a broken state. This has been resolved.
Fixed an issue with the Transformer editor Resolved a corner case whereby the editor could break when using Transformer editor code snippets.
November 22, 2021
Bug fixes
Adjustment type Fixed an issue where the type of adjustment was always shown as premium.
November 19, 2021
New features
The name of the EXIVITY_AGGR_COUNT column created by the 'aggregate' statement is now configurable
The 'aggregate' statement now supports an optional parameter (called 'counter_column') which allows you to specify the name of the column into which the aggregation counters will be written. Please refer to the 'aggregate' documentation for more details.
Transformer scripts will now log a warning if no RDF files were updated or created
When running a Transformer script that does not create or update an RDF file using the 'finish' statement, a warning will be written to the log to that effect.
The 'terminate' statement in a Transformer will now accept 'with error' as optional parameters
The statement 'terminate with error' is now supported by Transformer scripts and will cause the processing of the current day to abruptly end. A message will be written to the logfile to the effect that the error was explicitly requested and the task will terminate. If the 'continue' option is enabled, processing will resume on the next day in the range (if a range was specified) else the workflow will exit.
Bug fixes
Fixed an issue when executing quoted scripts from a workflow
When a script (i.e. Powershell) was quoted in a workflow step, the workflow would not execute the script. This has now been resolved.
Fixed an installer issue
When installing RabbitMQ into a program path that contains spaces, RabbitMQ could fail to install the service. This has now been resolved.
Increased the number of API handlers
This release now runs 18 concurrent API processes running on a node with the API service installed to allow a higher number of parallel requests to the REST API.
Fixed an issue with report level labels
Fixed an issue where report labels weren't updated.
Fixed an issue with the Adjustments
Resolved an issue where details in some cases weren't reflected in the form of the Adjustments screen.
Fixed issue with archiving old log files
In some cases, collecting old log files resulted in archiving some files that were already archived before.
Fixed issue with updating an extractor
When an extractor was updated with NULL arguments, the request was incorrectly considered invalid.
Fixed an issue when creating budgets
Resolved an issue where there was unexpected behavior when creating a multi-level budget in the GUI
DEBUG level logs no longer contain the value of parameters provided to subroutines
The values of parameters passed to subroutines are no longer included in DEBUG level logfiles as such values could potentially contain sensitive data
Fixed an issue when viewing proximity log
Solved an issue where certain log types caused the log screen to fail.
Fixes issue with some users not being able to change password
Some users with view_report permissions only were not able to change their password. This issue has now been fixed.
Log message could contain metadata
Logs entries can contain 'context' to help debugging. This is usually one JSON string. Some log entries could contain multiple contents, which was unexpected. Now if this occurs, these contexts are merged into one JSON string.
Fixed an issue when clicking arrow list
Solved issue where a list could disappear when clicking the arrow in a dropdown list in the GUI
Fixed an issue with the report level filter
Resolved an issue where a default report level wasn't selected when selecting a new report in the GUI
Clarified the log entry written if an extractor script has an unmatched double quote on the last line
In cases where the last line of an extractor script has an unmatched double-quote, and that last line is also not terminated with a carriage return, then an internal error would be generated. The result is correct (the script will not execute) but the error has now been changed to clearly indicate the exact nature of the problem.
Fixed an issue related to sorting of datasets
Resolved an issue with the date sort of datasets in the GUI
September 15, 2021
Release Candidate - available to selected users only
New features
Added more information to audit logs when deleting services The audit log now contains the service key when a service is deleted. Previously it recorded the event,\ but did not include specific information about the service itself.
Added Filtering on workflow-step type The feature allows users to filter workflow steps by type.
The installer has now valid code signing applied
As of this version, the Exivity installer will be signed using the Exivity code signing certificate in order to increase the security of the distributed installer package.
Bug fixes
Increased the number of API handlers This release now runs 18 concurrent API processes running on a node with the API service installed to allow a higher number of parallel requests to the REST API.
Fixed an issue with overwriting a Lookup table Solved an issue where saving a lookup could lead to unexpected behavior
Fixed an issue with user notification subscriptions Solved an issue where it was not possible to save a notification as a user
Fixed an issue when exporting the Consolidated Summary report Solved an issue where the consolidated export was only exporting the first account
Fixed a security issue related to User Access Control Solved an issue where the cache was not properly cleared and as such, the user could view invalidated data.
Fixed an issue with sorting service names in the summary report Services in the summary were sorted on ServiceName instead of Service Description. This has now been resolved.
Fixed an issue where a certain type of invalid XML parslet would cause an extractor crash In rare cases, using an XML parslet to extract an attribute value could cause a crash. This has now been fixed.
Fixed missing budget filter "service" bug When a user selected the "service" filter in a budget, this data was not stored correctly. This fixes this issue.
Fixed an issue with auto-completion There was an issue where auto-completion of example blocks in the Extractor/Transformer editor would not support "tabbing-through" function parameters. This has now been resolved. Additionally, an issue was fixed where the "rename" function would autocomplete wrongly to "convert".
Fixes issue with links generated from singular form of the resource type Links for relationships inside responses were generated using the singular form of the resource.
Fixes issue with incorrect links generated inside responses When a response contained a "link" attribute, it was incorrectly generated.
Better handling of hash and whitespace characters in quoted strings within extractor scripts Previously, a hash symbol at the start of a word in a quoted string literal would be treated as the start of a comment, causing the string to be truncated and an "Unmatched quote" error generated. Additionally, quoted strings containing multiple consecutive whitespace characters would be modified during execution such that they were replaced with a single space. These issues have both been fixed.
August 27, 2021
Release Candidate - available to selected users only
Bug fixes
Fixed an issue with the ordering of headings in the Transformer Preview There was an issue where the Transformer Preview would sort the headings in reversed order. This has now been resolved.
Fixed an issue when creating a Transformer from a template When creating a transformer from a template, after creation the GUI would show a "leave the page" popup. This has been resolved.
Fixed an issue when creating an Extractor from a template When creating an Extractor from a template, after creation the screen would not switch to the Variables tab. This has now been resolved.
Resolved issue where the user couldn't set alt_interval The user couldn't save a subscription that used alternative intervals because of invalid values. This is now solved.
August 18, 2021
Release Candidate - available to selected users only
Bug fixes
Fixed an installer error for the workflow migration script PHP environment errors could show up in the installer log during the execution of the workflow migration script. This has been resolved
Added cleanup steps to uninstaller to remove PSQL and RabbitMQ traces
Hidden scrollbar preview Resolved issue where the scrollbar wasn't visible in a Transcript Preview.
Fixed an issue with environments and global variables in Transformer scripts Attempting to use different environments, and the global variables within them, in a transformer script may have caused an error and the script to fail. This has now been fixed.
Workflow status Resolved an issue where the GUI could crash because of an invalid status state
July 26, 2021
Release Candidate - available to selected users only
New features
Implemented account lockout strategy User accounts will now get locked out for 15 minutes after 5 consecutive failed login attempts.
Account names are now set by the latest date seen Previously, when preparing reports account names were set based on the last data to be processed, even if that data was older than that used to originally set the account name. Now, when accounts are being synchronised during the report preparation process, names will only be updated if the data being processed is newer than that which was used to last set or update the name.
Increased calculation precision in Transformer scripts The precision of calculations in Transformer scripts has been increased to 14 decimal places.
Added new log files to the log rotation policy Log files generated by the Notification component and by the new Scheduler component are included in the log rotation policy.
Added the option to run a workflow for a date range When running a workflow manually, you can now choose to run it for a date range, not only for a single date. When running against a date range the steps are executed individually for each day in that range.
Redesign of the Data Pipelines > Workflows screen Complete overhaul of the Workflows page, improving the configuration user experience for individual steps and making it easier to review associated step logs.
Added a cascading DELETE parameter to the API
API now supports a new cascade: boolean query parameter. This parameter, if true, will allow the severing of non-nullable relationships and will remove the related record afterwards. This new parameter specifically applies to the workflowstep and workflowschedule endpoints.
Data aware report filters Report filters only show entities available in the selected reporting date range
Highly available scheduler The scheduler has been rewritten to support the new message bus architecture and has the ability to run on multiple nodes at the same time.
Creating a new report now requires the "all accounts" permission Only users that have access to all accounts will be able to create new reports. This avoids situations where a user could previously create a new report that was not visible after creation.
API support for changing user login method It is now possible to change an existing users authentication method to local, LDAP or SAML2. Currently this is only supported by invoking the the REST API.
Changed behaviour of logout when Single Sign-On is configured Exivity now allows the logout endpoint (SLO endpoint) to remain unconfigured, in which case the user will be logged out only from the application, not from the entire Single Sign-On environment.
Budget notification
Summary report notification Users can now choose to receive a pre-defined report in either CSV or PDF format through a notification
File available notification
Admin managed notifications Admin users and users with the manage_users permissions now have the ability to add notification channels and subscriptions to other users. This allows admin users to grant users subscriptions to notification types they would otherwise not have.
Minimum commitment and Adjustments included in cost summary subtotals Details of minimum commitments and adjustments are now included in the cost summary subtotals for service categories.
Webhook notification channel Allow webhooks as notification channel, forwarding the original event as JSON payload to a custom webhook URL
Implement internal communications between components using RabbitMQ message broker This allows for improved multi-node setups
Bug fixes
Fixed an issue with manually created accounts When creating accounts manually in the GUI, on occasion an account could be created which did not fit correctly into the account hierarchy. This has now been fixed.
Fixed and issue with updating account names When preparing a report, it could be that some account names were not updated when they should have been. This has now been fixed.
Fixed saving service resources including 'budgetitems' relationship
When saving a service resource including the budgetitems relationship, an error was returned. This has been fixed and the relationship between service and budgetitems works both ways.
Fixed an issue with some combinations of HTTP methods in extractors In extractor scripts, making an HTTP DELETE request could cause subsequent HTTP calls to continue to use the DELETE method even if another method was specified. This has now been fixed.
Added the 'notificationsubscription' relationship for a user entity
When requesting a user entity, the notificationsubscription relationship can now be included in the response.
Audit screen freeze Fixed an issue where the audit screen could become unresponsive
Fix Publish Report notification
The clear option for the account field now works.
Fix state update adjustments Fixed an issue where a field wouldn't be updated when commuting to the next resource.
Subscription inherited rates Fixed some incorrect behavior with rate inheritance on the subscription management screen
Removed the limit of CSV files that could be open at once in extractor scripts Previously, a maximum of 16 CSV files could be open for writing in extractor scripts. This limit has now been removed and any number of CSV files can be open simultaneously
Fix decimal inputs Fixed an issue where an input would not accept a decimal value with leading zero.
Lookup file Fixed an issue where switching between lookup files wouldnt cause the screen to be updated
Show manually created service Solved an issue where manually created services where not visible before a browser refresh
Disclaimer modal Fixed an issue where the disclaimer had to be dismissed twice
Fixed an issue with compound MSSQL statements in extractors where the first statement is not a SELECT
Usually, SQL statements executed in an extractor script are based around some form of SELECT. However in some cases more complex compound statements are required. Previously, a statement using a temporary table in MS SQL server, such as ...
CREATE TABLE #jos(Name char(20)) INSERT INTO #jos (teller) VALUES ('2') SELECT * FROM #jos;
... would not return any results, as the INSERT statement returns an empty result set. Extractors now ignore empty result sets in such statements and return the first result set that is not empty (assuming there is any data returned by the query at all).
Fixed an issue with transformer scripts when importing data with non-escaped quotes Importing data with non-escaped quotes could sometimes cause Transformer scripts to fail. This has now been fixed.
Fixed types for returned attributes When accessing various endpoints, id attributes were returned as integers instead of strings. This was fixed to comply with JSON:API specifications.
Fixed incorrect display of date fields in slack notifications When a slack notification was sent, the date of the notification was wrongly set to sometime in the future
Fixed incorrect display of extra HTML tags in emails When emails are sent, some HTML tags were shown as text in the email body
Fixed an issue with encoding of double quotes when exporting account data as CSV
Fixed long delay when displaying a large number of budgets when including reports
Fixed an issue with displaying workflow start date in dashboard Previously the start date of a workflow was displayed incorrectly in the Workflow dashboard widget
Fixed an issue with creating adjustments There was a problem where the GUI did not allow the creating of adjustments that had a non-integar amount. This has now been fixed.
Fixed a divide-by-zero error on reports In corner cases where the final chargeable quantity for all line-items on a report is 0, a divide-by-zero error would occur. This has now been fixed.
Fixed the description of the 'Budget' permission This permission only provides access to the Budget report. In the previous version, the description of this group permission contained false information.
Fixed an issue when configuring a Budget for a leaf node
If a leaf node in the budget structure has the distribution set to even or account, the Budget engine generated a false warning that an amount for the node is left undistributed. This has now been resolved.
March 22, 2021
New features
Usage data generation based on consumption start/stop events Added functionality to produce usage data from consumption start/stop/update events, including the consumption which spans several days. See event_to_usage statement description in Transform documentation for details.
February 24, 2021
Bug fixes
Fixed an issue with SAML user account access provisioning Previously, user account access provisioning would grant access to a matching account, including all its child accounts. With this release, the SAML user account access provisioning only grants a user access to the first matching account (i.e. with the lowest depth) in the account hierarchy.
Fixed an issue where reports could become invalid for certain user account permission configurations
Revolved a race condition related to preparing a report On rare occasions the actions of multiple users administering accounts and reports simultaneously could result in duplicate charges in reports. This has now been fixed.
Fixed a rare cosmetic issue when updating rates in ETL When executing a 'services' statement in a Transformer, if there was an existing service rate revision with a different rate for the same effective date as a new revision then an error containing the text "incorrect binary data format" would be logged. This issue was cosmetic, but has now been fixed.
Fixed a very rare issue with minimum commit Resolved an issue whereby it was possible on extremely rare occasions for the minimum commit calculations applied to one service to be incorrectly applied also to other services in a report.
December 08, 2020
Bug fixes
Fixed an issue with PDF export of the summary report In certain environments, the PDF export feature was broken. This issue has now been resolved.
Default SAML user group re-applied when a user logs in An issue has been resolved which caused the default user group to be re-applied when existing users were logging in through a SAML Identity Provider.
November 17, 2020
New features
Executing a Reports now requires a valid license
Bug fixes
Fixed a minor issue with the calendar widget
Fixed a minor issue with the SAML SLS endpoint
Fixed an issue with custom rates and subscriptions
Sometimes a subscription would not reflect changes in custom rates for a service. This has now been fixed.
November 02, 2020
Due to security considerations, starting with this release, each user is bound to the logon provider set in the source attribute. E.g. if a user has their source set to 'local', they can't log in with the SAML2 or LDAP SSO provider.
New features
Improved user provisioning (beta) Added options to provision user attributes from a SAML Identity Provider/AD response. It's possible to provision the users display name, username and email address.
Added claims based account access provisioning (beta) Added options to provision users permission levels using attributes from a SAML Identity Provider/AD. It's possible to provision both the usergroup and user account access (by matching either the account key or metadata value to a SAML/AD response attribute).
Bug fixes
Change to log filenames for extractors and transformers When scheduling extractors and transformers with workflows, the selected environment is now part of the log filename to distinguish their log files when they are running at the same time.
Fixed opening curly brace detection When opening curly brace wasn't preceded by space, sometimes it wasn't properly processed. It has been fixed
Improved error message on lookups screen Fixed an issue where a user might get a confusing error message when saving a lookup file.
Fixes initializing directories in home directory There was a bug when the home directories were initially created and two directories were wrongly created in the /home directory.
Improved support for UTF-8 in usernames
October 09, 2020
New features
Added support for the Safari web browser
Exivity now supports the Safari web browser
Added ability to set custom escape character in Transformer Previewer Added a dropdown to the transformer previewer where the client can select which escape character to use during Transformer preview mode
Bug fixes
Fixed an issue with the Dataset Manager Resolved regression issues regarding the display of RDF dates
Fixed an issue with report filters There was a bug if a user accidentally submits a string instead of a number on filtering 'parent_account_id' with report. This is now being fixed
Fixed an issue with displaying workflow start times in the workflow list The time column of the workflow list omitted the hour at which the workflow would run for daily and monthly workflows
Resolve global variables in "uri" statement
In some statements (i.e. uri) global variables weren't properly resolved. This has now been fixed
Fixed an issue with COGS charges for services with an average monthly charge model COGS charges for monthly services that used the average charge model could be slightly lower than they should be, as the COGS rate for the first day seen was not factored into the rate averaging. This has now been fixed
Fixed an issue when managing rates An aggregation_level error could occur when updating rates. This has now been resolved.
September 30, 2020
Bug fixes
Improved error message when creating invalid revision When creating a duplicate rate revision for the same effective date, an appropriate error message is now shown
Improved error message for LDAP Added a more meaningful message when unable to connect to an LDAP server
Service Category names must be unique It was possible to have duplicate service category names. This could lead to confusion and has now been resolved. The Service Category name must now be unique
Resolved a timezone offset issue for Workflows Resolved an issue where timezones were incorrectly applied when creating workflows
Resolved an issue with the Reports menu Fixed an issue where the report screen would crash after selecting multiple reports
September 9, 2020
Bug fixes
Increased the boundaries of some USE arithmetical operations
The operators +=, -=, *= and /= were limited to a 32-bit range and this could cause an overflow in certain real-world applications. The range has now been increased to match that of the other arithmentical operations which is based on the range of a 64-bit floating point value: 1.7E +/- 308 (up to 15 digits).
Fixed an issue where the CSV export of the instance report could fail The CSV export of the instance report did not work when the usage column was included and some of the instances did not have any usage data
One-off subscriptions should ignore the charge_day field Only recurring subscriptions require the "charge day" field, but it was required for all subscriptions. This has now been fixed
Fixed a decimal precision issue with the timeline graph The Y-axis of the timeline sometimes displayed long floats. To resolve this the precision has been fixed to two decimals
Fix for "space" as thousand separator It is possible to select different symbols as a thousand separator for large numbers shown to users. One of the options, space (" "), was not saving correctly. This has now been resolved
Fix error handling transformer Fixed an issue where it was possible that a failed transformer error was not processed properly which prevented the transformer from giving the user feedback
Added index to speed-up report preparation
During report preparation, Edify executes several queries to the adj_lookup table. An index has been adding to this table to improve performance
September 1, 2020
Quantity and Charge Adjustment are now handled in strict order As of this release, quantity adjustments are applied first and before any charge related adjustments. This in turn enables the ordering of individual adjustment policies.
Implemented translations settings (Beta) Implemented an automatic translation feature for Dutch, German, and French. This can be configured on a system or user level. Currently, this is released as a beta feature.
Delete old Nginx log files Old Nginx log files will now be deleted by the garbage collector. This will help reduce disk space.
Improve user error page Updated the error page to improve user experience. Customer logo will now be shown here, if it has been set.
Adjustment API endpoint will now accept order In the past, there was no way to reorder adjustments. Users had to delete and recreate them to do this. This has now been solved. The order field now takes in an order number. If this is not used, the current functionality will still work, using the creating date instead.
Reports can now include adjustment name The adjustment name can now be added to a report. This is useful for report users that do not have access to view the whole adjustment.
Support in UI for changing the order in which Adjustments are applied Since it is possible to create multiple Adjustments for a single account, they may affect each other. It is therefore desirable to have the ability to control the order in which they are applied. As of this version, it is possible to change the order in which Adjustments are applied.
Bug fixes
Fixed an issue when partial preparing manual services When partial prepare kicks in, it previously skipped manual services. This has now been resolve.
Fixed SAML ACS invalid schema error
Some users were experiencing an Invalid Schema bug when accessing the /v1/auth/saml/acs endpoint. This has now been fixed.
Fixes "Nginx Log File" not found error The location of some log files could not be found. This has been resolved.
Improved invalid token handling
Changed how this error is logged. It will now be treated as a notice instead of a warning.
Fixed an issue with running transformers
In release 3.2.0 there was a problem running transformers from the GUI. This has now been fixed.
July 21, 2020
Bug fixes
Premature validation warning
Resolved an issue where a user could get a premature validation error for certain input fields.
Fixed an issue with the budget report
With certain budget configurations, the budget report would display a message instead of the report even if the budget configuration was valid. This has now been resolved.
Improved user message when execution time is exceeded
Improved the message when the execution time of an Extractor, Transformer or Report is being exceeded
Fixed an issue with budget leaf account distribution When configuring a budget for a budget leaf, the distribution setting will be forced to be of the type "shared"
Fixed an issue when deleting services Previously it wasn't possible to delete a service if it still had any associated rates. This behavior has now been corrected so that services including its relationships will be deleted
Fixed an issue with the Garbage collector exits code The Garbage collector would previously always return an exit code of 259 when invoked manually. This has been resolved
Fixed an issue where subscriptions could show up on more than one report In some cases, a subscription (created for an account associated with a specific report) could be shown when a different report was run. This has now been fixed
Fixed graph image export When exporting a graph chart (i.e. Pie, Bar, Line) in any of the reports (Accounts, Services Instances) the file download would not start. This has now been resolved
Restored indicators for account access list Pencil-like indicators will inform the user where nested accounts have been selected. This functionality was unintentionally removed from the user management screen but has been placed back
Horizon performance improvements Implemented two seperate changes to improve Horizon budget execution performance. (1) budget gets validated only once after a budget configuration change and (2) added a new database index to improve database query execution
Always show decimal values Resolved an issue where some values would be displayed using their scientific notation
Resolved issue when deleting services Fixed an issue where the user would get an error when making multiple delete requests for services
July 3, 2020
New features
Ability to skip database backup when updating When updating the software, the user can now skip the PostgreSQL database backup to improve update speed
Bug fixes
Fixed an issue with the webproxy.conf proxy_pass URL
When updating the software, the proxy_pass URL for the webproxy.conf NGINX configuration was always reverted back to https://127.0.0.1:8002. This has now been resolved
June 28, 2020
Fixed charges no longer available As of version 3.1.0, we're removing fixed charges (or: interval-based charges) because of the limited amount of use cases and low customer adoption with this feature. In case you are an Exivity customer and are still using this service parameter, please reach out to support@exivity.com so we can provide you with alternative solutions.
New features
Improved auditing when managing accounts manually Added a number of new audit points when performing manual account management
Auditing token creation correctly logs client IP address when API is behind a proxy server
Updating user profile information now requires providing the current password
Implemented current password validation A user is now required to enter his/her current password before any changes to its profile are applied
Added logging audit entries for metadata, service subscriptions, and budgets
Bug fixes
Fixed an issue where some users could select budgets even if it would not contain any details
Fixed an issue which sometimes caused the workflows screen to load for a long time A resource-intensive operation that ran whenever the workflow page is visited got removed
getCUPRs function ported to PSQL getCUPRs function was changed from using SQLite global DB to PSQL global DB
Mitigated a potential security issue (internal reference EXVT-3773)
Fixed an issue were sometimes log files would appear malformed in the Transformer run tab
Reverted the way a user deletes an RDF
Fixed form behavior in rates screen
Fixed "other" option behavior at services
Fix report filter When switching reports, some of the filters were not reset and appeared unused while they actually held a value not related to the currently selected report. This made the reports appear empty. This has been resolved.
When the API runs out of memory (possible for very large reports) it will respond with a descriptive message
Incorrect display of minimum_commit_delta_charge and minimum_commit_delta_quantity When a minimum commit quantity was set for a service rate revision, the resulting charges could show up incorrectly by having the quantity and minimum commit delta swapped on the reports
May 26, 2020
Bug fixes
Fixed issue at /lookups where in specific cases an error was undefined
Fixed issue where user was shown an incorrect validation warning When providing a number with more then one decimal, the user interface would show an error message. This has been resolved.
Added cache check in partial preparation There was an issue that partial preparation code didn't check for cache table presence before analysis, which caused execution errors, and it has been fixed.
Fixed escape option behaviour in 'import' There was an error that '\"' sequence was always treated as escaped quoted even when 'escape' option switched off (default), it has been fixed.
Rate revision date never changes Transcript was trying to change rate revision date when it detects an attempt to create new revision with the same attributes but earlier date, which conflicted with database constraint, so it was removed in order to avoid execution errors.
Fixed issue where line breaks were not shown When providing an address for the summary report with multiple new lines, only the first would be shown. This has been resolved.
Fixed undefined tool tip at workflows The was a column which showed a tool tip with undefined as value. This has been resolved.
Select a single date for transformer run After introduction of a new calendar, the single date selection was missing. This has now been resolved.
Services and categories at subscriptions are now alphabetically sorted
Fixed an issue where dataset columns were missing After repreparing a report, the columns associated with a dataset were not visible. This has been resolved.
Fixed visibility newly created dataset When running a transcript, newly created datasets were only visible after refresh. This has been resolved.
Show red underline when incorrect value for textarea input
Show correct validation values when saving environment
When saving an environment, we will now show the correct error messages for missing values.
April 30, 2020
New features
New feature: Subscriptions When Beta features are enabled, users can now create one-off and recurring daily, monthly or yearly subscriptions for services for which a metered data source is not available.
New feature: Workflows widget When Beta features are enabled, a new Workflow traffic-light widget will be shown on the main Dashboard page.
New feature: Global Variables When Beta features are enabled, users can now manage Global Variables under System Administration. This enables users to decouple Extractor and Transformer variable values from scripts thus supporting different values on a per Exivity instance basis without changing underlying scripts.
New feature: Metadata for Services Metadata can now be added to all services just like with accounts. Define a metadata definition first, then attach the definition to a dataset in Data pipelines > Datasets. All services in this dataset will now use this set of metadata fields. Metadata information itself can be added and modified in Services > Overview and is available in the services reports.
Added more information to the summary report The summary report now contains the following additional columns: service_key, account keys (between 1 and 5 inclusive depending on the report), start_date, end_date. These columns are also included in CSV exports.
Improved quoting in exported CSVs To avoid potential complications with Excel, any cell values in CSVs exported from Exivity that are not numbers and begin with any of the characters =,+,- or @ are now preceded with a single quote.
A new setting has been added to allow users to stay logged in. If disabled (default), users need to log in each time they open Exivity in their browser.
Administrators can now set the token lifetime. When the token lifetime expires, users need to log in again.
Added ability for users to log out of all devices.
Mitigated a potential security issue (internal reference EXVT-3457)
Mitigated a potential security issue (internal reference: EXVT-3455)
Mitigated a potential security issue (internal reference EXVT-3270)
Implemented a new charge model for monthly services The charge for a monthly services may now be based on the usage of a specific day in the month.
Improvements in report performance Reports are now pre-loaded in the database which speeds up report loading times in the GUI.
Added support for LDAP authentication LDAP authentication was available as beta feature already, and is now generally available. A guide will be added to our documentation soon. Configuration options are available in the Settings screen (Single sign-on tab).
Bug fixes
Fixed an issue with manage_metadata_definitions Granting the "Manage Metadata" permission is now possible when creating a security group
Fixed an edge case for incorrect net value on accounts table Whenever cogs or charge had 0 as value the net value wouldn't be displayed correctly on the details table of the accounts report.
Fixed an issue when creating a manual service Manual per unit cogs type services will not require cogs column
Fixed invalid format issue metadata lists Previously empty lines we're persisted as list option. Options are now trimmed and validated before persisting.
The API now requires additional attributes for certain service types
Fixed a small issue with the metadata selector in the report management screen
Removed toolbar from PDF export The Summary report PDF export included the toolbar on the top of every page of the PDF document. This has now been removed.
Fixed an Extractor XML parsing issue USE could previously fail when iterating over an empty XML node inside a foreach loop. This has now been resolved.
Fixed modified display issue in Accounts Overview When switching from Account Name to Account Key in the Accounts Overview screen, the modified state / pencil icon disappeared previously. This has now been resolved.
Fixed an issue when scrolling in the Accounts Overview When having a large amount of Accounts in a report definition, the Accounts Overview screen could generate cosmetic glitches when scrolling
Fixed a Transformer issue with skip_corrupted_records
When import option skip_corrupted_records was set, import could fail if last column in the record is missing closing quote. This has been resolved.
Fixed a Transformer issue with aggregate When using the aggregate max function, Transcript could pick the wrong value.
Fixed a false warning about invalid COGS type in the logfile
When preparing reports, in some cases a lot of warnings would appear in the logfile that state:
Invalid cogs_type (0) in database for service ID nnn
where "nnn" is a service ID. This was a false warning and could be ignored but looked concerning and could lead to larger logfiles.
This issue has now been fixed.
Removed "remainder" option from lowest level accounts when setting budgets When a budget is set for an account at the deepest report level, the "remainder" option is no longer shown, as there are no sibling accounts to share the budget with.
Fixed an issue where (harmless) SQL errors could be logged when deleting services When deleting services, SQL errors could be present in the logfile for days where reports that reference those services had not been prepared. This has now been fixed.
Older release notes can be found here.
This page lists known issues with the latest version of Exivity. All isues that are resolved and released are mentioned under the Releases section
All versions of Exivity
When creating two Absolute Discount Adjustment Policies, where both Policies apply to the same Account, Service and date Period, there is a chance of generating negative charges for the services where both Adjustment Policies apply to as can be seen in the following screenshot:
The current workaround is to avoid stacking multiple Absolute Adjustments for each unique Account + Service combination. Instead, try to combine them within a single Adjustment Policy
Status
This issue is currently being resolved (internal reference: EXVT-<TBD>) and will be included with the release after version 3.5.7.
Exivity versions 3.5.0 till 3.5.6
When installing or upgrading to version 3.5.0 - 3.5.6, when installing RabbitMQ on the local system, the initial configuration values are incorrectly populated:
To work around this issue, you may uncheck the checkbox Install Local RabbitMQ Engine and then check the same checkbox again. The values will now be correctly populated.
In case you have already executed the upgrade progress and your system is broken, you will need to change the config.json to align the mq settings:
Status
This issue is currently being resolved (internal reference: EXVT-5042) and will be included with the 3.5.7 release.
Exivity versions 3.5.0 till 3.5.5
When a new Adjusmtent Discount policy is created, it will be created as a Premium policy instead. To confirm this behavior, when refreshing the screen they will show up as Premium:
In order to create a discount policy in version 3.5.5, use the API interface to create one.
Status
This issue is resolved with the 3.5.6 release.
Exivity versions 3.5.0 and higher
When running the installer, an error could popup where the installer is unable to update certain files such as the erlsrv.exe executable:
This issue occurs when Windows or a 3rd party application locks this file. This can happen with certain monitoring and anti-virus software. In the example given above, the solution would be temproraly stopping the Windows Event Log service, as shown in the screenshot below:
After stopping the service, click the Retry button in the installer.
All version of Exivity
When configuring a prorated monthly service, and then applying a relative quantity discount (adjustment), in certain cases the total charge can be less than 0 resulting in a credit. Although this might be an unusual configuration and might not always happen, it is important to be aware of this behavior.
This issue is pending. (internal reference: EXVT-1337)
Exivity version 3.5.0 and higher
Some Anti-Virus software vendors (specifically: McAfee and APEX) incorrectly flag the Exivity binary merlin.exe as a backdoor/trojan horse. As a result, the Exivity Backend Service might not be able to start as shown in the example below:
The Windows event viewer may display the following error (depending on your A/V vendor):
Response by certain A/V vendors:
[merlin.exe] is detected by [a coupe of] AV's as a BackDoor as it's functionality is very similar and malware authors can use this to their own advantage to compromise systems and gather sensitive system information.
We also understand that [merlin.exe] can be used for legitimate purposes and therefore we will change the classification of the sample submitted to PUP. This way will be able to exclude detection through our scanner configuration but at the same time we are still protecting our customers who may be not be aware of this software running on their systems.
Hence if you want to allow this application in your environment kindly exclude it by detection name.
General Exivity product announcements.
March 25, 2020
Bug fixes
Fixed an issue where deleting a metadata definition could remove a dataset metadata entry from the database. No actual data was affected by this bug.
Fixed an issue related to Transformer preview The preview menu could reference an incorrect line number when reporting a Transcript code error.
Fixed an issue when renaming a DSET
If you refered to an incomplete DSET name in a rename statement, Transformer could crash.
Fixed an escaping error when importing file in a Transfomer
The Transformer escape option doesn’t escape last quote: \“ at the end of a field
Fixed a Transformer issue with skip_corrupted_records
When import option skip_corrupted_records was set, import could fail if last column in the record is missing closing quote. This has been resolved.
February 03, 2020
New features
Added support for LDAP authentication LDAP authentication was available as beta feature already, and is now generally available. A guide will be added to our documentation soon. Configuration options are available in the Settings screen (Single sign-on tab).
When a user session is about to expire, an option is provided to prolong the session
Added option to specify custom dataset name in transform previewer
Added 'unsaved' warning in extractor/transformer editor A warning is displayed in the toolbar when editing an extractor or transformer and changes are not saved yet.
Added option to run workflow steps in parallel When adding new steps to a workflow, it is possible to uncheck the 'wait' toggle for any given step. This will then run the step simultaneously with the previous step. When the wait toggle remains checked, all previous steps will finish executing before the step is started.
Added ability to search parent account names In the Accounts report, it is now possible to search by parent account names in the 'detailed' table.
Added columns in reports data table Added account and service key as extra columns to the reports 'detailed' table. It is also possible to search for, and export, these values.
Updated Nginx and PHP Updated the Nginx webserver to version 1.17.8 and PHP to version 7.3.14
Custom escape character in extractors The statement now accepts an optional escape character to be used instead of the default backslash when escaping quotes in the value of a variable.
Implemented support for JWT web token authentication in extractors In order to support sources that require it, such as Google Cloud using OAuth 2.0, USE now supports the generation of a JSON Web Token. For more information please refer to
Search by account / service key In the Accounts and Services overview screens, it is now possible to search by account/service key instead of the name.
View metadata in report data tables Metadata fields can now be selected as optional columns in the Accounts and Services reports' 'detailed' tables. They also appear in search and CSV export.
Added support for Firefox browser
Improved detection of web-app URL This is especially useful in features such as SAML authentication and the sending of notifications (e.g. e-mail). The app URL is auto-detected and can be modified in the Settings screen (System tab).
New permissions added Two new permissions have been added to usergroup settings: Manage metadata definitions and Manage datasets. If a usergroup is set to have all permissions, these new ones will automatically be granted to members of that group.
Added metadata for services Metadata can now be added to all services just like with accounts. Define a metadata definition first, then attach the definition to a dataset in Data pipelines > Datasets. All services in this dataset will now use this set of metadata fields. Metadata information itself can be added and modified in the Services > Overview and is available in the Services reports.
Breakdown information available for monthly services For monthly services, breakdown information is now available in the Instances report 'detailed' table. To view this, make the 'Usage' column visible by clicking the overflow menu > Columns > Usage. The daily usage breakdown for monthly services is available in a pop-up screen.
Added account breadcrumbs in Account report legend
Datasets can now be managed from a dedicated screen Navigate to Data pipelines > Datasets to delete individual days from the set, and to assign a metadata definition to all services in a dataset. We've also made it easier to delete multiple days at once.
Bug fixes
Fixed a small visual issue in the 'about' page
Fixed a rare issue with metadata In some cases, selecting a value from a 'list' type metadata field could lead to a crash, This has been fixed.
Fixed a small visual issue with updating workflow schedules
Fixed a crash on the adjustments screen When no reports are available in the system, the adjustments screen could crash. This is now fixed.
Improved error reporting when importing non-matched CSVs in a single operation When importing multiple CSV files into a Transformer using a regex to match the filenames, if one had differing columns to the others then an internal error was reported. This is now correctly reported as a normal error as opposed to an internal one.
Sub-directories in the lookup folder are no longer displayed in the Lookup editor
Fixed an issue with services using an average charge model For monthly services with an average charge model, if there was usage on the last day of the month then the quantity on that last day may not have been not be factored into the average calculation. This issue has now been fixed.
Please note: It is recommended that the reports are reprepared where possible to ensure accurate historical reporting.
Fixed an issue with links on the Accounts overview page Fixed a minor issue where a link to Data pipelines > Report could sometimes be shown even if a user didn't have the appropriate permissions.
January 02, 2020
Bug fixes
Fixed an issue when aggregating an empty DSET Aggregating an empty DSET cauld case a Transcript crash, this has now been fixed.
Fixed resource leak in Workflow Engine Aeon was leaking Windows handles for internal I/O events that caused problems creating new workflow processes after several days of heavy use. This problem is fixed now.
December 11, 2019
New features
Improved aggregation function in Transcript
Aggregation performance improved, especially when processing large sorted datasets
Bug fixes
Resolved a rare Edify crash Resolved a rare bug with 'end of file' checking in the Edify pre-processor
Fixed an issue where changes to workflows were not processed in the scheduler
Case-sensitive column names in correlation
Correlate doesn't fail with SQL error when two columns in DSET have similar names that differ only in case
December 04, 2019
New features
Restricted editing of users logging in from SSO providers Updating the username and password in the API is no longer possible for users logging in from SSO providers.
Add PUBLIC_ROOT to configuration Previously, if Glass and Proximity were on different machines, Proximity would guess the Glass base URL. This is not possible to set this variable via the PUBLIC_ROOT configuration value.
Bug fixes
Fixed an issue with character encoding in the transformer and workflow API endpoints
November 25, 2019
Bug fixes
Implemented a fix to avoid global rate revision changes The global rate revision for services that are populated using the set_rate_using parameter in the services block, could cause the rate to be updated. This behavior has now been changed and existing rate revisions will no longer be touched.
November 15, 2019
New features
Improved the performance of the workflows screen when there are a large number of historical workflow runs.
Bug fixes
Fixed a cosmetic issue on the reports page.
Fixed an issue with the search not showing results on the dashboard.
Restored the functionality to cancel a login attempt.
In the transformer preview, datasets are now detected when the import statement has indentation.
Fixed an issue where no loading indicator was shown on the login screen.
Fixed an issue with overwriting services
Fixed an issue which would cause errors to appear in the log (and the service not to be updated) when overwriting a service with an updated version of itself.
November 13, 2019
New features
Added a warning when the current sessions is about to expire. A user then is given the option to renew the session without having to log in again.
Added transformer error annotations. When previewing a transformer script containing an error, the editor will show an annotation on the line where the error occurred.
Small changes to settings screen, single sign-on tab. In preparation of an upcoming release of an LDAP adapter for single sign-on, some small visual changes to the settings have been made.
SQLite version updated SQLite updated to version 3.30.1 to match Edify
Added HTTP redirect limit option in USE Added new option http_redirect_count that allows to limit number of HTTP redirects to follow, or disable following redirects (default) when building an Extractor
Added function in Transcript to capture parts of a string value
Bug fixes
Fixed an issue where some screens could display an Insufficient rights error. This was caused by API calls for which the currently logged in user didn't have access to. This has been fixed by not exposing this part of the functionality in the GUI.
Fixed an edge-case where the GUI displays a blank screen after an upgrade. In some circumstances, after an upgrade, the GUI would start with a blank screen. It was possible to access the GUI by refreshing the browser window. This is no longer needed.
Fixed an edge-case where removing a workflow could lead to an error. Sometimes, when deleting workflows containing workflow steps - which in turn contained references to reports which were removed since creating the workflow - the API could not remove the workflow. This has been fixed.
Fixed an edge-case where the GUI could crash when fetching reports.
Fixed a bug where the GUI could crash in older browsers in the transformer screen.
Fixed a bug where a license warning was shown even when there were no problem with the license.
Added regex validation for metadata string fields. When adding an invalid regex, the user will now get an appropriate message when saving metadata definitions.
Fixed a bug which could prevent workflows from being updated. In some cases, when a user visited the Status tab in the workflows screen, it would prevent workflows from being updated (including adding/removing schedules and steps). This has been fixed.
Corrected a off by 1 problem in the Transformer preview Previously, the execution of the transformer script would stop at the currently selected line in the editor, while the interface suggested that execution would stop_after_the currently selected line. This has now been corrected so that execution will include the currently selected line.
Fixed a bug where the list of columns was not updating when creating a new report.
Fixed an issue where Extractor arguments where truncated when 0 was one of the arguments.
Fixed an issue where the Run Now button on the workflow screen could be greyed out in some cases.
The service worker script in the front-end no longer relies on a third-party content delivery network (CDN).
Fixed an issue where API requests were not forwarded correctly on hosts with a web proxy setup. This was particularly a problem with requests to the API not invoked from within the GUI, e.g. when loading SAML2 endpoints on the API.
Solved a security issue (internal reference: EXVT-2812)
Fixed an issue whereby DST changes could affect certain operations
Exivity makes frequent use of date ranges for operations such as preparing and running reports. We have identified, and fixed, an issue whereby sometimes when a DST change resulted in the clocks going back an hour, the day in question would be treated as two separate days.
October 29, 2019
New features
Ability to rename a service category in the GUI In is now possible to manually rename a service category in the Glass interface
Notification are now out of beta The configuration of Workflow notifications is now GA
Ability in the GUI to create and delete and account It is now possible to manually create and delete accounts from a report definition by using the Accounts menu in the Glass interface
Ability to change the Unit label of a Service in the GUI It is now possible to manually change the Unit label of a Service in the Glass user interface
Improved Proximity error responses If any of the backend components returns a valid JSON error response, this will now be shown to the user.
API handling of requests when including relationship When Proximity handled a request to include relationships, it did not return a proper result for a relation that doesn't exists.
Removed empty arrays from the budget results Proximity now filters out empty arrays from the Horizon budget output
Add ability to specify a specific dataset while previewing the output from a transformer
Updating relationships through the API has been improved and is now more efficient in some edge case scenario's
support for option services = update When populating services in Transcript, it is now possible to update Unit label, Service description
Bug fixes
Fixed a Glass GUI overflow issue A GUI overflow in the details part of the account overview screen could occur. This has been resolved.
Proximity will enforce max account level Changed the code to now load the parent account and check the level before adding the relationship when creating new accounts in the GUI / API.
Dropdown expanding incorrectly to top In budget management screen, a drop down menu could previously expand to the upper region of the screen, while enough space was available downwards.
Resolved an issue with the Summary Report title When changing the Summary report title in the administration menu, changes were not reflected on the report. This has now been resolved.
Transformer margins and alignment Corrected visual margins and alignment in Transformer menu
Resolved 3 cosmetic issues for the Transformer Previewer Preview could go off-screen when moving the divider bar. The vertical scrollbar is now always visible. When not in full-screen, prevent having a vertical scrollbar in the editor as well as in the browser.
Removed the forward slash prefix from copied lookup file path
Fixed an issue where the summary report could show null when no header/footer were configured
Fixed an issue when importing CSV or Excel files in lookups
System wide date format configuration is now used for all summary report dates
Fixed an issue when removing a single service using bulk edit
Fixed an usability issue when moving back or forth multiple steps at once in the reports date range selector
Fixed an issue where Minimum commit line item was displayed instead of the user-configurable text
Added support for SAML2 when the API is behind a proxy server
Improved error reporting in the API for input validation errors
Proration is now applied correctly Solved an issue where monthly prorated service rates were applied incorrectly
Rate revision update issue Transcript will now create a new rate revision if needed, when using the serivces block in read only mode
September 18, 2019
Fixed a Glass GUI overflow issue A GUI overflow in the details part of the account overview screen could occur. This has been resolved.
Proximity will enforce max account level Changed the code to now load the parent account and check the level before adding the relationship when creating new accounts in the GUI / API.
Dropdown expanding incorrectly to top In budget management screen, a drop down menu could previously expand to the upper region of the screen, while enough space was available downwards.
Resolved an issue with the Summary Report title When changing the Summary report title in the administration menu, changes were not reflected on the report. This has now been resolved.
Transformer margins and alignment Corrected visual margins and alignment in Transformer menu
Resolved 3 cosmetic issues for the Transformer Previewer Preview could go off-screen when moving the divider bar. The vertical scrollbar is now always visible. When not in full-screen, prevent having a vertical scrollbar in the editor as well as in the browser.
Removed the forward slash prefix from copied lookup file path
Fixed an issue where the summary report could show null when no header/footer were configured
Fixed an issue when importing CSV or Excel files in lookups
System wide date format configuration is now used for all summary report dates
Fixed an issue when removing a single service using bulk edit
Fixed an usability issue when moving back or forth multiple steps at once in the reports date range selector
Fixed an issue where Minimum commit line item was displayed instead of the user-configurable text
Added support for SAML2 when the API is behind a proxy server
Improved error reporting in the API for input validation errors
Bug fixes
Fixed an issue with nested conditions in Transformers If a Transcript script opens a block of statements using 'if' or 'where' but does not have a closing brace, then if the transformer was run for multiple days it was possible to get an error stating that the maximum depth of nested statements had been reached. A check has now been implemented at the end of script execution which will verify that there are no unclosed statement blocks in effect. If there are then a meaningful error message will be logged and the task will fail.
Fixed a memory issue in USE A memory related corner case with certain Extractors could trigger an endless loop. This has now been resolved.
Removed error for future dates When reporting on a future Budget period start date , Horizon produced an invalid error message. This has now been resolved.
September 11, 2019
New features
Added header validation in the lookup editor
Added the ability to change mail server encryption It is now possible to select TLS, SSL or No mail encryption when configuring an e-mail server
Ability to disable SAML2 user creation Enabled an option to not automatically create new users in SAML2 configurations
Extractor/Transformer editor will wrap long lines The editor will now wrap very long lines in order to make them more readable.
Added Budget Viewer API Endpoint Ability in the Proximity API to call budget viewer.
Ability to Search for Service and Category Added Service Category as a searchable field in the Services Report. Also added Service as searchable field in Instance Report.
Added API suppot for Global Variables The Proximity API now has CRUD support for Global Variables. Support in Glass GUI, Extractors and Transformers will be added in a future release.
Solved an issue with component encoding The USE component-encode did previously not encode numeric values. This has now been resolved.
Added the ability to manually create services A user is now able to manually create one or multiple services in the Service Catalogue
If the logfile is not writable, terminate the Transcript task Transcript changed to fail if log file cannot be written, rather than falling back to using stderr
Adjustments are now included in service group subtotals Adjustments were previously ignored from service group subtotals. This behavior has now changed, so that adjustments are now included in both subtotal and total costs in the cost summary report
Backend support for previewing a custom DSET In Transcript preview mode, a user can select a non-default DSET to preview. GUI support will be added in future release
Added new table to database for Global Variables and Environments
Bug fixes
Transcript fails if service key exceeds allowed size
When adding services, if service key is longer then 127 characters, Transcript fails with descriptive error in log file
Improved diagnostics in Budget engine If there is no usage data to reporting period, Horizon returns more detailed diagnostic information
Fixed crash in case of invalid budget configuration Horizon was crashing when a budget revision contained no budget items. Issue has been resolved, and a more detailed error message has been added
Improved error diagnostics in Budget engine in case of a missing filter Horizon was producing incorrect error message when budget item referenced non-existent filter. Error message fixed to report specific probelm
Improved error handling when creating an RDF where 2 or columns have the same name with different cases Transcript will now analyse the column headings before creating an RDF and generates a meaningful log message when 2 or more column names would conflict before writing the daily RDF
Transcript fails if service key exceeds allowed size When adding services, if service key is longer then 127 characters, Transcript fails with descriptive error in log file
August 28, 2019
New features
Added endpoints to the API to create, update and delete service categories CRUD API endpoints are now available for both services and service categories.
Bug fixes
When encrypting variables, the encrypted result is now deterministic for any given system When encrypting variables, it was possible that for the same input value, different encrypted values would be generated in the script. This was harmless, but has now been fixed.
Buffer reset before HTTP retries There was an error were a buffer was not reset before retrying HTTP request. Therefore a buffer in some situations could contain the result of several tries, causing an invalid XML or JSON payload. This error was fixed by resetting the buffer before every retry.
Step output limited to 1MB Log from standard output of a Workflow step is limited now to 1 MB to minimize database polution
Added an automatic process to clean up excess workflow logs
The level parameter on the accounts API endpoint is now an integer instead of a string
Improved API error messages for validation errors
Added valid default value for service.charge_model
Newly added field charge_model contains NULL values for existing services. These are now migrated to peak (1) as that is the default service charging behavior.
Upgraded nginx to version 1.17.3
Upgraded the nginx web server from version 1.17.1 to version 1.17.3
August 08, 2019
Bug fixes
Fixed variable resolution after 'if' block
Transcript sometimes failed to resolve variables after skipped 'if' block.
Upgraded PHP to version 7.3.6
August 06, 2019
New features
Removed Github links in white labeled configurations Removed the github links for extractors and transformers when using a white labeled install
Removed default logo and icon for white labeled configurations Removed the standard Exivity logo's and icons when using a white labeled install
Updated disclaimer for white labeled configurations Disclaimer does not reference Exivity anymore when using a white labeled configuration
Extractor scripts can now use the HTTP PATCH method The 'http' statement now supports the HTTP PATCH method
Bug fixes
Fixed an issue with notification drivers
Pigeon could not find the notification drivers for Slack and SMS. This has now been fixed.
Fixed an invalid API error when resetting password Reset password could return an internal error (500), if a password didn't exist for the user. This was an invalid error, and has been changed to a valid 204 response.
July 31, 2019
Bug fixes
Fixed an issue where a log message could be incorrectly tagged as an error When creating services, an error message may be generated in the logfile which begins "services: set proration for service". This should be a debug level message and is not an error. This has now been fixed.
Fixed an issue where the error message generated by the append statement could be incorrect Fixed an issue whereby when appending one DSET to another, if the first DSET did not exist then the resulting error message in the log would state that it was the second DSET that doesn't exist.
Fixed an issue with embeds When the embed option is enabled, if the fields in the CSV to import were quoted then embed would not work as expected. This has now been fixed.
Don't resolve variables in 'else' branch being skipped Transcript doesn't resolve variables in 'else' branch when 'else' is not executed, therefore it removed 'unknown variable' errors in the situations when variables declared and used only inside 'else'
July 10, 2019
New features
Transcript script content is stored in RDF A copy of the ETL processing script used for generating data for any given day is now stored in each RDF alongside the processed data itself, thus enhancing support and diagnostic processes.
Support for auto retrying failed HTTP requests in USE If an HTTP request fails during the execution of an Extractor, the script can now be changed to set the option http_retry_count. This will determine the amount of times a HTTP request is retried. The default value is 2 retries.
Extractor scripts now have separate automatic variables for diagnosing HTTP related issues When connecting to a server over HTTP, the HTTP_STATUS_CODE variable will now always contain a numeric value. Any textual supplementary information pertaining to that value can be found in a new variable called HTTP_STATUS_TEXT. In the event that a timeout occurs and no HTTP response is received, the HTTP_STATUS_CODE variable will contain the value -1.
Bug fixes
June 21, 2019
New features
Service category totals are added to summary reports
Removed the 'Use local storage' configuration setting Now, the data in the interface is always synced with the server in order to better ensure data integrity.
When logging out, users are redirected to the login screen automatically Instead of showing an intermediate screen requiring a user to first click on a link to login again.
Streamlined the usergroup permissions to align with the new navigation structure
Implemented a new 'average' monthly charging model Monthly services may now be created which are charged based on the average quantity used throughout the month.
Deprecated the current budget feature in favour of a new implementation The new budget implementation will be released in Q2-2019.
Improved ordering of options in the side menu The menu on the left hand side of the interface has been overhauled in order to group the options in a more logical manner.
Now only privileged users will see an alert message when debugging is enabled
Manual account administration The Exivity back-end now supports manual creation, editing and deletion of accounts GUI support for this feature will be included in a future release
Added the option to modify or translate certain labels displayed by the Exivity interface. Find the new options by navigating to Administration > Settings > Translations.
Bug fixes
Workflow notifications status trigger Fixed an issue where a notifications was always send regardless whether the failed / successful condition was met
Workflow status log historical dates Fixed an issue where the status logs for a workflow could display incorrect timestamps
Fixed an issue which could lead to a crash in the GUI for certain edge-case adjustment configurations
Fixed a small issue which prevented empty metadata values to be stored
Do not send notifications for internal workflows Changed behavior of the Workflow engine to only send out notifications for user created workflows, and not for internal/garbage collector jobs
Updating Workflows during Workflow execution Fixed an issue were none of the workflows could be changed while a Workflow was running
Fixed a bug which prevented certain pages from showing correctly when no reports are defined
Fixed an issue whereby retrieving audit information from the API could fail
Fixed an issue with workflows containing 12 steps or more Fixed an issue where workflows with 12 or more steps could cause an API error.
Fixed an issue with newlines in scheduler log entries On occasion, it was possible that log entries generated by the scheduler would contain newline characters which could cause problems with the log viewer. This has now been fixed (log entries should no longer contain newlines).
Fixed an issue whereby credentials in a connection string could appear in the logfile When using a direct connection string with credentials to collect from an external database, under some circumstances an error message could contain a copy of the connection string. Log entries containing connection strings should no longer contain credentials.
Workflow management menu option is hidden from users without permission to manage workflows When logged in as a user who does not have permission to manage workflows, the GUI will hide workflow management options. Previously, users without the rights to do so were unable to perform any actual workflow management even though the option was displayed.
Fixed an issue that could cause workflows to stop working Fixed an issue whereby on rare occasions, the scheduler could leave its configuration database in a locked state, leading to problems running workflows.
Fixed an issue whereby the data status associated with a report would sometimes not be shown When accessing a report definition, sometimes the data status (the list of dates and the status of the data associated with each date) would not appear until the report definition was accessed a second time. This has now been fixed.
Fixed a bug which sometimes made it impossible to go into full-screen mode in the GUI
Fixed a bug which caused a warning to be incorrectly displayed when users tried to change their password
Fixed an issue whereby on rare occasions monthly services could have adjustments applied more than once Fixed an issue whereby if a monthly service had new instances appear on a date in the month after the date that the first instance of that service was seen, and if adjustments were applied to that service + instance combination then it was sometimes possible for the adjustments to be applied more than once.
Fixed an issue which could cause a GUI crash after deleting a service After deleting a service, it was possible that the GUI would display an error when subsequently viewing service rates. This has now been fixed.
Fixed a rare issue where importing into a transformation script could cause a crash When importing files using a wildcard, Transcript would crash if one of the files to import was empty and 'option embed' was enabled. This has now been fixed.
January 23, 2019
New features
Implemented user-definable timeout setting when retrieving data from HTTP sources
January 17, 2019
New features
Added editable labels for report levels
Added the ability to mass delete services
Invoice report is now called Summary report
The 'finish' statement in Transcript can be made to cause the task to fail if the DSET it's given is empty
Added the ability to exit a subscript invoked via #include in a transform script The 'return' statement, when used in a Transform script, will now cause script execution to resume from the statement following the #include statement in a parent script that referenced the script containing the 'return' statement.
Aggregation should now be significantly faster than it was previously
Bug fixes
Fixed a bug where users couldn't update their own details (including their password)
Fixed a bug where only 10 datasets were displayed when creating a new report
Fixed a bug where the mail sender name was not persisted in the configuration
Fixed an issue when importing CSV files containing quotes
When importing a CSV file with two successive quote characters at the end of a field, Transcript would reject the file as invalid. This has now been fixed.
Fixed an issue where deleting data let to GUI crashes on occasion When deleting data (RDFs) associated with a report, it could be that if one or more days had previously been overwritten, a stale database entry would cause issues after the RDFs were deleted. This has now been fixed.
December 13, 2018
Bug fixes
A bug was fixed which could lead to an error in the invoice report when using a rate with a minimum commit set
Fixed an issue with minimum commit It was possible that when applying minimum commit to a service, that other services would be affected by that minimum commit. This has now been fixed.
Fixed an issue when retrieving NULL fields from an ODBC query When using ODBC to collect data, the presence of NULL values in the results could cause USE to crash. This has been fixed.
November 30, 2018
New features
Added the ability to view instance level details on the invoice reports.
Added the ability to customize the report exports (CSV format only) field delimiter and decimal separator. These settings are system-wide and available to administrators by navigating to Administration > Configuration > Formatting.
Added the ability for users to reset their own passwords. This requires the email address of users to be set and a working server configuration for sending emails. This can be configured in Administration > System > Environment.
Logfiles generated from workflow tasks now include a timestamp. This prevents logfiles from consecutive runs of the same task from being overwritten.
Workflow status now automatically refreshes after a manual run.
Added a new Environment tab in Administration > System. In this tab, information about the system the Exivity instance is running on can be filled out. In the future this will be expanded to include more configuration options.
Invoice reports now include minimum commit uplifts as separate entries.
Carriage-returns and line-feeds in data extracted using ODBC are now replaced with spaces. When extracting data with USE, the presence of newlines in the data could cause corrupt CSV output. Carriage Return and Line Feed characters in data extracted from ODBC are therefore now replaced with spaces.
Enhanced expression support in the Extractor component. Conditional expressions have been enhanced in the Extractor component such that more complex conditions can be evaluated and additional operations can be performed. Additionally, it is now possible to set a variable value using an expression.
Services can now be manually deleted using the GUI.
Bug fixes
The datasets selector visible when creating a new report definition is now alphabetically sorted.
Fixed a bug which caused the contents of extractor editor to not update after updating variables. The contents of the extractor script itself was always saved after updating variables, only those changes were not visible in the editor.
Fixed a bug which caused the account depth selector to reset after performing an upgrade.
Fixed a bug which could cause the interface to become unresponsive after preparing a report.
Fixed an issue when running Transform scripts for days with 25 hours in them. When running a Transform script with a data-date representing a day where the clocks were adjusted such that the day had 25 hours in it, the script would be executed a second time automatically once the first had completed. This could lead to unexpected errors and log entries on occasion, and has now been fixed.
When writing to CSV files in USE, embedded CR/LF characters are converted to spaces. USE will now automatically strip out embedded carriage-return and line-feed characters when writing data to CSV files. Each unique occurrence of one or more sequential CR/LF characters will be replaced with a single space.
November 22, 2018
Bug fixes
Updated the documentation links in the header to point to our new documentation site.
Fixed grouping behaviour in the details table of the accounts report. In some cases, accounts could appear grouped under the wrong parent account in the 'Detailed' table in the accounts report.
October 31, 2018
Bug fixes
Fixed an issue with incorrect quantities sometimes showing on reports. Occasionally, when running a report for a range of dates, the quantities on one or more services differed from the quantity for that service shown when a report was run for a different date range (or just the day in question). This issue has now been fixed.
October 26, 2018
New features
The USE 'basename' statement can now write its results to a new variable. Previously, the 'basename' statement would always modify the value of the variable whose name was supplied as the argument. It can now also accept a literal string and create a new, or update an existing, variable to hold the result.
FIxed an issue whereby when running a Transform script the Audit database would be locked for the duration of the task. Transcript now only opens the Audit database when it needs to, reducing the likelihood of encountering errors pertaining to a locked audit database in the logfile.
A new system variable is now available containing the number of days in the current month. The existing dataMonth variable, which contains the yyyyMM of the current month is now supplemented with a new variable called dataMonthDays which contains the number of days in that month.
Changed default service type to 'automatic' in the 'services' statement in Transcript. When creating services, if no 'type' parameter is provided then the default service type will now be be set to 'automatic'.
Bug fixes
Fixed an issue whereby when creating a service, the audit indicated that the service creation failed. When a service definition is successfully created, Transcript will now correctly audit that event as opposed to indicating that the attempt failed.
Fixed an issue whereby over-writing services could result in database errors in the logfile. Sometimes when overwriting services, a constraint error would be logged in the logfile and the service would not have any rate associated with it. This has been fixed.
October 18, 2018
Bug fixes
Fixed an issue that could cause database corruption.
Fixed an issue that could cause database corruption due to the Aeon database being held open for long periods of time.
October 10, 2018
Added a live preview feature when working with transforms. A new feature has been added which can display a live preview of the transformer output. Note: this feature is currently in beta and will be further updated in the next release.
Charges for monthly services now take quantity into consideration as well as price. If two or more days in a month have the same highest price then the one with the highest quantity will be reported. Previously, the first seen was reported which could lead to discrepancies between the reported quantity and price on the report.
When running reports blank instance values are now displayed as a hyphen. When running reports against data with blank instance values in the usage data, the instance value will now be represented as a hyphen, which improves the aesthetics of the report.
Added hardware information to Transcript log-files. Log-files created by Transcript now contain information about the CPU and RAM at the top of the log.
Increased auditing information in Transcript. Events relating to service, rate and RDF changes are now audited
Bug fixes
Removed COGS option for users without rights to view COGS information. In the services and instances report, users with no access to view COGS will no longer be able to select the COGS type in the details table. Note: This bug never allowed users without appropriate access rights to view the actual COGS data.
Fixed a bug where the list of datasets on the report definition page was only showing the first 10 results. This could result in an inability to create new reports using datasets that were not included in those results
A link to the instances report has been added to the search feature in the header.
The service interval column in the instances report now contains data. Previously this column was always blank
Fixed a bug where searching for units within a services reports leads to a GUI crash.
Fixed an issue whereby very rarely a charge would not be included in reports. On very rare occasions, information in a record in the prepared report caches was not included in the output when a report was run. This has now been fixed.
Fixed an issue that could cause Aeon database corruption. Fixed an issue that could cause database corruption (and workflows to fail) due to the Aeon database being held open for long periods of time.
Fixed an issue whereby re-using an existing named buffer in USE for ODBC purposes could lead to unexpected results. Fixed an issue in USE whereby if an existing named buffer was re-used to store data retrieved from ODBC then a new buffer could have been created with the same name as the existing buffer, and attempts to reference it would return the old data.
Fixed an issue when executing ODBC queries that return no data. Using the ODBC capability to execute a query that returns no data will no longer cause an extractor to return an error.
September 05, 2018
New features
Bug fixes
Fixed an issue whereby 'append' could crash if one or other DSET was empty When executing the 'append' statement in a transformation script, if one or other of the DSETs involved in the operation was empty (having no data rows) then a crash could occur. This has now been fixed.
Fixed an issue where an expression that evaluated as FALSE could show the wrong line number in a log message The DEBUG level logfile entry indicating an expression is true or false would contain a reference to the wrong line number if the expression evaluated to false. This has now been fixed.
Fixed an issue whereby some comparisons would evaluate incorrectly in expressions Fixed and issue whereby in some cases where a value was quoted in an expression, the quotes would be considered part of the value itself.
Fixed a condition where reports were not showing for non-admins
Quantity metric is available again for the timeline chart on the services and instances reports
Reports in the navigation menu dropdown are now alphabetically ordered
August 28, 2018
New features
Ability to filter data in report using search query: The search bar in Accounts, Services and Instances reports now supports the use of operators (for example >and <) to filter your results based on column values or strings.
Add avg_unit_based_rate to report/run API endpoint Added the average per unit rate field to the report/run API endpoint and a placeholder for the average per interval rate which will be implemented later.
Bug fixes
Fixed an issue where deleting services could lead to adjustments not displaying correctly
Rate column in report details tables now use the configured rate precision setting
Fixed an issue whereby scheduled tasks that output more than 4kb of data to the console could suspend execution and do nothing until they timed out
August 22, 2018
New features
Transcript can now normalise scientific decimal numbers to standard format: When processing data that contains numbers in scientific format (such as 2.1E-5) the normalise statement can now be used to convert these to standard decimal notation (0.000021 in the above case) using the form normalise columncolNameas standardwhere colNameis the column containing the values to convert. Any values already in decimal will be unchanged, except that any trailing zeros will be removed from them. Non-numeric values will be converted to 0.
Support group and group_col as service parameters in Transcript: In the serviceand servicesstatements in Transcript, the parameters to define the service category are category and category_colThese parameters now have aliases of groupand group_col respectively, for those who prefer to use that terminology.
Bug fixes
The replace statement in Transcript will no longer behave unexpectedly when given an empty string as the target to replace: When using replace to update substrings within the values in a column, if the target string (the text to replace) is empty then Transcript will generate a meaningful log entry explaining that it cannot be used to replace empty strings, and will no longer overwrite non-blank column values with multiple copies of the replacement text.
The export statement in Transcript now supports backslashes as path delimiters: When specifying a relative path for the exportstatement, Transcript will automatically create any directories that do not exist in that path. Previously there was a bug whereby the auto-creation of those directories would only work if UNIX-style forward slashes were used as delimiters in the path. This has now been fixed and Windows or UNIX style delimiters may be used when specifying an export path.
Fixed a bug in the scheduler that could cause schedules to fail: In some cases schedules could fail for no obvious reason. This has now been fixed.
August 17, 2018
New features
USE scripts can now be forced to terminate with an error result Previously, the 'terminate' statement could be used to cancel script execution, but its use would always indicate that the script ran successfully. This may not be appropriate in all cases (for example if an error is detected by the script itself but ultimately cannot be resolved satisfactorily). The 'terminate' statement will still cause a script to exit with a success result by default, but may now be invoked as 'terminate with error' such that an error status is returned instead.
Added more service attributes as optional columns in the reports details table. The following extra service attributes can now be enabled as columns in the report details table: interval, charge type, cogs type and proration.
Support 'group' and 'groupcol' as service parameters in Transcript In the 'service' and 'services' statements in Transcript, the parameters to define the service category are 'category' and 'category_col'. These parameters now have aliases of 'group' and 'group_col' respectively, for those who prefer to use that terminology.
Reduced the chance of a 'database is locked' warning when preparing reports When preparing reports, on occasion it is possible for a warning to appear in the logfile pertaining to the global database being locked. When this warning happened, it could cause some days in the reporting period to remain unprepared. A known specific cause of this issue has been fixed, significantly reducing the likelihood of it happening.
Bug fixes
Fixed an issue where an ODBC connection could cause a crash in USE When executing an ODBC-based collection in USE, under certain circumstances an incorrect direct connection string could cause a crash. This has been fixed. Additionally, when an ODBC error occurs the error written to the logfile contains more detail than in previous releases.
The order of workflow steps in the status tab now corresponds to the order of workflow steps in the configuration tab.
An issue has been fixed where old user preferences could conflict by updates in the GUI, leading to errors when loading the service and instance reports.
An issue has been fixed where certain characters in a workflow status could lead to errors in the API. Sometimes, when running a scheduled task, the output written to the database contains non-printable characters. The API now re-encodes those characters, which means the GUI will now correctly show the status for those workflows.
When selecting a reporting period that spans multiple months, the charts will now only show a single label for each month.
Fixed a USE crash bug with certain combinations of conditional expressions Fixed an issue whereby if an expression with more than 2 parameters was followed later in the script by an expression with fewer parameters than the first, a crash would occur.
Fixed issue where an extractor could crash when using a parslet after formatting some JSON A bug has been fixed whereby if the 'json format' statement was used to prettify some JSON in a named buffer, use of a parslet to extract data from the JSON could cause a crash.
Fixed an issue where sometimes an XML parslet would cause an 'out of memory' error in USE When using an XML parslet, it was possible that an 'out of memory' error would be returned in the logfile and the script would fail, even on small input files. This has now been fixed.
August 03, 2018
Bug fixes
The 'export' statement in Transcript now supports backslashes as path delimiters
When specifying a relative path for the 'export' statement, Transcript will automatically create any directories that do not exist in that path. Previously there was a bug whereby the auto-creation of those directories would only work if UNIX-style forward slashes were used as delimiters in the path. This has now been fixed and Windows or UNIX style delimiters may be used when specifying an export path.
The 'replace' statement in Transcript will no longer behave unexpectedly when given an empty string as the target to replace
When using 'replace' to update substrings within the values in a column, if the target string (the characters to replace) is empty then Transcript will generate a meaningful log entry explaining that it cannot be used to replace empty strings, and will no longer overwrite non-blank column values with multiple copies of the replacement text.
July 25, 2018
New features
Increased default timeout when retrieving data from HTTP servers
Currently a USE script will fail if more than 3 minutes elapse without response when downloading data from an HTTP server. This has been increased to 5 minutes to cater for slow APIs.
July 19, 2018
New features
Transcript can now normalise scientific decimal numbers to standard format When processing data that contains numbers in scientific format (such as 2.1E-5) the 'normalise' statement can now be used to convert these to standard decimal notation (0.000021 in the above case) using the form 'normalise column colName as standard' where 'colName' is the column containing the values to convert. Any values already in decimal will be unchanged, except that any trailing zeros will be removed from them. Non-numeric values will be converted to 0.
When accessing the GUI via http visitors will be redirected to https automatically
Filter selectors show which items are present in the current report The service category selector in the services and instances report, and service selector in the instances report will show items not available in the current report grayed out.
On-demand workflow execution Workflows can now be executed on demand. Also the schedule for a Workflow can be disabled.
Single workflows can now have multiple schedules
Workflows can now be scheduled in a specific timezone
Added the average rate column to the reports details table
Added the ability to show various totals in reports summary widget Summary widget now has the option to show all totals (previous behaviour) or only the totals for the current search results, or for the current pinned items.
Added report shortcuts to the dashboard
COGS, fixed COGS and fixed prices are now evaluated per instance when preparing reports Previously, if a service was created that used any of fixed_price_col, cogs_col or fixed_cogs_col to indicate that the rate in question should be obtained from the usage data for any given day, then the charge engine would use a single value from the specified column(s) and apply that to all instances of the services for the day. Now, each row of usage is individually consulted when preparing reports such that the specific value on that row is used (as is already the case when using 'rate_col' for pass-through rates)
When extracting XML or JSON values, parse errors no longer cause the USE script to terminate Previously, when using a static parslet to extract XML or JSON values from the contents of a named buffer, if the buffer contained invalid JSON or XML then the USE script failed with an error in the log saying that the contents of the buffer could not be parsed. Now, if a named buffer contains data that is not valid JSON or XML, any attempt to extract a value from it using a static parslet will be expanded to the value EXIVITY_INVALID_XML or EXIVITY_INVALID_JSON.
Improved performance and lowered memory requirements when running a report Previously, in some circumstances running a report could take longer than expected and consume large amounts of memory in the process. The performance and memory use of the report engine have both been improved.
Reduced memory and increased performance when preparing reports Previously it was possible for some installations to use large amounts of memory and exhibit unreasonably slow performance when preparing reports. Preparing reports is not intended to be a realtime feature and will always incur some time overhead, but this time should now be significantly reduced in many cases, and the memory required to complete the process will be much less.
Free formatted ODBC connect strings are now supported in USE This exposes all ODBC driver options to the user, and avoid the requirement of creating manually DSN at the operating system level.
The 'split' statement now supports discarding unwanted result columns When using the 'split' statement it is now possible to discard all but a selected range of the resulting new columns.
Bug fixes
Fixed a Transcript crash when deleting a DSET Transcript will no longer crash in certain circumstances when deleting a DSET using the 'delete dset' statement.
The Transcript 'export' statement now creates a path automatically When exporting data from Transcript, if a relative path is specified as part of the export filename, Transcript will automatically create the path if it does not exist. The path will be created relative to /exported
Changed the behaviour of some columns in the report tables The optional _per unit charges and per interval charges columns on the report pages represent a fraction of the total charge and as such should be considered a subtotal rather than a rate._
Allow users to see anonymous roll-up accounts even if they have no access When a user only has access to some children of a parent account, reports will now show the combined usage of those accounts grouped as an unknown account in the reports.
Fixed a rare bug where incorrect character encoding in the data source could lead to reports not loading
Currency symbol is no longer shown for quantity graphs
An issue has been fixed which could lead to empty reports when there actually was report data In some cases, selecting certain combination of filters could lead to reports showing No data while there actually was report data for the current set of filters. This behaviour was observed mainly on the instances report page.
Usernames are now allowed to contain special characters As a side effect of changing usernames to be case-insensitive, using special characters was no longer permitted since v1.8.1. This restriction is now removed.
Changed the behaviour of clearing the charge engine caches Clearing the charge engine (Edify) caches unprepares all reports. The button on the About page now reflects this.
It is now possible to use decimal values for adjustment amounts Previously this was only possible through the API. The GUI has been updated to also support this.
Changing the date in the invoice report no longer resets the account selection Previously, when changing the date range on the invoice report screen, the current account selection (dropdown inside the invoice page) would automatically select the first account in the list. This has now been fixed to remember the selection when changing the date.
Exivity now works correctly when installed in a directory containing spaces
Transcript variables were not properly expanded when using in an import filter
Export of consolidated invoice now contains data for all accounts Previously, selecting the CSV or Excel export of a consolidated invoice would only export data for the first account on the invoice.
Fixed crash bug in the 'services' statement When creating services, Transcript will no longer crash if a blank interval or model value is encountered while building the service definitions.
June 06, 2018
New features
Minimum commit is now supported in the charge engine When generating a report, the results for any services that have a minimum commit value (and for which the usage does not meet that minimum commit quantity) will be adjusted to reflect that minimum commit value.
The charge engine now includes information about proration adjustments When applying proration to a monthly service, the charge engine will now include information in the raw report data which shows the amount that the unprorated charged was reduced by. This information will be used by the GUI in a future release.
Proration is now applied to monthly services where applicable Report results for monthly services that are flagged as being prorated will now reflect a percentage of the monthly charge, based on the number of days in the month that the service is used.
GUI preferences are now saved for each user For example, selected reports, date ranges and filters are now persisted for each user, so they can be restored after logging out and in again.
The charge engine can now execute a script passed to it via standard input The charge engine can now execute a reportfile passed to it via standard input. This internal change results in fewer termporary files on disk during normal use.
Error reporting can now be disabled in configuration
Transcript can now import usage data from existing RDFs The 'import' statement in Transcript can now retrieve the raw usage data from an existing RDF file.
Usernames are no longer case sensitive when loggin in
Transformers now always run with loglevel = warn when triggered in workflows
Service and service category filters now only show items actually in the visible report
USE will now trap more HTTP errors When enacting some HTTP operations, if an error such as a timeout or invalid host is encountered, USE will now return an error in the HTTP_STATUS_CODE variable instead of automatically terminating the script.
Added daily usage information for monthly services in the charge engine When generating a report, the charge engine will now include information about the usage quantity for each day in the charge interval. This information will be used by the GUI in a future release.
Drilldown functionality is now available from the legend in reports
Reference account information in ETL Account information can now be imported directly during the data transformation step, such that existing account data can be used to enrich the data being processed.
Increased HTTP client timeout USE will now wait for a three minutes by default before deciding that the connection has timed out if no data is received after the initial connection to a server has been made.
Improved the syntax for options to the 'import' statement in Transcript The options supported by the 'import' statement must now be formatted such that there is a single option per line of script. This removes the previous requirement to quote the list of column names when using 'select' and 'ignore', as well as the requirement to quote the expression used by the 'filter' option.
Added a system variable to return the last day of any given month A new system variable has been implemented which will return the last day in any given calendar month.
The 'correlate' transform now supports a default DSET for column names The 'correlate' statement always uses the default DSET as the destination for correlated columns but now supports an 'assume' parameter which determines the default DSET within which to locate non-fully-qualified source columns.
Added a button to the detailed widget in reports to toggle search field
Added an option to configuration to add a custom Google Analytics property
The charge engine can now be used to identify unused service definitions
The charge engine now supports the ability to retrieve a list of services which are not used by any existing reports.
Bug fixes
Fixed an issue where the depth filter wouldn't reload after preparing a report
Improved the print/PDF layout of consolidated invoices
Fixed a bug where the summary in Instance reports would sometimes remain empty
The charge engine now correctly deletes un-needed RDFs The charge engine now includes a mechanism to 'unload' historical data. This is an internal mechanism which will be used by the GUI in a future release.
Services for users with limited access to accounts are now filtered
When creating services, an instancecol parameter is now required _Previously it was possible to create services with no instance_col specified. This would result in missing data in reports if no instance_col was specified. Transcript now requires that an instance_col parameter is provided to the 'service' and 'services' statements.
Consolidated invoices can now be exported to PDF
Fixed an issue where in some circumstances the reports wouldn't load
It is now possible to view budget audit trails
The 'import' statement in Transcript now correctly imports usage data in all forms of the statement Fixed a bug whereby when using automatic source and alias tagging, the 'import' statement would not permit the importing of usage data from an existing RDF
Improved readability of text when a light background colour is chosen
USE will no longer reject some valid expressions In some cases, a valid expression in a script was rejected as having an unbalanced number of brackets. This has now been fixed.
The charge engine can now delete services associated with DSETs that are unused by any reports Fixed a bug where the charge engine would not correctly delete services if there were no RDF files for the DSET that the service is associated with.
The reset pins button has been moved to the top of the detailed widget in reports
May 03, 2018
New features
Implemented search field in report details table Ability to filter and pin a selection using a search query in the Accounts, Services and Instances report details table
Quantity adjustments can now be applied to a customer Adjustments can now also be set to affect quantities instead of charges. Both relative and absolute quantity adjustments are supported.
Ability to show consumed quantity in a report
Ability in transcript to convert number bases The following is now possible in a transcript: convert colName from dec|hex to dec|hex
It is now possible in the Invoice cost report to consolidate all child accounts on a single page
Added option to create workflow step which purges Proximity cache.
Beta version of budget manager & viewer is now available.
Bug fixes
When encrypting a variable it could get corrupted
Transcript could previously crash when running for a large date range
Workflows status tab did not consistently show historical log files
Fix for Invoice report error "Depth can't be empty, 0 or greater than 5"
April 13, 2018
Bug fixes
Extractor arguments where not used correctly when running USE script interactively from GUI
Report timeline graph could previously show zero when there's consumption
v1.6.1
April 13, 2018
New features
Add profile page where logged in users can change their own e-mail address and password.
Bug fixes
Fixed issue where scheduling multiple steps could corrupt Workflow WARNING: as of this release it is required to re-create your Workflows from scratch, to avoid potential issues
Fix loading overlays to improve multitasking in GUI
Fixed an upgrade bug which caused creating report definitions to be broken
Ability to specify to and from dates for transformers in workflows.
v1.6.0
April 8, 2018
Notable new features
March 26, 2018
Notable new features
March 19, 2018
Notable new features
March 16, 2018
Notable new features
February 23, 2018
Notable new features
February 23, 2018
Notable new features
A full changelog is available upon request.
February 09, 2018
Notable new features
A full changelog is available upon request.
February 03, 2018
Notable new features
A full changelog is available upon request.
February 02, 2018
Notable new features
A full changelog is available upon request.
January 12, 2018
Initial release.
Please ensure to read and understand all subjects that are mentioned here. Implement the suggested changes (where applicable) before upgrading to version 3 to avoid unexpected behavior.
Upgrading to a v3.x.x will require upgrading to version v2.10.x first. The installer will verify this and displays a warning when this requirement is not satisfied.
The biggest single change is the use of a new database engine powering all application state, audit logs and processed report data. Upgrading to this new database engine is transparent and the installer will take care of installing the database server as part of the regular installation process. After or during upgrade, it is possible to leverage an .
In version 2.x.x there were only two Exivity Services installed:
With Exivity version 3.0.0 up to version 3.4.3, assuming all components are installed on a single host system, there was a total of 4 different services:
As of version 3.5.0, the following Exivity services are to be considered when all components are deployed on a single host, there will be 7 different services:
In case you were using a service account in Exivity version 2.x.x for the Exivity Scheduling Service and the Exivity Web Service, you will have to reconfigure this service account for both services, as well as the Exivity API Service. In most cases, the Exivity Database Service may continue to run un the Local System account.
In case your current Exivity version 2.x.x installation runs inside an Active/Passive Windows Cluster, you will need to re-register the Cluster Roles for the Exivity Scheduling Service. Additionally, a new Cluster Role should be created for the Exivity Database Service, in case you decide to not use an external database host.
In v2.x.x, the default port for the Exivity GUI was 8001 and 8002 for the Proximity API. Both services were already available through port 443 (the default port used for https traffic, which means clients don't have to explicitly specify the port) and in v3.x.x this will be used by default:
As of Exivity version 3 more strict security settings are applied by default. These can be found under Administration > Settings. One important item which should be considered when upgrading a multi node environment, is the use of CORS. It is required to list all possible front end UI nodes in the CORS origins field:
Previously, it would only check for files in the system and exported folders within the Exivity home directory, and if a specified path + filename did not start with system/ or exported/ then these would be prepended automatically before the check as done.
This behaviour has been changed in the following ways in v3.x.x:
Any path + filename within the Exivity home directory can now be checked
path + filenames are now accepted and treated as being relative to the Exivity home directory
The folders system/ and exported/ are no longer automatically prepended
Consider a file somefile.csv in the %EXIVITY_HOME_PATH%/exported folder. Previously with version 2.x.x a user could check for the existence of this file using the following example:
In version 3 it is required to include the entire path relative to the %EXIVITY_HOME_PATH% :
This change may require modifications to existing Transformer scripts. This is because system and exported will no longer be automatically prepended.
The default behaviour of the HTTP subsystem in USE was changed to fully validate server SSL certificates, which may cause some USE scripts to fail. This typically applies to data Extractors, which are connecting to on premise data sources that use self signed SSL certificated. In version 2.x.x, the default behavior was to ignore these certificate errors. You can identify these errors in your Extractor logs:
The above will apply to all of your Data Extractors where you are connecting to (most likely internal) data sources which are using self signed certificates. Make sure to apply this change before upgrading to v3 to avoid any data extraction errors.
There are some minor changes in the GUI which are not backwards-compatible:
We removed the Excel export options from the reports. They were using the CSV format under the hood (i.e. they never actually produced valid Excel worksheets). In the future, we plan to implement proper Excel export formats for the reports, including a full summary report Excel export.
Due to changes to the processing of reports, when making changes to either services, rates or adjustments, associated reports should be prepared. In v2.x.x this was already required when making changes to services or rates. Since v3.x.x this will be also required when making changes to adjustments. We've planned further improvements making this more transparent (i.e. handling preparation of reports automatically in the background).
Some endpoints were returning dates and timestamps in different formats. This has been normalised in such a way that all responses use the same serialisation for dates and timestamps:
A date is always represented as ISO-8601 string: "yyyy-mm-dd", e.g. "2020-01-29"
A date/time is always represented as ISO-8601 string in UTC: "yyyy-mm-ddThh:mm:ssZ", e.g. "2020-01-29T11:26:52Z". Note that the Z suffix denotes the UTC time standard.
This affects the responses (attributes) for the following group of API endpoints:
/v1/audit
created_at
/v1/budgetrevisions
effective_from
/v1/dsets
earliest_rdf
latest_rdf
rdf_detail.created
rdf_detail.updated
/v1/extractors
last_modified
/v1/log
created
lines.date
/v1/reports
data_status.first_date
data_status.last_date
data_status.status.date
/v1/services
created_at
updated_at
/v1/transformers
last_modified
/v1/workflows
created_at
updated_at
/v1/workflowschedules
start_time
next_run
/v1/reports/{id}/run endpointThe JSON format in the response is simplified. See examples below.
This is an example response from v2.x.x:
This is an example response from v3.x.x:
/v1/usergroups endpointRemoved deprecated permission aliases.
upload_files (use manage_files instead)
manage_configuration (use manage_settings instead)
manage_system (use manage_settings instead)
/v1/reports/{id}/run endpointFunctionality deprecated in v2.x.x has been removed. If you were relying on any of the following functionality, please use the suggested replacement instead:
The pdf/invoice option for the format query parameter has been removed. Please use pdf/summary instead.
The invoice_options query parameter has been removed in favour of summary_options.
/v1/configuration endpointConfiguration keys prefixed with INVOICE_ are replaced by respective keys prefixed with SUMMARY_.
/v1/workflowsteplogs endpointAlso relevant to other endpoints including workflowsteplogs .
Removed the last_log attribute. Include the last log by specifying the query parameters include=steplogs&related[steplogs][limit]=1&related[steplogs][sort]=-start_ts.
Removed the timestamp attribute. Use start_timestamp instead.
Removed the message attribute. Use a combination of status and output instead.
/v1/file endpointOnly applicable to POST requests to this endpoint.
The filename in the response from this endpoint will no longer include the import/ prefix to better align for other requests in this endpoint. See example below.
/v1/usergroups endpointThe following permissions has been removed in favour of their new counterparts:
UPLOAD_FILES has become MANAGE_FILES
MANAGE_CONFIGURATION has become MANAGE_SETTINGS
MANAGE_SYSTEM has become MANAGE_SETTINGS
Check for key column presence before correlation The function in Transformers now checks for the existence of the key column in the default DSET. Previously, if the column was missing this resulted in an SQL error being logged. In such cases a clearer error message is now generated.
When building a Transformer, it is now possible to obtain values from a cell by using the @EXTRACT_BEFORE and @EXTRACT_AFTER functions. For more information visit
Added the ability to create and view budgets It is now possible to create and report on multi-level budgets. More information on this feature can be found at
Added the ability to Manage and View Budgets It is now possible to create and report on multi-level budgets. More information on this feature can be found at
Added a warning when using Etc timezones When a timezone in a Workflow is set to Etc, the user will receive a warning to make them aware of the Etc timezone behavior. To learn more, please consult the following article: )
If your Exivity installation connects to the Internet thru the use of an Internet Proxy, you will need to ensure that a number of system variables are in place according to before upgrading to version 2.4.7 or higher
New import option to include the name of the imported file(s) It is now possible in a Transformer to automatically add a column to each dataset, which will contain the name of the imported file(s). This can be achieved by enabling the filename_column = true option.
Continue when http get_header yields no results When retrieving the text of a header from an HTTP response in an Extractor using the feature, the Extractor script wil now continue to execute even if no header content was found.
Avoid overwriting of services because of charge_model introduction Fixed a bug where if the charge_model was changed, the update would not be reflected after executing the Transformer script. Now, when using in a script, services will be recreated if the charge model changes.
When creating services sometimes a rate revision would be created when there was no need to do so Under certain circumstances, when updating the service definitions via the statement in Transcript a new revision could be created which was a duplicate of the existing revision. This has now been fixed.
Fixed an issue when aggregating a DSET that is not the default DSET The statement may have failed to correctly aggregate a DSET if the DSET ID specified in the statement was not the default DSET. This has now been fixed.
Created new automatic variable UNIX_UTC in USE A new is now available in extractor scripts. This is called UNIX_UTC and will return the current UTC time as a UNIX timestamp value.
Added an option to Transcript to skip invalid records during import It is now possible to skip invalid records during the phase of a Transformer.
Added lookup editor for ad-hoc data sources Lookup data sources can be used for various types of data sources not obtainable through automated extractors. Edit lookups by navigating to Data pipelines > Lookups. Read more at .
Implemented support for ZIP files that expand to more than 2Gb
Fixed nested foreach loops Fixed issues where a nested loop could cause issues with some XML
The 'export' statement will no longer generate an error if asked to process an empty DSET while is set
When retrieving data from HTTP sources, the number of seconds to wait for a server response before timing out can be defined using .
Previously, when creating an RDF, the statement would perform no action if the DSET to create the RDF from was empty. This is still the case if ' mode = permissive' is in force, but if ' mode = strict' then the statement will now generate an error. The error will cause the task to fail for the current day, and if ' continue' is not enabled, the task will be terminated, else the task will move onto the next date in the range of days being processed.
When importing multiple files using the option to , any invalid files will be skipped When multiple files matching a pattern are imported, any of those files that are malformed or otherwise non-importable will be skipped.
Increased the performance of the statement Correlation should now be significantly faster than it was previously
Increased the performance of the statement
Fixed a bug whereby using within the body of an statement in a Transform script could cause an error Invoking in an block when running a transform script against a range of dates could cause the error The maximum number of nested blocks (32) is already in use. This has now been fixed.
Fixed a bug whereby the body of an 'if' statement in the Transformer could terminate prematurely. In some cases, using an '' statement with an 'options' block within the body of an '' statement could cause statements following the 'import' to be skipped. This has now been fixed.
Archives in GZIP format can now be decompressed using USE. USE now supports the 'gunzip' statement which can be used to inflate GZIP'd data. Details of how to use this statement may be found at
The code editor has been updated. The code editor for Extractor and Transformer scripts has been updated (it now uses the open source Monaco editor - ) resulting in a significant improvement over our previous editor. This greatly enhances the user experience when editing scripts in the GUI. Note: This change also lays the foundation for more advanced features going forwards.
Upgraded the underlying API framework For more information, please refer to the .
Progress indicator in the report/run endpoint can be disabled To disable, set the progress parameter to 0. More information at our .
New output format for the /report/run endpoint in the API Due to changes to the charge engine, the output format of the /report/run endpoint in the API has changed. An up-to-date overview of the attributes returned by this endpoint can be found at our .
Updates to internal service and rate schema The rate attributes min_commit and threshold are now implemented and the API will return a slightly different schema for the /v1/services endpoint (and related /v1/dump models) - the rate_type attribute is now called charge_type. More information can be found at
[] - Implement day and month name variables in USE
[] - Add option to enable.disable client certificate support in USE
[] - Scheduler is now called workflows
[] - When using COLNAME_NOT_EXISTS in a filter, it always evaluates to 'TRUE'
[] - Internal Error when applying filters in a 'where' statement if import options are used
[] - Garbage collector
[] - Glass config file for default port/host
[] - Stacked bar chart option in accounts/services report, including optimized legend
[] - Extend `Run` tab in Transformer with `from` and `to` date
[] - Added Instance reports
[] - Make toggle so reports can go fullscreen
[] - Connect graphs + legend
[] - All lists in the front-end are now sorted alphabetically
[] - Ability to pin report items
[] - Add support for SAML Single Sign-On
[] - Add ability in transcript aggregate to average the values from a column
[] - Add ability to base new extractor on templates from GitHub repository
[] - Enhance the 'hash' statement in USE to support base-64 encoding of the result
[] - Fixed manually editing the value of an encrypted variable in USE can cause a crash
[] - Fixed Eterenity hourly schedule does not consider start date
[] - Fixed OSI_TIME_UTC variable is missing a trailing Z
[] - Fixed some accounts show slight discrepancies when comparing to Excel calculation
[] - Fixed radio buttons don't update when changing adjustments in Glass
[] - Fixed duplicate headings not always eliminated in filtered import in Transcript
[] - Fixed a Transcript crash on 'move rows' or 'delete' after a 'replace'.
[] - Added 'include' statement to Transcript
[] - Added UTC versions of time-related variables in USE
[] - The API can now render an invoice report as a native PDF document
[] - Ability to change the service description via the GUI
[] - Made updating of Extractor variables more robust, and added support for encrypted variables in the GUI
[] - Added the ability to use wildcard in import statement in USE
[] - Added a daterange wrapper for Transcript
[] - Fixed a corner case where USE can stop working when executed from Glass
[] - Add option to choose custom currency symbol
[] - Move report selector to sidebar
[] - Improved syntax highlighting for USE and Transcript in the Glass script editor
[] - Implement in Transcript
[] - Add additional checks to in Transcript
[] - Implement in Transcript
[] - Add escaping option to import statement in Transcript
[] - Added scheduler interface
[] - Add report depth breadcrumbs to reports
[] - Extractor log is now shown when running on-demand through GUI
[] - Fixed an issue which caused small discrepancies when using different reporting definitions
[] - Ability to extract data from databases using ODBC connection
[] - Scheduler endpoints in API
[] - Ability to schedule the preparation of report definitions through the GUI
[] - Enhanced conditional execution in Transcript with support for regex matching
[] - Fix for Cannot read property 'relationships' of undefined error when logging in as a user with limited account permissions.
[] - Syntax highlighting for USE
[] - Add support for XML data extraction in USE
[] - Enable parallel processing in Eternity
[] - Create USE script for reading AWS S3 bucket
[] - Extractor and Transformer execution must show last 25 lines of corresponding log file
[] - Perform cross-browser test and add warning in unsupported browsers.
[] - Improve orbit performance when syncing large amounts of records
[] - Select single days in datepicker
[] - Support in Eternity for hourly and monthly schedules
This is achieved by shipping a web proxy configuration for nginx, which routes all requests starting with /v1/ to port 8002 and all other requests to port 8001. The recommended configuration is to not expose port 8001 to the public and only accept incoming traffic on port 443. Port 8002 may still be opened to external hosts, typically in a configuration where the Web and API/Backend components are deployed . In such a scenario it is advisable to allow communication from the .
Multiple hosts including https:// may be added, while seperating each URL using a , (comma) symbol. Wildcards may also be used as part of the hostname to match multiple URL's in one go such as: https://*.cors.exivity.io. An overview of all current security policies can be found .
The and functions in Transcript have been modified in a manner which may require changes to scripts that use them.
It is highly recommended to use valid SSL certificates, properly signed by a trusted CA (Certificate Authority). However it is still possible to switch certificate validation off by specifying in an Extractor script before executing an HTTP request.
In version 2.x.x it was still possible to schedule an Extractor or Transformer from the editor screen. This feature is removed from version 3.x.x. In case you still have schedules which are configured thru this interface, you should Unschedule these and create an approperiate instead.
Removed functionality which would take a custom API port from the #port=xxx location hash parameter on the login screen. Specifying a custom API port (and hostname) is still possible by on your system.
