# option

## Overview

The `option` statement is used to set global parameters during the execution of a Transcript task.

## Syntax

**`option`***`option = setting`*

**`option noquote`**

## Details

The `option` statement can be used multiple times within the same task script and always takes immediate effect. It is therefore possible (for example) to import a CSV file delimited with commas and quoted with single quotes, change the options and then export it with semicolon delimiters and double quotes.

The supported options are as follows:

| Option                  | Default    | Notes                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| ----------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| *continue*              | `disabled` | Determines whether to proceed to the next day in a date range or bail out if an error occurs on any given day when processing a range of dates - see notes below.                                                                                                                                                                                                                                                                                    |
| *delimiter* *separator* | `,`        | Specifies the delimiter character between each field in a CSV file. This character will be used when importing and exporting Datasets. The default value is a comma. If a dot character is used, then the `option` statement will be ignored and a warning generated in the logfile. Either `delimiter` or `separator` can be used - they act in exactly the same way.                                                                               |
| *services*              | `readonly` | May be set to `readonly` ,`overwrite` or `update`.  This determines whether service definitions can be updated and, if so, which fields to update. Please refer to the notes in the next section for more detail.                                                                                                                                                                                                                                    |
| *mode*                  | `strict`   | May be set to `strict` or `permissive`. This option specifies whether or not to terminate a transcript task if an error is encountered - see notes below                                                                                                                                                                                                                                                                                             |
| *quote*                 | `"`        | Specifies the quote character used for quoting fields in a CSV file. This character is taken into consideration when importing and exporting Datasets. When importing a Dataset, any fields that begin and end with this character will have it removed during the import process. When exporting a Dataset, all non-blank fields will be quoted using this character. Note that when specifying a literal quote character, it must be escaped: `\"` |
| *noquote*               | *n/a*      | Specifies that no quoting is to be performed either when importing or exporting Datasets. A subsequent `option quote` statement will override this option.                                                                                                                                                                                                                                                                                           |
| *loglevel*              | `INFO`     | Sets the logging level - see notes below                                                                                                                                                                                                                                                                                                                                                                                                             |
| *logmode*               | `SAFE`     | Sets the logging mode - see notes below                                                                                                                                                                                                                                                                                                                                                                                                              |
| *overwrite*             | `yes`      | May be set to `yes`or `1`, `no` or `0`. If set to `no`then statements that update cell values will only affect blank cells. Refer to the documentation articles for any given statement for more information.                                                                                                                                                                                                                                        |
| *embed*                 | `no`       | May be set to `yes`or `1` , `no`or `0`. If enabled, then when importing a CSV file, any text between an opening curly brace - `{` - and its matching closing brace is not checked in any way. This permits separators and quotes to be included in the value. The curly brackets may be nested to any depth. If there is no matching `}` for the opening bracket then an error will be generated and the task will fail.                             |
| *merge\_blank*          | *(none)*   | The default value for [create mergedcolumn](https://github.com/exivity/docs/tree/60a265079e19e329e990b94f7836ea2024c5f214/transform/language/create/README.md#merging-column-values-to-create-a-new-column) to use if a source column is blank                                                                                                                                                                                                       |
| *merge\_nomatch*        | *(none)*   | The default value for [create mergedcolumn](https://github.com/exivity/docs/tree/60a265079e19e329e990b94f7836ea2024c5f214/transform/language/create/README.md#merging-column-values-to-create-a-new-column) to use if no match is found for a regular expression in a source column value                                                                                                                                                            |

{% hint style="warning" %}
When using options, there must be whitespace on each side of the `=` sign
{% endhint %}

## Additional notes

### Continue

`option continue = yes|enabled`

`option continue = no|disabled`

When executing a task file repeatedly against each day in a date range, by default Transcript will abort the whole run if a task failure occurs. In cases where this is undesirable, setting the *continue* option to *enabled* or *yes* (both work in exactly the same way) will change the behaviour such that if a task failure occurs then execution will resume with the next day in the range.

{% hint style="info" %}
When combining the *continue* option with `option mode = permissive` it is possible to process a range of dates for which usage or other data is not available, because the *mode* option will prevent a failed [import](https://olddocs.exivity.io/3.5.4/data-pipelines/transform/language/import) statement from being treated as a fatal error.
{% endhint %}

### Delimiter / Separator

When specifying a quote or tab as the separator it is necessary to escape it in order to prevent Transcript from interpreting it as a meaningful character during the parsing of the task script. For example:

```
option delimiter = \t # Specify a literal TAB character
option delimiter = \" # Specify a literal quote
```

### Services

This option may be specified as `readonly` , `overwrite` or `update` and influences the behaviour of the [service ](https://olddocs.exivity.io/3.5.4/data-pipelines/transform/language/broken-reference)and [services ](https://olddocs.exivity.io/3.5.4/data-pipelines/transform/language/services)statements as follows:

| Value       | Notes                                                                                                                                                                          |
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `readonly`  | Existing service definitions which have keys matching those in the data being processed will not be overwritten with new versions (though new services may be created).        |
| `overwrite` | Existing services which have keys matching those in the data being processed will be overwritten.                                                                              |
| `update`    | Existing services which have keys matching those in the data being processed will have their *Description* and *Unit Label* fields updated, but no other changes will be made. |

{% hint style="danger" %}
**Any services that are overwritten  through use of `option overwrite`will lose any custom rates associated with them and a single default rate will be created instead.**
{% endhint %}

{% hint style="success" %}
In all cases, rate revisions may be created depending on the data being processed, but existing global rates will not be replaced with new ones if the *effective date* of the updated service matches that of an existing global revision.
{% endhint %}

### Execution mode

```
option mode = strict
option mode = permissive
```

Transcript supports two modes of execution for tasks:

* In `strict` mode, if an error is encountered at any point in the task, the error will be logged and execution will terminate
* In `permissive` mode, many errors that would otherwise have caused the task to fail will be logged, the statement that caused the error will be skipped and execution will continue from the next statement in the task.

{% hint style="info" %}
The *mode* option can be used multiple times and changed at any point during task execution. This means that specific sections of the task can be more error tolerant.
{% endhint %}

Errors that can be handled in `permissive` mode are mainly syntax errors or those involving invalid parameters to Transcript statements. There are error conditions that can arise during the execution of a statement will cause the task to fail even in `permissive` mode.

### Log levels

Transcript generates a considerable amount of logging information during the execution of a task. The `loglevel` option can be used to increase or decrease the level of detail written to the logfile. All logging levels must be specified in UPPER CASE. The following levels can be set:

| Level      | Details                                                                                              |
| ---------- | ---------------------------------------------------------------------------------------------------- |
| `INTERNAL` | Self-diagnostic messages indicating an internal error detected with Transcript itself                |
| `FATAL`    | Non-recoverable errors, usually as a result of an Operating System error such as no available memory |
| `ERROR`    | A transcript task error (syntax or data related)                                                     |
| `WARN`     | An unexpected event or a syntax error that can be recovered from                                     |
| `INFO`     | Informational messages about actions performed during the execution of a transcript task             |
| `DEBUG`    | Detailed logs of actions performed during the execution of a transcript task                         |
| `DEBUGX`   | Extended debugging information (may cause very large logfiles and a minor reduction in performance)  |

The order of the logging levels in the table above is significant, in that for any given level, all levels above it in the table are also in effect. Therefore a logging level of `WARN` will result in log entries for `ERROR`, `FATAL`, and `INTERNAL` level events, as well as warnings.

{% hint style="info" %}
The loglevel option can appear multiple times within a transcript task and will take immediate effect whenever it is used. This means that within a task, the loglevel can be increased for certain statements and reduced for others.
{% endhint %}

Regardless of the logging level, some events will always create a logfile entry, for example the success or failure of a transcript task at the end of its execution.

### Log mode

In order to minimise the effect on performance when logging, Transcript opens the logfile when it is first run and then holds it open until the completion of the task being executed. The logfile can be accessed in one of two modes:

| Mode   | Details                                                                              |
| ------ | ------------------------------------------------------------------------------------ |
| `SAFE` | After every message, the logfile is flushed to disk                                  |
| `FAST` | The logfile is not explicitly flushed to disk until the termination of the task file |

The default is `SAFE`. It is not recommended that this be changed.

## Examples

The following Transcript task will import a CSV file quoted with double quotes and delimited with commas and then export a copy with semicolons as delimiters and quoted with single quotes:

It also increases the logging level for the import statement

```
option quote = \"
option loglevel = DEBUGX
import usage from Azure
option loglevel = INFO
option separator = ;
option quote = '
export azure.Usage as c:\transcript\exported\azure_modified.csv
```
