Editing Lists and Attributes

Editing a list or attribute is done by starting an unlang policy line with the & character. This character indicates that the following text should be interpreted as commands to edit lists and/or attributes.

Syntax

&attribute := <expression>
&attribute = <expression>
&attribute += <expression>
&attribute -= <expression>
...

&list1 := &list2
&list1 += { &attribute = value, ... }
&list1 += " attribute = value, ... "
...

In version 4, the update statement has been deprecated. Using update may still work, but will give warnings. In addition, the update sections are incapable of dealing with nested attributes, and will simply not work with them. We recommend switching to the new edit syntax, which is more powerful, and less verbose.

Unlike version 3, attribute editing can result in a fail return code. That is, edits either return noop on success, or fail on failure.

In most cases, an edit failure will result in the processing section exiting, and returning fail. This behavior can be overridden with a transaction, or a try / catch block.

An edit statement has the following standard syntax:

Syntax

<lhs> <op> <rhs>

Where we have:

<lhs>: An attribute reference, such as the name of a list or an attribute.
<op>: The operator such as =, :=, etc.
<rhs>: The value which is assigned to the attribute. This value can sometimes be an "in-place" list, in which case it is delimited by brackets, as in {…}.

The <rhs> may be value such as 192.0.2.1, or `5, or an expression such as 1 + 2, or "foo" + "bar".

Despite the edit statements having syntax similar to update sections, the meaning of the operators have changed significantly. You cannot convert an update section to the new syntax simply by removing the update keyword.

Atomic "Transactions" and Errors

The edit process is atomic, in that either all of the attributes are modified, or none of them are modified. If the edit fails for any reason, then all of the results are discarded, and the edit does not affect any attributes.

Note also that the server tracks overflows, underflows, and division by zero. These issues are caught, and cause the problematic calculation to fail.

For example, if an attribute is of type uint8, then it can only contain 8-bit integers. Any attempt to overflow the value to more than 255, or underflow it to lower than zero (0), or to divide by zero, will cause the edit operation to fail.

Nothing bad will happen to the server, it will just encounter a run-time failure while editing the attribute, and the edit operation will not succeed. All other packet processing will continue processing as normal.

In short, failed edit operations are effectively a "noop" operation, and do not result in any changes.

Grouping Edits

Multiple attributes may be grouped into a set by using the transaction keyword. When changes are done in a transaction, then either all of the changes are applied, or none of them are applied. This functionality is best used to conditionally apply attribute changes, generally when retrieving data from a database.

List Editing

Syntax

&<list> <op> <rhs>

List editing can be done for the usual lists such as request, reply, control, etc. However, as version 4 also supports "nested" and "grouped" attributes, list editing also applies to nested attributes.

The operation to be performed depends on the operator, as given in the table below. These operations are based on set theory, which is a mathematical system which gives consistency to the operations. All of the list operations are simple, and well defined.

Table 1. List Editing Operators
Operator	Description
=	Set the list to the contents of the <rhs>, if the <list> does not exist. If the list already exists, nothing is done. If the list does not exist, it is created, and the contents set to the value of the <rhs>
:=	Override the list to the contents with the <rhs>. If the list already exists, its value is over-written. If the list does not exist, it is created, and the contents set to the value of the <rhs>
+=	Perform list append operation. The contents of the <rhs> are appended to the <list>. The resulting list is <list><rhs>.
^=	Perform a list prepend operation. The contents of the <rhs> are prepended to the <list>. The resulting list is <rhs><list>.
-=	Remove attributes from the <list> which match the <rhs> attribute or list.
\|=	Perform a list union. The resulting list has all of the contents of the original <list>, and the <rhs> list.
&=	Perform a list intersection. The resulting list has only the attributes which are in both the original <list>, and the <rhs> list.
>=	Perform a priority merge of two lists. The resulting list has all of the contents of the original <list>, and of all <rhs> list attributes which are not already in <list>.
<=	Perform a priority merge of two lists. The resulting list has all of the contents of the original <rhs>, and of all <list> list attributes which are not in <rhs> are left alone..

Note that operations on lists are generally performed recursively on sub-lists. For example, the \|= operator will perform the union of not only two lists, but also will recurse to perform a union of any sub-lists.

The <rhs> can be a reference to another list (e.g. request, reply), or to a nested / grouped attribute.

The <rhs> can also be an "in-place" list, which has syntax similar to the update section. However, for "in-place" lists, the only operator which is permitted is =. The server will not start if any other operator is used.

If the right-hand side of an assignment is a list, then only operator allowed inside of the right-hand side list is =.

If the <rhs> is an "in-place" list, then all of the dynamic expansions are valid, just as are attribute references.

As a special case, the <rhs> can also be a string, or a dynamic expansion. If so, the string is interpreted as a set of attribute definitions, as if it was an "in-place" list. For example, "Filter-Id = foo"

This functionality is complex, so some examples should make this clearer.

Clearing a list

A lists contents can be removed by creating an empty list, and assigning the empty list to the destination.

Example 1. Clearing a list contents, or creating an empty list.

&reply := {}

In most other contexts, the empty list is ignored. i.e. Appending an empty list to request does nothing.

Adding an attribute to a list

Attributes (or lists of attributes) can be added using the += operator.

The following example appends the Filter-Id attribute to the tail of the reply list. Note again that the operator associated with the Filter-Id attribute is simply =.

This operation can best be understood as a two-step process:

Create a temporary "in-place" list from the <rhs> of the edit operation. This "in-place" list is not associated with any previous list, but instead exists on its own, independent of anything else. As such, there is no need to use operators for the <rhs> list. Instead, the attributes for this list are created in order, exactly as they are given.
Perform the += ("list append") operation, in which case the "in-place" list is appended to the reply list.

Example 2. Appending the Filter-Id attribute to the reply list

&reply += {
	&Filter-Id = "foo"
}

As a special case, where the right side is an attribute reference, it is possible to use +=. In that case, a copy of the referenced attribute is appended to the list.

Example 3. Appending the User-Name attribute from the request list, to the reply list.

&reply += &request.User-Name

Over-riding the contents of a list

The := (override) operator will delete the contents of a list. We note that the empty list example above is just a special case of overriding the contents of a list.

Example 4. Set the contents of the reply list to the Filter-Id attribute.

&reply := {
	&Filter-Id = "foo"
}

Aftet this operation, the contents of the reply list will be one attribute: Filter-Id.

Removing attributes from a list

Attributes can be removed from a list using the -= (remove) operator.

Example 5. Remove the first instance of Filter-Id from the reply list.

&reply -= &Filter-Id

Example 6. Remove all instances of Filter-Id from the reply list.

&reply -= &Filter-Id[*]

Example 7. Remove instance of Filter-Id which have value bar

&reply -= {
    &Filter-Id == "bar"
}

Multiple attributes can be specified in the <rhs> list. All attributes which match the comparison are removed.

This syntax is clearer and more consistent than the old !* ANY hacks.

Table 2. List Removal Operators
Operator	Description
==	attributes matching the value exactly
<	attributes having value less than the given one
<=	attributes having value less than or equal to the given one
>	attributes having value greater than the given one
>=	attributes having value greater than or equal to the given one

For now, regular expression operators are not supported.

List to List Operations

Lists can also be copied using the operators.

Remove all existing attributes in the reply list, and

copies all of the request list contents to the reply list.

&reply := &request

Example 8. Append the contents of the request list to the reply list.

&reply += &request

Parsing strings as lists

It is also possible to have strings on the <rhs> of a list assignment. This functionality is most useful for putting attribute lists into a database, and then reading them back when a request is processed.

Note that the pairs in the string cannot have list qualifiers. That is, &reply += "request.foo …" is not allowed.

Example 9. Assigning attributes taken from a string

&reply += "Filter-Id = 'foo'"

The above example has the same result as the earlier example of adding Filter-Id to the reply, using an "in-place" list.

Example 10. Append pairs read from SQL to the reply

&reply += "sql("SELECT pairs FROM pair_table WHERE username = '%{User-Name}'")

In this example, the pair_table could contain two columns: username and pairs. The pairs column could have free-form text strings, such as Filter-Id = "foo".

Attribute Editing

Syntax

&<attribute> <op> <rhs>

Attribute editing can be done for any attribute such as request.User-Name, etc. However, as version 4 also supports "nested" and "grouped" attributes, attribute editing also can be done for nested attributes.

The operation to be performed depends on the operator, as given in the table below. Unlike the list operations above, attribute operations change the attribute value.

Table 3. Attribute Editing Operators
Operator	Description
=	Set the attribute to the contents of the <rhs>, if the <attribute> does not exist. If the attribute already exists, nothing is done. If the attribute does not exist, it is created, and the contents set to the value of the <rhs>
:=	Delete all existing copies of the named attribute, and create a new attribute with the contents set to the value of the <rhs>
+=	Perform addition. The contents of the <rhs> are added to the value of the <attribute>.
-=	Perform subtraction. The contents of the <rhs> are subtracted from the value of the <attribute>.
*=	Perform multiplication. The value of the <attribute> is multiplied by the contents of the <rhs>.
/=	Perform division. The value of the <attribute> is divided by the contents of the <rhs>.
\|=	Perform logical "or". The value of the <attribute> is "or"ed with the contents of the <rhs>.
&=	Perform logical "and". The value of the <attribute> is "and"ed with the contents of the <rhs>.
<<=	Perform left shift. The value of the <attribute> is shifted left by the value of <rhs>
>>=	Perform right shift. The value of the <attribute> is shifted right by the value of <rhs>

There are also _filtering_operators. These operators ensure that the value of the attribute passes the filter. If the attribute being checked does not exist, it is created.

Table 4. Attribute Filtering Operators
Operator	Description
<	Ensure that the <lhs> attribute exists, and has value less than the <rhs>
⇐	Ensure that the <lhs> attribute exists, and has value less than or equal to the <rhs>
>	Ensure that the <lhs> attribute exists, and has value greater than the <rhs>
>=	Ensure that the <lhs> attribute exists, and has value greater than than or equal to the <rhs>

The <rhs> can be a reference to another attribute (e.g. request.Filter-Id). If the field is a double-quoted string, it undergoes dynamic expansion, and the resulting value is processed as described above.

In most cases, the edit operations "do the right thing". For example, adding a number to an ipv4prefix results in an ipv4addr data type. Similarly, subtracting two 'ipv4addr' data types results in a numerical value. Adding a time_delta or integer to a date will result in a date.

Similarly, assignments and/or modifications can be done across differing data types. Adding a uint8 value to an uint16 value will "just work", as the data types will be automatically converted before assignments or operations are done.

Operations on `string` and `octet` Data Types

The operators also apply to variable-sized values.

Table 5. Attribute Editing Operators for `string` and `octet`
Operator	Description
=	Set the attribute to the contents of the <rhs>, if the <attribute> does not exist. If the attribute already exists, nothing is done. If the attribute does not exist, it is created, and the contents set to the value of the <rhs>
:=	Override the attribute with the contents with the <rhs>. If the attribute already exists, its value is over-written. If the attribute does not exist, it is created, and the contents set to the value of the <rhs>
+=	Perform string append. The contents of the <rhs> are appended to the <attribute>.
-=	Inverse of string append. The contents of the <rhs> are deleted from from the <attribute>, if the `<rhs>` is a suffix of <attribute>
^=	For `string`, performs a "prepend" operation. The contents of the <rhs> are prepended to the <attribute>. This is the opposite of `+=`.
	For `octets`, perform logical "xor". The value of the <attribute> is "or"ed with the contents of the <rhs>. Both strings must be of the same length.
\|=	Perform logical "or". The value of the <attribute> is "or"ed with the contents of the <rhs>. Both strings must be of the same length.
&=	Perform logical "and". The value of the <attribute> is "and"ed with the contents of the <rhs>. Both strings must be of the same length.
<<=	Perform left shift / truncation. The first <rhs> bytes of <attribute> are dropped. i.e. shifted off of the start of the string.
>>=	Perform right shift / truncation. The last <rhs> bytes of <attribute> are dropped. i.e. shifted off of the end of the string.

Note that the ^= operator behaves differently for string and octets. The output of "xor"ing two strings is likely to be binary data, and therefore not a printable string. As a result, it is more useful for strings to have ^= be a "prepend" operation.

Editing Lists and Attributes

Atomic "Transactions" and Errors

Grouping Edits

List Editing

Clearing a list

Adding an attribute to a list

Over-riding the contents of a list

Removing attributes from a list

List to List Operations

Parsing strings as lists

Attribute Editing

Operations on string and octet Data Types

Operations on `string` and `octet` Data Types