DB_Table.tokenize_to_rows

tokenize_to_rowscolumnpatterncase_sensitivityat_least_one_row

Group: Conversions

Documentation

Tokenizes a column of text into a set of new rows using a regular expression. If the pattern contains marked groups, the values are concatenated together; otherwise the whole match is returned. The values of other columns are repeated for the new rows.

Arguments

• column: The name or index of the column to tokenize the text of.
• pattern: The pattern used to find within the text.
• case_sensitivity: Specifies if the text values should be compared case sensitively.
• at_least_one_row: If True, a tokenization that returns no values will still produce at least one row, with Nothing for the output column values. Equivalent to converting a tokenization output of [] to [Nothing].