Language Model Action Set

This example is not available for the Lua programming language.

This section contains Python code. For more information about coding in Python, see Getting Started with SAS Viya for Python and SAS Viya: System Programming Guide.

The following code shows how you can use the nGramCount action to remove redundant n-grams and merge their counts :

                    casOut = {"name": "count_table"})

You need to modify values for the table and casOut parameters to fit the in-memory table of n-gram data and the output table, respectively, that you want to use.

The input of the nGramCount action is the output of the textToNgram action. An example of a table that contains n-gram data is illustrated in Figure 11.

The output of the nGramCount action still contains n-gram data, but the action removes redundant n-grams and merges their counts. An example of the output is shown in Figure 12.

Figure 12: Sample Output Table

Note that there will be no identical n-grams in the output table. The output table of the nGramCount action is also called the count table. It is used as the input table of the countToLm action.

This example is not available for the R programming language.