DOCUMENTATION
Sign In

Get access to My SAS, trials, communities and more.

SAS Investigation Content Pack: Administrator’s Guide
 

Using Existing Data to Improve Concept Recognition

About Improving Concept Recognition

This section describes how you can configure a model to automatically identify concepts that are already part of your data set. You can schedule a job to run regularly to continuously improve the model, and you can then configure the Markup control or the Conversation Viewer control to use the new model.

Configure Model Builder

This topic describes the steps required to set up the Model Builder for the first time.

Note: The example configuration generates a model that identifies threat groups from the Threat Group entity, and names from the Person and Missing Person entities.
  1. Define the source data connections. For example:
    1. From the Applications menu (the Applications menu button), select Develop Code and Flows.
    2. Select Library Connections (the Library Connections button) from the navigation menu.
    3. Click Add (the Create a new connection button).

      The Select a Data Source window appears.

    4. Select the data source that your deployment uses to store internal SAS Visual Investigator data.
    5. On the Basic tab, create a new caslib called vidata that connects to the fdhdata schema. For example:
      • Connection name: vidata
      • Short name: vidata
      • Set the connection scope: Global
      • Database: SharedServices
      • Port number: 5432
      • Schema name: fdhdata
      • Server: host name of server
      • User name: dbmsowner
      • Password: dbmsowner password
    6. On the Access Policy tab, ensure that the data is available only to the required users.
    7. Click Save and connect.
  2. Create the configuration for the Model Builder:
    1. Sign in to SAS Studio.
    2. Open Products/SAS Intelligence & Investigation Management/Concept/Concept Model Configuration.sas.
      IMPORTANT The files in this folder are provided as examples only, and might be updated if you upgrade your system. It is recommended that you take a copy of the file and complete the following steps on the copied file.
    3. Update the configuration as required. For information about language interpretation and text interpretation rules, see SAS Text Analytics for Business Applications: Concept Rules for Information Extraction Models. For information about the concept compilation process, see your SAS Visual Text Analytics documentation.
    4. Run the code.

Schedule the Model Builder Job

This topic describes the steps required to set up the Model Builder job and schedule when it runs.

  1. Access the SAS Job Execution Web Application by using this URL: http://host:port/SASJobExecution
    Note: If you do not specify a value for port, the default value of 80 is used.
  2. Select Productsthen selectSAS Intelligence & Investigation Managementthen selectConcept Model Builder and click Submit.
  3. In the Work caslib field, enter the caslib used for working data.
  4. In the Configuration caslib field, enter the caslib that stores the configuration table.
  5. In the Configuration table field, enter the table that stores the configuration.
  6. In the Output caslib field, enter the caslib that stores the binary model table.
  7. In the Output table field, enter the table that stores the binary model.
  8. In the Language field, enter the language used by the concepts.
  9. Select Include predefined concepts if you want to include SAS predefined concepts.
  10. Click Submit.
  11. Check the statistics tables to review what has been generated.
  12. Review the log for any error messages.
  13. In SAS Job Execution, select Jobs from the navigation menu.
  14. Select the last successful run of Concept Model Builder.
  15. Click Schedule job.
  16. Set the job frequency as required.

Configure the Markup Control to Use the New Model

  1. In the Manage Investigate and Search interface, click Code Editor on the application toolbar.
  2. In the Solution Extensions list, double-click Markup Plug-in — Extract Concepts.

    The Markup plug-in code appears.

  3. Ensure that the MODEL_TABLE constant points to the correct caslib and table.
  4. Ensure that the VTA_CONCEPTS constant specifies the concept names from the configuration table.
  5. Ensure that the VTA_LANGUAGE constant specifies your preferred language.
  6. Click Update.
  7. In Page Builder, for each page that includes a Markup control:
    1. Open the page.
    2. Select the Markup control.
    3. In the Text group configuration area, add or edit Markup groups as required. Click An image showing the Edit a group button to edit existing Markup groups, or click An image showing the Add a group button to create a new Markup group. For each Markup group, ensure that the appropriate concept is selected in the Name field on the Text Group window.
    4. Click An image showing the Save Page button to save the updated page.

Configure the Conversation Viewer Control to Use the New Model

  1. In the Manage Investigate and Search interface, click Code Editor on the application toolbar.
  2. In the Solution Extensions list, double-click File Loader Plug-in — Conversation Loader.

    The plug-in code appears.

  3. In the ConversationFileLoaderDrugs class, set the targetConceptModel: 
    const targetConceptModel = "public.nlpModel";
  4. (Optional) Define categories to be displayed to users in the Conversation Viewer control. In the ConversationFileLoaderDrugs class, set the targetCategoryDefinitionFile value to your categories file. For example, to use the example categories file that is included with SAS Investigation Content Pack, specify the following:

    const targetCategoryDefinitionFile = "CategoriesDrugsModel.csv";

    For more information, see Define Categories for the Conversation Viewer Control and View Messages by Category in SAS Investigation Content Pack: User’s Guide.

  5. Click Update.

Define Categories for the Conversation Viewer Control

Categories that you define in the categories file are displayed on the Categories tab of the Conversation Viewer control. Users can filter messages based on their assigned category.

  1. Sign in to SAS Studio.
  2. Do one of the following:
    • To update the example categories file that is included with SAS Investigation Content Pack, open the CategoriesDrugs.csv file, which is located at Products/SAS Intelligence & Investigation Management/Conversation Viewer/Categories/CategoriesDrugs.csv.
      IMPORTANT This file is provided only as an example, and might be updated if you upgrade your system. It is recommended that you take a copy of the file and complete the following steps on the copied file.
    • To create a new set of categories, create a new CSV file based on the structure of the CategoriesDrugs.csv file in this location: Products/SAS Intelligence & Investigation Management/Conversation Viewer/Categories/.
      Note: If you create a new file, you must set the targetCategoryDefinitionFile value in the plug-in to the file that you created. For more information, see Configure the Conversation Viewer Control to Use the New Model.
  3. Edit the CSV file to update the categories as required or to create new categories.

    The first line of the file lists the columns for each category. Each subsequent line defines a single category. For example, here is the Drug References category:

    1:Drug References:(OR, "drug", "drugs", "cocaine", "coke", "heroin", "dealing", "smack")    1    100    high
    The Components of Each Category Entry

    Column Name

    Data Type

    Description

    config

    String

    This string has three colon-separated elements: a unique ID, a category name, and the list of concepts that are included in the category. For example: 

    1:Drug References:(OR, "drug", "drugs", "cocaine", "coke", "heroin", "dealing", "smack")

    Note: The category name is displayed on the Categories tab of the Conversation Viewer control.

    id

    Numeric

    A unique identifier for the category. (For example, 1.)

    score

    Numeric

    A score for the category. (For example, 100.)

    severity

    String

    A severity for the category. (For example, high.)

    For more information about categories, see Overview of the Categories Node in SAS Visual Text Analytics: User’s Guide.

Last updated: October 9, 2026