TEXTMINE Procedure

The input data must be a table on your CAS server, and a CAS session must be set up. For more information, see the sections Using CAS Sessions and CAS Engine Librefs and Loading a SAS Data Set onto a CAS Server in Chapter 2, Shared Concepts.

The following DATA step creates the mylib.getstart data table in your CAS session. It contains 16 observations that have two variables: the text variable contains the input documents, and the did variable contains the IDs of the documents. Each row in the data table represents a document for analysis.

data mylib.getstart;
    infile datalines delimiter='|' missover;
    length text $150;
    input text$ did;
    datalines;
      Reduces the cost of maintenance. Improves revenue forecast.        | 1
      Analytics holds the key to unlocking big data.                     | 2
      The cost of updates between different environments is eliminated.  | 3
      Ensures easy deployment in the cloud or on-site.                   | 4
      Organizations are turning to SAS for business analytics.           | 5
      This removes concerns about maintenance and hidden costs.          | 6
      Service-oriented and cloud-ready for many cloud infrastructures.   | 7
      Easily apply machine learning and data mining techniques to data.  | 8
      SAS Viya will address data analysis, modeling and learning.        | 9
      Helps customers reduce cost and make better decisions faster.      | 10
      Simple, powerful architecture ensures easy deployment in the cloud.| 11
      SAS is helping industries glean insights from data.                | 12
      Solve complex business problems faster than ever.                  | 13
      Shatter the barriers associated with data volume with SAS Viya.    | 14
      Casual business users, data scientists and application developers. | 15
      Serves as the basis for innovation causing revenue growth.         | 16
      ;
run;

These statements assume that your libref is named mylib.

The following statements parse the input collection and use singular value decomposition followed by a rotation to discover topics that exist in the sample collection. The statements specify that all terms in the document collection, except for those on the stop list, are to be kept for generating the term-by-document matrix. The summary information about the terms in the document collection is stored in a data table named mylib.terms. The SVD statement requests that the first three singular values and singular vectors be computed. The topic assignments of the documents are stored in a data table named mylib.docpro, and the descriptive terms that define each topic are stored in a data table named mylib.topics.

proc textmine data=mylib.getstart;
doc_id      did;
variables   text;
parse
            outterms      = mylib.terms
            reducef       = 1;
svd
            k             = 3
            outdocpro     = mylib.docpro
            outtopics     = mylib.topics;
savestate   rstore        = mylib.aStoreTab;
run;

The output from this analysis is presented in Figure 1, Figure 2, and Figure 3.

The mylib.terms data table lists the discovered terms. The mylib.docpro data table contains four variables: the first variable is the document ID, and the remaining three variables are obtained by projecting the original document onto the three left-singular vectors that have been rotated using the default orthogonal (varimax) rotation. The mylib.topics data table has three variables that contain summary information about the discovered topics. Finally, the mylib.astoretab table contains a binary representation of a scoring model.

The following statements use PROC PRINT to show the contents of the first 10 rows of the sorted mylib.docpro data table that is generated by PROC TEXTMINE:

 data docpro;
    set mylib.docpro;
 run;
 proc sort data=docpro;
    by did;
 run;
 proc print data = docpro (obs=10);
 run;

Figure 1 shows the output of PROC PRINT. For information about the output of the OUTDOCPRO= option, see the section OUTDOCPRO= Data Table.

The following statements use a DATA step and PROC PRINT to show the contents of the mylib.topics data table that is generated by PROC TEXTMINE:

 data topics; set mylib.topics; run;
 proc print data = topics;
 run;

Figure 2 shows the output of PROC PRINT. The three discovered topics are listed together with four descriptive terms to characterize each topic.

The following statements use a DATA step and the SORT and PRINT procedures to show the first 10 observations of the mylib.terms data table that is generated by PROC TEXTMINE:

 data terms; set mylib.terms; run;
 proc sort data = terms; by key; run;
 proc print data = terms (obs=10);
 var term role freq numdocs key parent;
 run;

Figure 3 shows the output of PROC PRINT, which provides details about the terms that are identified by PROC TEXTMINE.

Only the values of the variables term, role, freq, numdocs, key, and parent are displayed. For information about the output of the OUTTERMS= option, see the section OUTTERMS= Data Table.

The following DATA step and statements create data and then score those data by using PROC ASTORE:

data mylib.scoreData;
    infile datalines delimiter='|' missover;
    length text $150;
    input text$ id;
    datalines;
      Deployment in the cloud or on-site.   | 1
      SAS for business analytics.           | 2
      Maintenance and hidden costs.         | 3
      ;
run;

 proc astore;
 score rstore=mylib.aStoreTab
       data=mylib.scoreData
       out= mylib.scoreResults
    copyVars= id;

 run;

 proc sort data=mylib.scoreResults out=scoreResults;
    by id;
 run;
 proc print data = scoreResults;
 run;

Figure 4 shows the output of PROC PRINT, which provides the topic score for the documents that are processed by PROC ASTORE.