BOOLRULE Procedure

This example uses the same input table and the same TEXTMINE procedure call that are used in Example 6.1 to illustrate how you can match extracted rules in documents. Then it adds the DATA step to generate testing data. The DATA step and procedure call are repeated here for convenience.

The following DATA step creates the mylib.reviews data table, which contains nine observations that have four variables. The text variable contains the input reviews. The positive variable contains the sentiment of the reviews: a value of 1 indicates that the review is positive, and a value of 0 indicates that the review is negative. The category variable contains the category of the reviews. The did variable contains the ID of the documents. Each row in the data table represents a document for analysis.

data mylib.reviews;
   infile datalines delimiter='|' missover;
   length text $300 category $20;
   input text$ positive category$ did;
   datalines;
      This is the greatest phone ever! love it!|1|electronics|1
      The phone's battery life is too short and screen resolution is low.|0|electronics|2
      The screen resolution is low, but I love this tv.|1|electronics|3
      The movie itself is great and I like it, although the resolution is low.|1|movies|4
      The movie's story is boring and the acting is poor.|0|movies|5
      I watched this movie on tv, it's not good on a small screen. |0|movies|6
      watched the movie first and loved it, the book is even better!|1|books |7
      I like the story in this book, they should put it on screen.|1|books|8
      I love the author, but this book is a waste of time, don't buy it.|0|books|9
   ;
run;

The following DATA step generates the testing data, which contain two observations that have two variables. The text variable contains the input reviews. The did variable contains the ID of the documents. Each row in the data table represents a document for analysis.

data mylib.reviews_test;
   infile datalines delimiter='|' missover;
   length text $300;
   input text$ did;
   datalines;
      love it! a great phone, even better than advertised|1
      I like the book, GREATEST in this genre|2
   ;
run;

The following TEXTMINE procedure call parses the mylib.reviews data table, stores the term-by-document matrix in the mylib.reviews_bow data table in transactional format, and stores terms that appeared in the mylib.reviews data table in the mylib.reviews_terms data table:

 proc textmine data=mylib.reviews;
     doc_id
         did;
     var
         text;
     parse
         nonoungroups
         notagging
         entities    = none
         outparent   = mylib.reviews_bow
         outterms    = mylib.reviews_terms
         outconfig   = mylib.parseconfig
         reducef     = 1;
 run;

The following statements run PROC BOOLRULE to extract rules from the mylib.reviews_bow data table. TARGETTYPE=BINARY is specified. One target variable, positive, is specified; this variable indicates whether the reviews are positive or negative.

 proc boolrule
         data        =   mylib.reviews_bow
         docid       =   _document_
         termid      =   _termnum_
         docinfo     =   mylib.reviews
         terminfo    =   mylib.reviews_terms
         minsupports =   1
         mpos        =   1
         gpos        =   1;
     docinfo
         id          =   did
         targettype  =   binary
         targets     =   (positive);
     terminfo
         id          =   key
         label       =   term;
     output
         ruleterms   =   mylib.ruleterms
         rules       =   mylib.rules;
 run;

The TMSCORE procedure uses the parsing configuration that is stored in the mylib.parseconfig data table to parse the mylib.reviews_test data table. The term-by-document matrix is stored in the mylib.reviews_test_bow data table.

proc tmscore
   data       = mylib.reviews_test
   terms      = mylib.reviews_terms
   config     = mylib.parseconfig
   outparent  = mylib.reviews_test_bow;
   doc_id did;
   var text;
run;

The following statements run PROC BOOLRULE to match rules in the testing data and run PROC PRINT to show the matching results:

proc boolrule
   data        =   mylib.reviews_test_bow
   docid       =   _document_
   termid      =   _termnum_;
   score
      ruleterms = mylib.ruleterms
      outmatch  = mylib.match;
run;

proc print data=mylib.match; run;

The mylib.match data table in Output 6.4.1 shows which documents satisfy which rules.