The TMSCORE Procedure

Note: Input data must be in a CAS table that is accessible in your CAS session. You must refer to this table by using a two-level name. The first level must be a CAS engine libref, and the second level must be the table name. For more information, see the sections Using CAS Sessions and CAS Engine Librefs and Loading a SAS Data Set onto a CAS Server in Chapter 2, Shared Concepts.

The following DATA steps generate two data tables: the mycas.getstart data table contains 36 observations, and the mycas.getstart_score data table contains 31 observations. Both data tables have two variables: the text variable contains the input documents, and the did variable contains the ID of the documents. Each row in each data table represents a "document" for analysis.

data mycas.getstart;
   infile datalines delimiter='|' missover;
   length text $150;
   input text$ did;
   datalines;
      High-performance analytics hold the key to |1
      unlocking the unprecedented business value of big data.|2
      Organizations looking for optimal ways to gain insights|3
      from big data in shorter reporting windows are turning to SAS.|4
      As the gold-standard leader in business analytics |5
      for more than 36 years,|6
      SAS frees enterprises from the limitations of |7
      traditional computing and enables them |8
      to draw instant benefits from big data.|9
      Faster Time to Insight.|10
      From banking to retail to health care to insurance, |11
      SAS is helping industries glean insights from data |12
      that once took days or weeks in just hours, minutes or seconds.|13
      It's all about getting to and analyzing relevant data faster.|14
      Revealing previously unseen patterns, sentiments and relationships.|15
      Identifying unknown risks.|16
      And speeding the time to insights.|17
      High-Performance Analytics from SAS Combining industry-leading |18
      analytics software with high-performance computing technologies|19
      produces fast and precise answers to unsolvable problems|20
      and enables our customers to gain greater competitive advantage.|21
      SAS In-Memory Analytics eliminate the need for disk-based processing|22
      allowing for much faster analysis.|23
      SAS In-Database executes analytic logic into the database itself |24
      for improved agility and governance.|25
      SAS Grid Computing creates a centrally managed,|26
      shared environment for processing large jobs|27
      and supporting a growing number of users efficiently.|28
      Together, the components of this integrated, |29
      supercharged platform are changing the decision-making landscape|30
      and redefining how the world solves big data business problems.|31
      Big data is a popular term used to describe the exponential growth,|32
      availability and use of information,|33
      both structured and unstructured.|34
      Much has been written on the big data trend and how it can |35
      serve as the basis for innovation, differentiation and growth.|36
run;

data mycas.getstart_score;
   infile datalines delimiter='|' missover;
   length text $150;
   input text$ did;
   datalines;
      Big data according to SAS|1
      At SAS, consider two other dimensions|2
      when thinking about big data:|3
      Variability. In addition to the|4
      increasing velocities and varieties of data, data|5
      flows can be highly inconsistent with periodic peaks.|6
      Is something big trending in the social media?|7
      Perhaps there is a high-profile IPO looming.|8
      Maybe swimming with pigs in the Bahamas is suddenly|9
      the must-do vacation activity. Daily, seasonal and|10
      event-triggered peak data loads can be challenging|11
      to manage - especially with social media involved.|12
      Complexity. When you deal with huge volumes of data,|13
      it comes from multiple sources. It is quite an|14
      undertaking to link, match, cleanse and|15
      transform data across systems. However,|16
      it is necessary to connect and correlate|17
      relationships, hierarchies and multiple data|18
      linkages or your data can quickly spiral out of|19
      control. Data governance can help you determine|20
      how disparate data relates to common definitions|21
      and how to systematically integrate structured|22
      and unstructured data assets to produce|23
      high-quality information that is useful,|24
      appropriate and up-to-date.|25
      Ultimately, regardless of the factors involved,|26
      I believe that the term big data is relative|27
      it applies (per Gartner's assessment)|28
      whenever an organization's ability|29
      to handle, store and analyze data|30
      exceeds its current capacity.|31
run;

The following statements use PROC TEXTMINE for processing the input text data table mycas.getstart and create three data tables (mycas.outconfig, mycas.outterms, and mycas.svdu), which can be used in PROC TMSCORE for scoring:

 proc textmine data    = mycas.getstart;
 doc_id      did;
 variables   text;
 parse
             outterms  = mycas.outterms
             outconfig = mycas.outconfig
             reducef   = 2;
 svd
             k         = 5
             svdu      = mycas.svdu;
 run;

The following statements then use PROC TMSCORE to score the input text data table mycas.getstart_score. The statements take the three data tables that are generated by PROC TEXTMINE as input and create a data table named mycas.docpro, which contains the projection of the documents in the input data table mycas.getstart_score.

 proc tmscore
             data      = mycas.getstart_score
             terms     = mycas.outterms
             config    = mycas.outconfig
             svdu      = mycas.svdu
             svddocpro = mycas.docpro;
 doc_id      did;
 variables   text;
 run;

The output from this analysis is presented in Figure 1.

The following statements use PROC PRINT to show the content of the first 10 rows of the sorted mycas.docpro data table, which is generated by the TMSCORE procedure:

 data docpro;
    set mycas.docpro;
 run;
 proc sort data=docpro;
 by did;
 run;
 proc print data = docpro (obs=10);
 run;

Figure 1 shows the output of PROC PRINT.