中央社新聞(CNA news)

• Establish Subdirectory
  • Program (for perl programs)
  • SrcData (for test data)
    • CNA9101_Temp.txt
    • CNA9101
    • CNA9102
    • CNA9103
• Split news articles.
  • >perl 1_1CNAClassStatistics.pl
    • input:CNA9101_Temp.txt
• Extract the reporter from a news article.
  • >perl 1_2CNAClassStatistics_Reporter.pl
    • input:CNA9101_Temp.txt
• Extract the date of one news article.
  • ../Program/jdwang_lib/jdwangPackage.pm
  • program> perl 1_3CNAClassStatistics_Date.pl
    • input:CNA9101_Temp.txt
• Extract the types of one news article.
  • > perl 1_4CNAClassStatistics_NewTypes.pl
• Extract all information of one news article.
  • >perl 1_5CNAClassStatistics_SplitsNews.pl

=======================PerlCNAParseKgram_jdwang2010_3_30.zip===============================

• How to read filenames from a given directory
  • > perl 2_1CNANewStatistics.pl
    • input:../SrcData/CNA9101_Temp.txt
• Parse the news article files under one directory "../SrcData"
  • >perl 2_2CNAParseNews.pl
  • input:>../SrcData/CNA9101_Temp.txt
  • output:>../ParsedData/CNA9101_Temp.txt_Parsed.txt

==================================================================

1. Check the ParsedData
   • >perl 2_3CNANewStatistics_SplitOneNews.pl
   • input:>../ParsedData/CNA9101_Temp.txt_Parsed.txt
   • output:>none
2. Check the ParsedData (Sentences)
   • >perl 2_4CNANewStatistics_OneNewsInfo.pl
   • input:>../ParsedData/CNA9101_Temp.txt_Parsed.txt
   • output:>none
3. Check the ParsedData (Sentences Length)
   • >perl 2_5CNANewStatistics_SplitOneNews.pl
   • input:>../ParsedData/CNA9101_Temp.txt_Parsed.txt
   • output:>(Sentences Length)
4. The statistics of Chinese characters(1-gram)(one sentence)
   • >perl 2_6CNANewStatistics_SplitOneNews.pl
   • input:>../ParsedData/CNA9101_Temp.txt_Parsed.txt
   • output:>character count
5. The statistics of Chinese characters (k-mers, k-grams)(one news article)
   • >perl 2_7CNANewStatistics_SplitOneNews.pl
   • input:>../ParsedData/CNA9101_Temp.txt_Parsed.txt
   • output:>
6. The statistics of Chinese characters (k-mers, k-grams)=> outputfile
   • >perl 2_8CNANewStatistics_SplitOneNews.pl
   • input:>../ParsedData/CNA9101_Temp.txt_Parsed.txt
   • output:>../StatisticsData/CNA_CharStatistic.txt
7. The statistics of Chinese characters (k-mers, k-grams)=> outputfile(+tf distribution)
   • >perl 2_9CNANewStatistics_SplitOneNews.pl
   • input:>../ParsedData/CNA9101_Temp.txt_Parsed.txt
   • output:>"../StatisticsData/CNA_CharStatistic.txt"
8. The statistics of Chinese characters (k-mers, k-grams)=> outputfile(tf_df)
   • >perl 2_10CNANewStatistics_SplitOneNews.pl
   • input:>../ParsedData/CNA9101_Temp.txt_Parsed.txt
   • output:>"../StatisticsData/TF_DF_IDF_Statistic.txt";

==================================================================

1. perl Creat_MySQLTable_kgram_tf.pl (匯入CNA_CharStatistic.txt, TF_DF_IDF_Statistic.txt)