TFIDF-LSI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据预处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "767"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stop_path = 'data/stopwords.txt'\n",
    "stopwords = open(stop_path).readlines()\n",
    "stopwords = [word.strip() for word in stopwords]\n",
    "len(stopwords)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_path = 'data/射雕英雄传.txt'\n",
    "docs = open(data_path).readlines()\n",
    "docs = [doc.strip() for doc in docs if doc.strip() != '']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8611 892286\n",
      "507 郭靖点头道：“对，幸亏大哥聪明，料到了这着，倘若是我，定是上了他们的大当。”周伯通摇头道：“说到聪明伶俐，天下又有谁及得上黄老邪的？只不知他用甚么法子，居然找到了一个跟他一般聪明的老婆。那时候黄家嫂子微微一笑，道：‘周大哥，你号称老顽童，人可不胡涂啊，你怕我刘备借荆州是不是？我就在这里坐着瞧瞧，看完了马上还你，也不用到天黑，你不放心，在旁边守着我就是。’“我听她这么说，就从怀里取出经书，递了给她。黄家嫂子接了，走到一株树下，坐在石上翻了起来。黄老邪见我神色之间总是有点提心吊胆，说道：‘老顽童，当世之间，有几个人的武功胜得过你我两人？’我道：‘胜得过你的未必有。胜过我的，连你在内，总有四五人罢！’黄老邪笑道：‘那你太捧我啦，东邪、西毒、南帝、北丐四个人，武功各有所长，谁也胜不了谁。欧阳锋既给你师哥破去了“蛤蟆功”’，那么十年之内，他是比兄弟要逊一筹的了。还有个铁掌水上飘裘千仞，听说武功也很了得，那次华山论剑他却没来，但他功夫再好，也未必真能出神入化。老顽童，你的武功兄弟决计不敢小看了，除了这几个人，武林中数到你是第一。咱俩联起手来，并世无人能敌。’我道：‘那自然！’黄老邪道：‘所以啊，你何必心神不定？\n"
     ]
    }
   ],
   "source": [
    "# 总行数\n",
    "print(len(docs), sum([len(doc) for doc in docs]))\n",
    "# 总字数\n",
    "print(len(max(docs, key=len)), max(docs, key=len))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Building prefix dict from the default dictionary ...\n",
      "Loading model from cache /tmp/jieba.cache\n",
      "Loading model cost 0.991 seconds.\n",
      "Prefix dict has been built succesfully.\n"
     ]
    }
   ],
   "source": [
    "import jieba\n",
    "\n",
    "all_docs = []\n",
    "for doc in docs:\n",
    "    result = []\n",
    "    words = jieba.cut(doc)\n",
    "    for word in words:\n",
    "        if word != '' and word not in stopwords:\n",
    "            result.append(word)\n",
    "    all_docs.append(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gensim import corpora, models, similarities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "45209"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 制作字典库\n",
    "dictionary = corpora.Dictionary(all_docs)\n",
    "len(dictionary.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dictionary.save('data/dictionary') # 保存\n",
    "# dictionary = Dictionary.load('data/dictionary') # 加载 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 制作语料库\n",
    "corpus = [dictionary.doc2bow(doc) for doc in all_docs]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# corpora.MmCorpus.serialize('data/corpuse', corpus) # 保存\n",
    "# corpus = corpora.MmCorpus('data/corpuse') # 加载"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['射雕', '英雄', '英雄传', '中', '武功', '天下', '第一', '天下第一']\n",
      "[(61, 1), (282, 1), (1081, 1), (2673, 1), (4402, 1), (11505, 1), (15710, 1)]\n"
     ]
    }
   ],
   "source": [
    "##### 测试数据 #####\n",
    "\n",
    "test = '射雕英雄传中谁的武功天下第一'\n",
    "\n",
    "# 结巴 搜索引擎模式\n",
    "test_list = [word for word in jieba.cut_for_search(test) if word not in stopwords]\n",
    "test_vec = dictionary.doc2bow(test_list)\n",
    "\n",
    "print(test_list)\n",
    "print(test_vec)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TF-IDF / LSI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "tfidf = models.TfidfModel(corpus)\n",
    "lsi = models.LsiModel(corpus)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "similarity = similarities.Similarity('Similarity-TFIDF-index',\n",
    "                                     corpus, \n",
    "                                     num_features=len(dictionary.keys()),\n",
    "                                     num_best=20)\n",
    "similarity_lsi = similarities.Similarity('Similarity-LSI-index',\n",
    "                                       lsi[corpus],\n",
    "                                       num_features=len(dictionary.keys()),\n",
    "                                       num_best=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 测试文档 TF-IDF 值\n",
    "test_tfidf = tfidf[test_vec]\n",
    "# 测试文档 LSI 值\n",
    "test_lsi = lsi[test_tfidf]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(61, 0.1475027481247322),\n",
       " (282, 0.31460561035969264),\n",
       " (1081, 0.20059470585041056),\n",
       " (2673, 0.4598651730371035),\n",
       " (4402, 0.36642297536463514),\n",
       " (11505, 0.5739969129055862),\n",
       " (15710, 0.40473915419438267)]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_tfidf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0, 0.038713925324015123),\n",
       " (1, 0.04320560776704137),\n",
       " (2, 0.000595628840861822),\n",
       " (3, 0.021252005349497054),\n",
       " (4, -0.028251906023248407),\n",
       " (5, -0.05157281475882069),\n",
       " (6, 0.009567302449942592),\n",
       " (7, -0.007077466651043111),\n",
       " (8, -0.04210656533991361),\n",
       " (9, 0.01322904068594456),\n",
       " (10, 0.006591622739835083),\n",
       " (11, 0.029318324204961884),\n",
       " (12, 0.02765832066690668),\n",
       " (13, 0.0018267262376269865),\n",
       " (14, 0.04735929602688122),\n",
       " (15, -0.07916252151959602),\n",
       " (16, 0.08826928374744426),\n",
       " (17, -0.041044933963931406),\n",
       " (18, 0.03964628237768773),\n",
       " (19, 0.002438125502848562),\n",
       " (20, -0.0963361341791512),\n",
       " (21, -0.022978682273933952),\n",
       " (22, 0.04379449641692116),\n",
       " (23, 0.014814967738253965),\n",
       " (24, 0.030875631282805557),\n",
       " (25, -0.06440440671418661),\n",
       " (26, -0.037580362745804036),\n",
       " (27, 0.020347650674062617),\n",
       " (28, -0.012496162745846048),\n",
       " (29, -0.02564470355439857),\n",
       " (30, -0.016951241988005978),\n",
       " (31, 0.035305869488492285),\n",
       " (32, -0.011382564180566875),\n",
       " (33, -0.029137473467409178),\n",
       " (34, 0.04286289739373995),\n",
       " (35, -0.03783389117337444),\n",
       " (36, -0.036621826418326585),\n",
       " (37, -0.00012125421335759234),\n",
       " (38, 0.005592350317132545),\n",
       " (39, 0.0023443828625811436),\n",
       " (40, -0.05766971368030456),\n",
       " (41, 0.0006608130267914993),\n",
       " (42, -0.01108894804196273),\n",
       " (43, -0.014871645122342493),\n",
       " (44, -0.00789730941827876),\n",
       " (45, 0.01174085854348966),\n",
       " (46, -0.003482284086520907),\n",
       " (47, -0.037445276229011414),\n",
       " (48, -0.02343577227697141),\n",
       " (49, 0.05694336288304955),\n",
       " (50, 0.02377814148329135),\n",
       " (51, -0.039826616509814776),\n",
       " (52, 0.006547148536945949),\n",
       " (53, 0.003795121264666277),\n",
       " (54, 0.0078771808620251),\n",
       " (55, 0.018955485550238904),\n",
       " (56, 0.009132696810338617),\n",
       " (57, -0.00030401473417795126),\n",
       " (58, -0.021926608341187443),\n",
       " (59, -0.015643259485118858),\n",
       " (60, -0.0038705885872031744),\n",
       " (61, -0.007634079010710794),\n",
       " (62, -0.024329700120059267),\n",
       " (63, 0.0017203938019606426),\n",
       " (64, 0.0017233068027551262),\n",
       " (65, -0.00569091136344943),\n",
       " (66, -0.005912487244140795),\n",
       " (67, -0.0032150692583779697),\n",
       " (68, -0.00932668871505565),\n",
       " (69, -0.006962242082395436),\n",
       " (70, -0.010492812736611595),\n",
       " (71, 0.004913815991955114),\n",
       " (72, -0.0056264097970208345),\n",
       " (73, -0.006601670046559575),\n",
       " (74, -0.006741739803463959),\n",
       " (75, 0.0023568280748667613),\n",
       " (76, 0.02308495324099746),\n",
       " (77, -0.004948305549912884),\n",
       " (78, 0.014818408069022434),\n",
       " (79, 0.004372369370747652),\n",
       " (80, 0.0038319787102985402),\n",
       " (81, 0.0018341038596007113),\n",
       " (82, -0.005368524325308769),\n",
       " (83, 0.005019818116473013),\n",
       " (84, 0.007982176938205883),\n",
       " (85, 0.006947925045058569),\n",
       " (86, 0.007539410499092885),\n",
       " (87, 0.014661173183605245),\n",
       " (88, 0.0032664057274772148),\n",
       " (89, 0.00105391010937571),\n",
       " (90, 0.011486374683041757),\n",
       " (91, -0.009837381230463633),\n",
       " (92, 0.0009454312653215285),\n",
       " (93, 0.02514332490035855),\n",
       " (94, -0.0022501479583867433),\n",
       " (95, 0.0027640360748114787),\n",
       " (96, -0.0016516382052680178),\n",
       " (97, 0.0048936386420818274),\n",
       " (98, -0.0015729305163972376),\n",
       " (99, 0.0052799064331379595),\n",
       " (100, -0.0030377527590153383),\n",
       " (101, 0.009019106290962253),\n",
       " (102, 0.011100364850279949),\n",
       " (103, 0.018674121524770573),\n",
       " (104, 0.0080074767385409),\n",
       " (105, 0.000344236636768157),\n",
       " (106, 0.008543353551893925),\n",
       " (107, -0.0015357854072139865),\n",
       " (108, -0.007478350005202559),\n",
       " (109, 0.004326113214391713),\n",
       " (110, 0.0035793681121839044),\n",
       " (111, -0.010137875388832632),\n",
       " (112, -0.024445866340614006),\n",
       " (113, -0.002970052449775039),\n",
       " (114, 0.015270247953476177),\n",
       " (115, 0.00970154246979153),\n",
       " (116, 0.004350980890126444),\n",
       " (117, -0.013121874198540138),\n",
       " (118, 0.011718003799011474),\n",
       " (119, 0.014982191634315549),\n",
       " (120, 0.004706912433635864),\n",
       " (121, -0.0007611451331715818),\n",
       " (122, -0.005814107622458274),\n",
       " (123, -0.0016170157716317102),\n",
       " (124, 0.0001187084666741489),\n",
       " (125, -0.015701876048375956),\n",
       " (126, -0.00503529188435556),\n",
       " (127, 0.011217783105427466),\n",
       " (128, 0.02065518228185981),\n",
       " (129, -0.001626620822725531),\n",
       " (130, -0.030091541832738745),\n",
       " (131, 0.000404332355586168),\n",
       " (132, -0.001573055646208927),\n",
       " (133, 0.009675354434846616),\n",
       " (134, 0.018807027883717374),\n",
       " (135, -0.017500300135850532),\n",
       " (136, -0.005657471017701874),\n",
       " (137, -0.009630347652179603),\n",
       " (138, 0.028042742825773917),\n",
       " (139, -0.00438046576152476),\n",
       " (140, -0.013791182758100637),\n",
       " (141, 0.0026876358677057023),\n",
       " (142, -0.020255253954492784),\n",
       " (143, -0.008371000482671379),\n",
       " (144, -0.023964877531971095),\n",
       " (145, -0.0031677132509530725),\n",
       " (146, -0.005842965410501947),\n",
       " (147, 0.005270551591666945),\n",
       " (148, 0.005420634393442971),\n",
       " (149, 0.009524300705206133),\n",
       " (150, -0.0026741569303318146),\n",
       " (151, 0.01653820025470904),\n",
       " (152, -0.006771742312280768),\n",
       " (153, 0.005490452251054264),\n",
       " (154, 0.022483443426851),\n",
       " (155, -0.010243123901524709),\n",
       " (156, -0.0038792220922180486),\n",
       " (157, 0.017753141786701596),\n",
       " (158, -0.0013293495584027644),\n",
       " (159, 0.009180966876072751),\n",
       " (160, -0.009813333003556395),\n",
       " (161, 0.023860394284890875),\n",
       " (162, 0.0010900679841670518),\n",
       " (163, -0.0069920600720317465),\n",
       " (164, -0.016127754029356706),\n",
       " (165, 0.0203006232179636),\n",
       " (166, 0.0029524411883924445),\n",
       " (167, 0.013338750151404748),\n",
       " (168, 3.574300060005427e-05),\n",
       " (169, 0.013974277022050143),\n",
       " (170, 0.027870329065397915),\n",
       " (171, 0.003961649215152182),\n",
       " (172, 0.01084191025038568),\n",
       " (173, 0.00816491876255113),\n",
       " (174, 0.022032429539520313),\n",
       " (175, -0.025940017533563682),\n",
       " (176, -0.012714094390850573),\n",
       " (177, 0.01265051095866932),\n",
       " (178, -0.0299590171145439),\n",
       " (179, 0.004907263839591892),\n",
       " (180, 0.04318218748821242),\n",
       " (181, -0.0005501200277448483),\n",
       " (182, 0.005723413492236066),\n",
       " (183, 0.00740852543199614),\n",
       " (184, 0.018387510109902798),\n",
       " (185, -0.007954875870431533),\n",
       " (186, 0.015236548370256118),\n",
       " (187, 0.009621273432416618),\n",
       " (188, -0.0319185120425091),\n",
       " (189, -0.04382652649162435),\n",
       " (190, -0.00016323476302918145),\n",
       " (191, -0.02418214561987708),\n",
       " (192, -0.01961725474299723),\n",
       " (193, -0.02546034011061688),\n",
       " (194, -0.020941407258864963),\n",
       " (195, -0.0014565947194734993),\n",
       " (196, -0.003890752238780567),\n",
       " (197, -0.030304183023946756),\n",
       " (198, -0.01343931176248285),\n",
       " (199, 0.0321567022047512)]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_lsi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1061 0.25669920444488525 第五回　弯弓射雕(1)\n",
      "1172 0.25669920444488525 第五回　弯弓射雕(2)\n",
      "3880 0.20211602747440338 郭靖涨红了脸，答道：“我想，王真人的武功既已天下第一，他再练得更强，仍也不过是天下第一。我还想，他到华山论剑，倒不是为了争天下第一的名头，而是要得这部《九阴真经》。他要得到经书，也不是为了要练其中的功夫，却是相救普天下的英雄豪杰，教他们免得互相所杀，大家不得好死。”\n",
      "2880 0.18251502513885498 武功天下第一的王真人已经逝世，剩下我们四个大家半斤八两，各有所忌。\n",
      "8377 0.18021109700202942 你上得华山来，妄想争那武功天下第一的荣号，莫说你武功未必能独魁群雄，纵然是当世无敌，天下英雄能服你这卖国好徒么？”\n",
      "8433 0.17648662626743317 “天下英雄，唯使君与叫化啦。我见了你女儿，肚里的蛔虫就乱钻乱跳，馋涎水直流。咱们爽爽快快的马上动手，是你天下第一也好，是我第一也好，我只等吃蓉儿烧的好菜。”\n",
      "796 0.16413959860801697 令郎更是英雄人物，老英雄怎么不提？”王罕笑道：“老汉死了之后，自然是他统领部众。但他怎比得上他的两个义兄？札木合足智多谋。铁木真更是刚勇无双，他是赤手空拳，自己打出来的天下。蒙古人中的好汉，哪一个不甘愿为他卖命？”完颜洪烈道：“难道老英雄的将士，便不及铁木真汗的部下吗？”\n",
      "2745 0.1605529934167862 黄蓉笑道：“你老人家料事如神。你说我爹爹很厉害，是不是？”洪七公冷冷的道：“他当然厉害，可也不见得是天下第一。”黄蓉拍手道：“那么定是您第一啦。”\n",
      "3253 0.16004645824432373 黄蓉道：”既然你老人家武功第一，那部经书该归您所有啊。”裘千仞道：”我也懒得跟人家争了。那东邪、西毒、南帝、北丐四人都是半斤八两，这些年来人人苦练，要争这大下第一的名头。二次华山论剑，热闹是有得看的。”黄蓉道：“还有二次华山论剑么？”裘千仞道：“二十五年一世啊。\n",
      "8337 0.1580556035041809 黄蓉道：“你到华山来，想是要争那‘武功天下第一’的名号了？”郭靖道：“蓉儿，你怎么又来取笑？我是要向周大哥请教一个法子，怎生将已会的武功尽数忘却。”当下将这些日来自己所思各节一一说了。\n",
      "3872 0.1562965214252472 他们五人约定在华山论剑，谁的武功天下第一，经书就归谁所有。”郭靖道：“那经书终究是落在你师哥手里了。”\n",
      "8607 0.1495915651321411 当晚成吉思汗崩于金帐之中，临死之际，口里喃喃念着：“英雄，英雄……”想是心中一直琢磨着郭靖的那番言语。\n",
      "8487 0.14730820059776306 欧阳锋仰天长笑，双臂在半空乱舞。向黄药师道：“段皇爷，你服不服我？”黄药师心中不忿，暗想：“武功天下第一的名号，竟教一个疯子得了去，我眼老叫化二人岂不教天下好汉耻笑？”但若上前再斗，自忖却又难以取胜，只得点了点头。\n",
      "5153 0.14457952976226807 正自悠然神往，黄蓉忽地一扯他衣袖，跃到亭后花木丛中，在他肩头按了按，两人蹲下身来，只听脚步声响，有人走入亭中。过了一会，听得一人说道：“韩世忠自然是英雄了。他夫人梁红玉虽出身娼妓，后来擂鼓督战，助夫制胜，也算得是女中人杰。”郭靖听这声音有些耳熟，一时却想不起是谁。又听一人道：“岳飞与韩世忠虽说是英雄，但皇帝要他死，要夺他的兵权，韩岳二人也只好听命，可见帝皇之威，是任何英雄违抗不来的。”郭靖听这人的口音正是杨康，不觉一怔，心想他怎么会在此处？\n",
      "4295 0.13707391917705536 欧阳锋淡淡一笑，说道：“我瞧你我也不必枉费心力来争了。武功天下第一的名号，早已有了主儿。”洪七公奇道：“有了主儿？莫非你毒兄已练成了举世无双的绝招？”欧阳锋微微一笑，说道：“想欧阳锋这点儿微末功夫，怎敢觊觎‘武功天下第一’的尊号？我说的是传授过这位郭贤侄功夫的那人。”洪七公笑道：“你说老叫化？这个嘛，兄弟想是想的，但药兄的功夫日益精进，你毒兄又是越活越命长，段皇爷的武功只怕也没搁下，这就挨不到老叫化啦。”\n",
      "2746 0.13396157324314117 洪七公道：“那倒也未必。二十多年前，我们东邪、西毒、南帝、北丐、中神通五人在华山绝顶比武论剑，比了七天七夜，终究是中神通最厉害，我们四人服他是天下第一。”黄蓉道：“中神通是谁呀？”\n",
      "3561 0.133798748254776 只听程大小姐道：“承老英雄仗义援手，晚辈感激无已，一切全凭老英雄吩咐。”黎生道：“姑娘是千金之体，就是给这狂徒多瞧一眼也是亵渎了。”\n",
      "683 0.1326950490474701 你没听见过吗？”哲别心中一凛：“早听说博尔术是蒙古人中的英雄，原来是他。”横目斜睨，哼了一声。\n",
      "8508 0.13249383866786957 黄药师哈哈大笑，一手挽了女儿，一手挽着郭靖，向洪七公道：“七兄，武学之道无穷无尽，今日见识到老毒物的武功，实令人又惊又愧。自重阳真人逝世，从此更无武功天下第一之人了。”\n",
      "3623 0.13070784509181976 洪七公落下梁来，说道：“你称他做英雄，可是他打不过你，那么你更是大英雄了，哈哈，不害臊么？”欧阳克好生着恼，只是自知武功与他差得太远，不敢出言冲撞，只得强忍怒气，不敢作声。洪七公道：“你仗着得了老毒物的传授，便想在中原横行，哼哼，放着老叫化没死，须容你不得。”\n"
     ]
    }
   ],
   "source": [
    "##### TF-IDF #####\n",
    "result = similarity[test_tfidf]\n",
    "\n",
    "for idx, confidence in result:\n",
    "    print(idx, confidence, docs[idx])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2880 0.6950327754020691 武功天下第一的王真人已经逝世，剩下我们四个大家半斤八两，各有所忌。\n",
      "8377 0.6451570987701416 你上得华山来，妄想争那武功天下第一的荣号，莫说你武功未必能独魁群雄，纵然是当世无敌，天下英雄能服你这卖国好徒么？”\n",
      "7965 0.6377695798873901 两人回到帐中，这番当真研习起《九阴真经》上的武功来，谈论之下，均觉对方一年来武功大有长进，均感欣慰。\n",
      "5277 0.5926315784454346 武术中有言道：”未学打人，先学挨打。”初练粗浅功夫，却须由师父传授怎生挨打而不受重伤，到了武功精深之时，就得研习护身保命、解穴救伤、接骨疗毒诸般法门。须知强中更有强中手，任你武功盖世，也难保没失手的日子。这《九阴真经》中的“疗伤篇”，讲的是若为高手以气功击伤，如何以气功调理真元，治疗内伤。至于折骨、金创等外伤的治疗，研习真经之人自也不用再学。\n",
      "4332 0.5838804244995117 他牢牢记住师兄王重阳的遗训，决不敢修习经中所载武功，哪知为了教导郭靖，每日里口中解释、手上比划，不知不觉的已把经文深印脑中，睡梦之间，竟然意与神会，奇功自成，这时把拳脚施展出来，却是无不与经中所载的拳理法门相合。他武功深湛，武学上的悟心又是极高，兼之《九阴真经》中所载纯是道家之学，与他毕生所学本是一理相通，他不想学武功，武功却自行扑上身来。他纵声大叫：“糟了，糟了，这叫做惹鬼上身，挥之不去了。\n",
      "1232 0.5692782402038574 郭靖心中奇怪：“怎么他也知道这回事？”那道人又道：“那小道士虽然摔了你一个筋斗，但他全以巧劲取胜，讲到武功根基，未必就强得过你。\n",
      "2166 0.5652763247489929 原来这参仙老怪本是长白山中的参客，后来害死了一个身受重伤的前辈异人，从他衣囊中得了一本武学秘本和十余张药方，照法修练研习，自此武功了得，兼而精通药理。药方中有一方是以药养蛇、从而易筋壮体的秘诀。\n",
      "2626 0.5579832792282104 丘处机道：“韩女侠，天下武学之士，肩上受了这样的一扳，若是抵挡不住，必向后跌，只有九指神丐的独家武功，却是向前俯跌。只因他的武功刚猛绝伦，遇强愈强。穆姑娘受教时日虽短，却已习得洪老前辈这派武功的要旨。她抵不住王师弟的一扳，但决不随势屈服，就算跌倒，也要跌得与敌人用力的方向相反。”\n",
      "3222 0.5532119274139404 陆庄主虽没见过裘千仞的武功，但素仰他的威名，知道当年东邪、西毒、南帝、北丐、中神通五人在华山绝顶论剑，也曾邀他到场，只是他适有要事，未能赴约，但既受到邀请，自是武功卓绝，非同小可，纵使不及王重阳等五人，谅亦相差不远，有他在这里，黑风双煞是不能为恶的了，当下向郭靖及黄蓉道：“两位还没走，真好极了。这位裘老前辈武功极高，常人难以望其项背，天幸今日凑巧到来，我还忌惮甚么对头？待会两位请自行在卧室中休息，只要别出房门，那就没事。”\n",
      "1154 0.5407816171646118 这十六年来，朱聪不断追忆昔日醉仙楼和法华寺中动手的情景，丘处机的一招一式，在他心中尽皆清晰异常，尤胜当时所见。但要在他武功中寻找甚么破绽与可乘之机，实非己之所能，有时竟会想到：“只有铜尸铁尸，或能胜得过这牛鼻子。”\n",
      "4007 0.5281177759170532 这番传授武功，可与普天下古往今来的教武大不相同，所教的功夫，教的人自己竟是全然不会。他只用口讲述，决不出手示范，待郭靖学会了经上的几招武功，他就以全真派的武功与之拆招试拳，果见经上武功妙用无穷。\n",
      "3849 0.5248280763626099 周伯通说道：”他主怕这部大道藏刻错了字，皇帝发觉之后不免要杀他的头，因此上一卷一卷的细心校读。不料想这么读得几年，他居然便精通道学，更因此而悟得了武功中的高深道理。他无师自通，修习内功外功，竟成为一位武功大高手。兄弟，这个黄裳可比你聪明得多了。我没他这般本事，料想你也没有。”郭靖道：“这个自然。五千多卷道书，要我从头至尾读一遍，我这一辈子也就干不了，别说领会甚么武功了。”\n",
      "3827 0.5170783996582031 你真的不愿么？我师哥王重阳武功比我高得多，当年他不肯和我结拜，难道你的武功也比我高得多？我看大大的不见得。”郭靖道：“晚辈的武功比你低得太多，结拜实在不配。”周伯通道：“若说武功一样，才能结拜，那么我去跟黄老邪、老毒物结拜？他们又嫌我打他们不过了，岂有此理！你要我跟这又聋又哑的家伙结拜？”说着手指那老仆，双脚乱跳，大发脾气。\n",
      "8441 0.5033394694328308 数招一过，黄药师大是惊异：“这傻小子的武功怎么竟练到了这个地步？\n",
      "3246 0.4853397011756897 “少庄主的武功虽非极高，但怎么能说‘无用’？”\n",
      "3858 0.4841059446334839 周伯通道：“不是乔装改扮。你想，黄裳的几十个仇人，个个都是好手，武功包含诸家各派，何等深奥，何等繁复？他要破解每一人的绝招，可得耗费多少时候心血？原来他独自躲在深山之中钻研武功，日思夜想的就只是武功，别的甚么也不想，不知不觉竟已过了四十多年。”郭靖惊道：“过了四十多年？”\n",
      "3872 0.482600599527359 他们五人约定在华山论剑，谁的武功天下第一，经书就归谁所有。”郭靖道：“那经书终究是落在你师哥手里了。”\n",
      "8243 0.4727248549461365 郭靖沉吟片刻，道：“道长之言虽然不错，但想当今之世，江湖好汉都称东邪、西毒、南帝、北丐四人武功最强。弟子仔细想来，武功要练到这四位前辈一般，固是千难万难，但即令如此，于人于己又有甚么好处？”\n",
      "3448 0.4710177183151245 陆乘风在桃花岛上学得一身武功，虽然双腿残废，但手上功夫未废，心中又深知武学精义，眼见自己独子虽然练武甚勤，总以未得明师指点，成就有限，自己明明有满肚子的武功诀窍可以教他，但格于门规，未敢泄露，为了怕儿子痴缠，索性一直不让他知道自己会武，这时自己取得列于恩师门墙，又得师父允可教子，爱子武功指日可以大进，心中如何不喜？要想说几句感激的话，喉头却硬住了说不出来。\n",
      "7378 0.4702221751213074 第三十五回　铁枪庙中(1)\n"
     ]
    }
   ],
   "source": [
    "##### LSI #####\n",
    "result = similarity_lsi[test_lsi]\n",
    "\n",
    "for idx, confidence in result:\n",
    "    print(idx, confidence, docs[idx])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}