12. regex.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regex （正则表达式）通用篇\n",
    "## 本文目标\n",
    "本文目标让你明白正则表达式是什么？并对其有一些基本了解，可以在自己的程序或网页中使用它。<br>\n",
    "如果听说过机器学习的一个分支：NLP，即自然语言处理，正则表达式，正是NLP的开山斧。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 如何使用本教程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "正则表达式一般的写法充斥着大量的(),[],{},\\\\,/，,\\*, ^, $等字符，不过别被这些复杂的表达式吓倒，只要跟着我一步步来，会发现正则表达式其实没有想象中的那么难。<br>\n",
    "当然，如果你看完了这篇教程之后，发现自己明白了很多，却又几乎什么都记不得，那也是很正常的——我认为，没接触过正则表达式的人在看完这篇教程后，能把提到过的语法记住80%以上的可能性为零。<br>\n",
    "这里只是让你明白基本的原理，以后你还需要多练习，多使用，才能熟练掌握正则表达式。<br>\n",
    "除了作为入门教程之外，本文还试图成为可以在日常工作中使用的正则表达式语法参考手册。就本人的经历来说，这个目标还是完成得不错的——你看，我自己也没能把所有的东西记下来，不是吗？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 正则表达式到底是什么东西？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在编写处理字符串的程序或网页时，经常会有查找符合某些复杂规则的字符串的需要。正则表达式就是用于描述这些规则的工具。换句话说，正则表达式就是记录文本规则的代码。\n",
    "\n",
    "很可能你使用过Windows/Dos下用于文件查找的通配符(wildcard)，也就是*和?。如果你想查找某个目录下的所有的Word文档的话，你会搜索*.doc。在这里，*会被解释成任意的字符串。和通配符类似，正则表达式也是用来进行文本匹配的工具，只不过比起通配符，它能更精确地描述你的需求——当然，代价就是更复杂——比如你可以编写一个正则表达式，用来查找所有以0开头，后面跟着2-3个数字，然后是一个连字号\"-\"，最后是7或8位数字的字符串(像010-12345678或0376-7654321)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 入门"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "学习正则表达式的最好方法是从例子开始，理解例子之后再自己对例子进行修改，实验。下面给出了不少简单的例子，并对它们作了详细的说明。\n",
    "\n",
    "假设你在一篇英文小说里查找hi，你可以使用正则表达式hi。\n",
    "\n",
    "这几乎是最简单的正则表达式了，它可以精确匹配这样的字符串：由两个字符组成，前一个字符是h，后一个是i。通常，处理正则表达式的工具会提供一个忽略大小写的选项，如果选中了这个选项，它可以匹配hi,HI,Hi,hI这四种情况中的任意一种。\n",
    "\n",
    "不幸的是，很多单词里包含hi这两个连续的字符，比如him,history,high等等。用hi来查找的话，这里边的hi也会被找出来。如果要精确地查找hi这个单词的话，我们应该使用\\bhi\\b。\n",
    "\n",
    "\\b是正则表达式规定的一个特殊代码（好吧，某些人叫它元字符，metacharacter），代表着单词的开头或结尾，也就是单词的分界处。虽然通常英文的单词是由空格，标点符号或者换行来分隔的，但是\\b并不匹配这些单词分隔字符中的任何一个，它只匹配一个位置。\n",
    "\n",
    "假如你要找的是hi后面不远处跟着一个Lucy，你应该用\\bhi\\b.\\*\\bLucy\\b。\n",
    "\n",
    "这里，.是另一个元字符，匹配除了换行符以外的任意字符。*同样是元字符，不过它代表的不是字符，也不是位置，而是数量——它指定*前边的内容可以连续重复使用任意次以使整个表达式得到匹配。因此，.\\*连在一起就意味着任意数量的不包含换行的字符。现在\\bhi\\b.\\*\\bLucy\\b的意思就很明显了：先是一个单词hi,然后是任意个任意字符(但不能是换行)，最后是Lucy这个单词。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def getretext(keyword, text):\n",
    "    pattern = re.compile(keyword, re.I)\n",
    "    \n",
    "    return pattern.findall(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function compile in module re:\n",
      "\n",
      "compile(pattern, flags=0)\n",
      "    Compile a regular expression pattern, returning a pattern object.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "help(re.compile)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Hi']\n",
      "['Hi, him, Lucy']\n",
      "['Hi', 'hi', 'hi']\n"
     ]
    }
   ],
   "source": [
    "text = r'Hi, him, Lucy, history'\n",
    "keyword = r'\\bhi\\b'\n",
    "print(getretext(keyword, text))\n",
    "keyword = r'\\bhi\\b.*\\bLucy\\b'\n",
    "print(getretext(keyword, text))\n",
    "keyword = r'hi'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果同时使用其它元字符，我们就能构造出功能更强大的正则表达式。比如下面这个例子：\n",
    "\n",
    "0\\d\\d-\\d\\d\\d\\d\\d\\d\\d\\d匹配这样的字符串：以0开头，然后是两个数字，然后是一个连字号\"-\"，最后是8个数字(也就是中国的电话号码。当然，这个例子只能匹配区号为3位的情形)。\n",
    "\n",
    "这里的\\d是个新的元字符，匹配一位数字(0，或1，或2，或……)。-不是元字符，只匹配它本身——连字符(或者减号，或者中横线，或者随你怎么称呼它)。\n",
    "\n",
    "为了避免那么多烦人的重复，我们也可以这样写这个表达式：0\\d{2}-\\d{8}。这里\\d后面的{2}({8})的意思是前面\\d必须连续重复匹配2次(8次)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['021-12345678', '020-12345678']\n"
     ]
    }
   ],
   "source": [
    "text = '021-12345678 020-12345678'\n",
    "keyword = r'0\\d{2}-\\d{8}'\n",
    "# keyword = '\\d'\n",
    "print(getretext(text=text, keyword=keyword))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 测试正则表达式\n",
    "如果你不觉得正则表达式很难读写的话，要么你是一个天才，要么，你不是地球人。正则表达式的语法很令人头疼，即使对经常使用它的人来说也是如此。由于难于读写，容易出错，所以找一种工具对正则表达式进行测试是很有必要的。\n",
    "\n",
    "不同的环境下正则表达式的一些细节是不相同的，这里介绍两种可用的免费测试工具"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[免费的regex matcher tracer](http://www.regexlab.com/zh/mtracer/)\n",
    "![截图示例](./image/2018-08-12 16_43_52-Untitled - Regex Match Tracer.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[免费的正则在线工具](https://tool.lu/regex/)\n",
    "![截图示例](./image/2018-08-12 16_44_53-正则表达式测试工具.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 元字符\n",
    "现在你已经知道几个很有用的元字符了，如\\b,.,\\*，还有\\d.正则表达式里还有更多的元字符，比如\\s匹配任意的空白符，包括空格，制表符(Tab)，换行符，中文全角空格等。\\w匹配字母或数字或下划线或汉字等。\n",
    "\n",
    "下面来看看更多的例子：\n",
    "\n",
    "\\ba\\w\\*\\b匹配以字母a开头的单词——先是某个单词开始处(\\b)，然后是字母a,然后是任意数量的字母或数字(\\w\\*)，最后是单词结束处(\\b)。\n",
    "\n",
    "注：好吧，现在我们说说正则表达式里的单词是什么意思吧：就是不少于一个的连续的\\w。不错，这与学习英文时要背的成千上万个同名的东西的确关系不大 :)\n",
    "\n",
    "\\d+匹配1个或更多连续的数字。这里的+是和\\*类似的元字符，不同的是\\*匹配重复任意次(可能是0次)，而+则匹配重复1次或更多次。\n",
    "\n",
    "\\b\\w{6}\\b 匹配刚好6个字符的单词。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['arena1234']\n",
      "['word', 'with']\n"
     ]
    }
   ],
   "source": [
    "text = r'arena1234 Mike'\n",
    "keyword = r'\\ba\\w*\\b'\n",
    "print(getretext(keyword, text))\n",
    "\n",
    "text = 'python is a word with 6 characters.'\n",
    "keyword = r'\\b\\w{4}\\b'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 常用的元字符\n",
    "![常用的元字符](./image/2018-08-12 16_54_51-正则表达式30分钟入门教程.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注：正则表达式引擎通常会提供一个\"测试指定的字符串是否匹配一个正则表达式\"的方法。这里的匹配是指是字符串里有没有符合表达式规则的部分。如果不使用^和\\$的话，对于\\d{5,12}而言，使用这样的方法就只能保证字符串里包含5到12连续位数字，而不是整个字符串就是5到12位数字。\n",
    "\n",
    "元字符^（和数字6在同一个键位上的符号）和\\$都匹配一个位置，这和\\b有点类似。^匹配你要用来查找的字符串的开头，$匹配结尾。这两个代码在验证输入的内容时非常有用。\n",
    "\n",
    "比如一个网站如果要求你填写的QQ号必须为5位到12位数字时，可以使用：^\\d\\{5,12\\}$\n",
    "\n",
    "这里的{5,12}和前面介绍过的{2}是类似的，只不过{2}匹配只能不多不少重复2次，{5,12}则是重复的次数不能少于5次，不能多于12次，否则都不匹配。\n",
    "\n",
    "因为使用了^和\\$，所以输入的整个字符串都要用来和\\d{5,12}来匹配，也就是说整个输入必须是5到12个数字，因此如果输入的QQ号能匹配这个正则表达式的话，那就符合要求了。\n",
    "\n",
    "和忽略大小写的选项类似，有些正则表达式处理工具还有一个处理多行的选项。如果选中了这个选项，^和\\$的意义就变成了匹配行的开始处和结束处。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['12345622', '22256', '123456']\n"
     ]
    }
   ],
   "source": [
    "text = r'hello12345622key, 22256 123456key'\n",
    "containnumber = r'\\d{5,12}'\n",
    "print(getretext(containnumber, text))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['8019068']\n",
      "[]\n"
     ]
    }
   ],
   "source": [
    "righttext = r'8019068'\n",
    "mustbenumber = r'^\\d{5,12}$'\n",
    "print(getretext(mustbenumber, righttext))\n",
    "wrongtext = r'80190myqq 12345678'\n",
    "print(getretext(mustbenumber, wrongtext))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 字符转义\n",
    "如果你想查找元字符本身的话，比如你查找.,或者\\*,就出现了问题：你没办法指定它们，因为它们会被解释成别的意思。这时你就得使用\\来取消这些字符的特殊意义。因此，你应该使用\\\\.和\\\\\\*。当然，要查找\\\\本身，你也得用\\\\\\\\\n",
    "\n",
    "例如：china\\\\.net匹配china.net，C:\\\\\\\\Windows匹配C:\\\\Windows。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['12344435324']\n"
     ]
    }
   ],
   "source": [
    "righttext = r'*12344435324s344'\n",
    "mustbenumber = r'\\d{5,}'\n",
    "print(getretext(mustbenumber, righttext))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 重复\n",
    "已经看过了前面的\\*,+,{2},{5,12}这几个匹配重复的方式了。下面是正则表达式中所有的限定符(指定数量的代码，例如\\*,{5,12}等)："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "常用的限定符"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![常用的限定符](./image/2018-08-12 17_15_06-正则表达式30分钟入门教程.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面是一些使用重复的例子：\n",
    "\n",
    "Windows\\\\d+匹配Windows后面跟1个或更多数字\n",
    "\n",
    "^\\\\w+匹配一行的第一个单词(或整个字符串的第一个单词，具体匹配哪个意思得看选项设置)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Windows10']\n"
     ]
    }
   ],
   "source": [
    "keyword = r'Windows\\d+'\n",
    "text = r'Windows10'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 字符类\n",
    "要想查找数字，字母或数字，空白是很简单的，因为已经有了对应这些字符集合的元字符，但是如果你想匹配没有预定义元字符的字符集合(比如元音字母a,e,i,o,u),应该怎么办？\n",
    "\n",
    "很简单，你只需要在<b>方括号</b>里列出它们就行了，像\\[aeiou\\]就匹配任何一个英文元音字母，\\[.?!\\]匹配标点符号(.或?或!)。\n",
    "\n",
    "我们也可以轻松地指定一个字符范围，像\\[0-9\\]代表的含意与\\d就是完全一致的：一位数字；同理\\[a-z0-9A-Z_\\]也完全等同于\\\\w（如果只考虑英文的话）。\n",
    "\n",
    "下面是一个更复杂的表达式：\\\\(?0\\d\\{2\\}\\[\\) -\\]?\\\\d\\{8\\}。\n",
    "\n",
    "注意：\"\\(\"和\"\\)\"也是元字符，后面的分组节里会提到，所以在这里需要使用转义。\n",
    "\n",
    "这个表达式可以匹配几种格式的电话号码，像(010)88886666，或022-22334455，或02912345678等。我们对它进行一些分析吧：首先是一个转义字符\\\\(,它能出现0次或1次(?),然后是一个0，后面跟着2个数字\\(\\\\d\\{2\\}\\)，然后是)或-或空格中的一个，它出现1次或不出现\\(?\\)，最后是8个数字\\(\\\\d\\{8\\}\\)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['a', 'e']\n"
     ]
    }
   ],
   "source": [
    "text = r'Face'\n",
    "keyword = r'[aeiou]'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['(010)88886666', '022-22334455', '02912345678', '021 22334455', '021 2233445']\n"
     ]
    }
   ],
   "source": [
    "text = r'(010)88886666, 022-22334455 02912345678 021 22334455  021 2233445'\n",
    "keyword = r'\\(?0\\d{2}[\\) -]?\\d{7,8}'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 分支条件\n",
    "不幸的是，刚才那个表达式也能匹配010)12345678或(022-87654321这样的\"不正确\"的格式。要解决这个问题，我们需要用到分枝条件。正则表达式里的分枝条件指的是有几种规则，如果满足其中任意一种规则都应该当成匹配，具体方法是用|把不同的规则分隔开。听不明白？没关系，看例子：\n",
    "\n",
    "0\\d{2}-\\d{8}|0\\d{3}-\\d{7}这个表达式能匹配两种以连字号分隔的电话号码：一种是三位区号，8位本地号(如010-12345678)，一种是4位区号，7位本地号(0376-2233445)。\n",
    "\n",
    "```\n",
    "\\(0\\d{2}\\)[- ]?\\d{8}\n",
    "|\n",
    "0\\d{2}[- ]?\\d{8}\n",
    "```\n",
    "这个表达式匹配3位区号的电话号码，其中区号可以用小括号括起来，也可以不用，区号与本地号间可以用连字号或空格间隔，也可以没有间隔。你可以试试用分枝条件把这个表达式扩展成也支持4位区号的。\n",
    "\n",
    "\\d{5}-\\d{4}|\\d{5}这个表达式用于匹配美国的邮政编码。美国邮编的规则是5位数字，或者用连字号间隔的9位数字。之所以要给出这个例子是因为它能说明一个问题：使用分枝条件时，要注意各个条件的顺序。如果你把它改成\\d{5}|\\d{5}-\\d{4}的话，那么就只会匹配5位的邮编(以及9位邮编的前5位)。原因是匹配分枝条件时，将会从左到右地测试每个条件，如果满足了某个分枝的话，就不会去再管其它的条件了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['(021)']\n"
     ]
    }
   ],
   "source": [
    "phone = r'021 14332244 (021) 22883322 (033 12235533'\n",
    "keyword = r'\\(0\\d{2}\\)[- ]?\\d{8}|0\\d{2}[- ]?\\d{8}'\n",
    "# keyword = '\\(.*\\)'\n",
    "print(getretext(keyword, phone))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['12345', '12345-4433']\n"
     ]
    }
   ],
   "source": [
    "zipcode = r'12345 12345-4433'\n",
    "keyword = r'\\d{5}-\\d{4}|\\d{5}'\n",
    "print(getretext(keyword, zipcode))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['12345', '12345']\n"
     ]
    }
   ],
   "source": [
    "zipcode = r'12345 12345-4433'\n",
    "keyword = r'\\d{5}|\\d{5}-\\d{4}'\n",
    "# 匹配分枝条件时，将会从左到右地测试每个条件，\n",
    "#如果满足了某个分枝的话，就不会去再管其它的条件了\n",
    "print(getretext(keyword, zipcode))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 分组\n",
    "我们已经提到了怎么重复单个字符（直接在字符后面加上限定符就行了）；但如果想要重复多个字符又该怎么办？你可以用小括号来指定子表达式(也叫做分组)，然后你就可以指定这个子表达式的重复次数了，你也可以对子表达式进行其它一些操作(后面会有介绍)。\n",
    "\n",
    "(\\d{1,3}\\\\.){3}\\d{1,3}是一个简单的IP地址匹配表达式。要理解这个表达式，请按下列顺序分析它：\\d{1,3}匹配1到3位的数字，(\\d{1,3}\\.){3}匹配三位数字加上一个英文句号(这个整体也就是这个分组)重复3次，最后再加上一个一到三位的数字(\\d{1,3})。\n",
    "\n",
    "注：IP地址中每个数字都不能大于255. 经常有人问我, 01.02.03.04 这样前面带有0的数字, 是不是正确的IP地址呢? 答案是: 是的, IP 地址里的数字可以包含有前导 0 (leading zeroes).\n",
    "\n",
    "不幸的是，它也将匹配256.300.888.999这种不可能存在的IP地址。如果能使用算术比较的话，或许能简单地解决这个问题，但是正则表达式中并不提供关于数学的任何功能，所以只能使用冗长的分组，选择，字符类来描述一个正确的IP地址：((2[0-4]\\d|25[0-5]|[01]?\\d\\d?)\\\\.){3}(2[0-4]\\d|25[0-5]|[01]?\\d\\d?)。\n",
    "\n",
    "理解这个表达式的关键是理解2[0-4]\\d|25[0-5]|[01]?\\d\\d?，这里我就不细说了，你自己应该能分析得出来它的意义。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "def getlistiterbyregex(keyword, text):\n",
    "    pattern = re.compile(keyword, re.I)\n",
    "    return re.finditer(pattern, text)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "127.0.0.1\n",
      "0.\n",
      "192.168.1.1\n",
      "1.\n"
     ]
    }
   ],
   "source": [
    "keyword = r'(\\d{1,3}\\.){3}\\d{1,3}'\n",
    "ip = '127.0.0.1 192.168.1.1'\n",
    "matchlist = getlistiterbyregex(keyword, ip)\n",
    "for match in matchlist:\n",
    "    print(match.group(0))\n",
    "    print(match.group(1))\n",
    "#     print(match.group(2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在python中，可以通过\n",
    "```\n",
    "(?P<groupname>正则表达式)\n",
    "```\n",
    "来实现通过groupname获取信息的目的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "127.0.0.18\n",
      "192.168.1.15\n"
     ]
    }
   ],
   "source": [
    "keyword = r'(?P<ip>(\\d{1,3}\\.){3}(\\d{1,3}))'\n",
    "ip = '127.0.0.18 192.168.1.15 is ip addresses'\n",
    "matchlist = getlistiterbyregex(keyword, ip)\n",
    "for match in matchlist:\n",
    "    print(match.group('ip'))\n",
    "#     print(match.group(2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "255.255.255.1\n",
      "211.1.38.1\n",
      "56.110.11.2\n"
     ]
    }
   ],
   "source": [
    "keyword = r'((2[0-4]\\d|25[0-5]|[01]?\\d\\d?)\\.){3}(2[0-4]\\d|25[0-5]|[01]?\\d\\d?)'\n",
    "ip = '255.255.255.1 211.1.38.1 256.110.11.2'\n",
    "matchlist = getlistiterbyregex(keyword, ip)\n",
    "for match in matchlist:\n",
    "    print(match.group(0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 反义\n",
    "有时需要查找不属于某个能简单定义的字符类的字符。比如想查找除了数字以外，其它任意字符都行的情况，这时需要用到反义："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![反义](./image/常用反义代码.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "例子：\\S+匹配不包含空白符的字符串。\n",
    "```\n",
    "<a[^>]+>\n",
    "```\n",
    "匹配用尖括号括起来的以a开头的字符串。\n",
    "如果想匹配一切字符，包括空行，可以通过:\n",
    "```\n",
    "[\\s\\S]*\n",
    "```\n",
    "来实现这个目的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "date is: May 1, 2018\n",
      "match text is: Effective May 1, 2018, \n",
      "the Long/Short Large-Cap Portfolio will be closed to new allocations \n",
      "and only Contract Owners that have Contract Value in the Long/Short Large-Cap \n",
      "Portfolio as of the end of the Business Day on April 30, 2018, \n",
      "may continue to make Purchase Payments, transfers\n",
      "date is: April 30, 2018\n",
      "match text is: Effective April 30, 2018, \n",
      "If you have Contract Value in the Long/Short Large-Cap Portfolio \n",
      "and withdraw or transfers\n"
     ]
    }
   ],
   "source": [
    "text = r\"\"\"Effective May 1, 2018, \n",
    "the Long/Short Large-Cap Portfolio will be closed to new allocations \n",
    "and only Contract Owners that have Contract Value in the Long/Short Large-Cap \n",
    "Portfolio as of the end of the Business Day on April 30, 2018, \n",
    "may continue to make Purchase Payments, transfers, \n",
    "or withdrawals involving the Long/Short Large-Cap Portfolio. \n",
    "Effective April 30, 2018, \n",
    "If you have Contract Value in the Long/Short Large-Cap Portfolio \n",
    "and withdraw or transfers out 100% of the Contract Value, \n",
    "you will not be able to re-invest in the Long/Short Large-Cap Portfolio. \n",
    "For 30 calendar days before the Transfer Date and for 30 calendar days \n",
    "after the Transfer Date, you may make a one-time transfer of all \n",
    "or a portion of the Contract Value you have in the Long/Short \"\"\"\n",
    "keyword = r'Effective (?P<pendingdate>(january|february|march|april|may|june|july|august|september|october|november|december)[\\s]*[0-9]{1,2}[\\s]*,[\\s]*[0-9]{4}), [\\s\\S]*?transfers'\n",
    "matchlist = getlistiterbyregex(keyword, text)\n",
    "for match in matchlist:\n",
    "    print('date is: {0}'.format(match.group('pendingdate')))\n",
    "    print('match text is: {0}'.format(match.group(0)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['<a href=www.sina.com>']\n"
     ]
    }
   ],
   "source": [
    "text = r\"<a href=www.sina.com>\"\n",
    "keyword = r'<a[^>]+>'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 后向引用\n",
    "使用小括号指定一个子表达式后，<b>匹配这个子表达式的文本</b>(也就是此分组捕获的内容)可以在表达式或其它程序中作进一步的处理。默认情况下，每个分组会自动拥有一个组号，规则是：从左向右，以分组的左括号为标志，第一个出现的分组的组号为1，第二个为2，以此类推。\n",
    "\n",
    "呃……其实，组号分配还不像我刚说得那么简单："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 分组0对应整个正则表达式\n",
    "* 实际上组号分配过程是要从左向右扫描两遍的：第一遍只给未命名组分配，第二遍只给命名组分配－－因此所有命名组的组号都大于未命名的组号\n",
    "* 你可以使用(?:exp)这样的语法来剥夺一个分组对组号分配的参与权"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "后向引用用于重复搜索前面某个分组匹配的文本。例如，\\1代表分组1匹配的文本。难以理解？请看示例：\n",
    "\n",
    "\\b(\\w+)\\b\\s+\\1\\b可以用来匹配重复的单词，像go go, 或者kitty kitty。这个表达式首先是一个单词，也就是单词开始处和结束处之间的多于一个的字母或数字(\\b(\\w+)\\b)，这个单词会被捕获到编号为1的分组中，然后是1个或几个空白符(\\s+)，最后是分组1中捕获的内容（也就是前面匹配的那个单词）(\\1)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "group 0 is: go go\n",
      "group 1 is: go\n",
      "group 0 is: run run\n",
      "group 1 is: run\n"
     ]
    }
   ],
   "source": [
    "text = r'go go lets run run'\n",
    "keyword = r'\\b(\\w+)\\b\\s+\\1\\b'\n",
    "matchlist = getlistiterbyregex(keyword, text)\n",
    "for match in matchlist:\n",
    "    print('group 0 is: {0}'.format(match.group(0)))\n",
    "    print('group 1 is: {0}'.format(match.group(1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "你也可以自己指定子表达式的组名。要指定一个子表达式的组名，请使用这样的语法：\n",
    "```\n",
    "(?<Word>\\w+)\n",
    "```\n",
    "(或者把尖括号换成'也行：(?'Word'\\w+)),这样就把\\w+的组名指定为Word了。\n",
    "\n",
    "要反向引用这个分组捕获的内容，你可以使用\n",
    "```\n",
    "\\k<Word>,\n",
    "```\n",
    "所以上一个例子也可以写成这样：\n",
    "```\n",
    "\\b(?<Word>\\w+)\\b\\s+\\k<Word>\\b。\n",
    "```\n",
    "使用小括号的时候，还有很多特定用途的语法。下面列出了最常用的一些："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![常用分组](./image/常用分组语法.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 零宽断言\n",
    "接下来的四个用于查找在某些内容(但并不包括这些内容)之前或之后的东西，也就是说它们像\\b,^,$那样用于指定一个位置，这个位置应该满足一定的条件（即断言），因此它们也被称为零宽断言。最好还是拿例子来说明吧：\n",
    "\n",
    "注：断言用来声明一个应该为真的事实。正则表达式中只有当断言为真时才会继续进行匹配。\n",
    "\n",
    "(?=exp)也叫零宽度正预测先行断言，它断言自身出现的位置的后面能匹配表达式exp。比如\\b\\w+(?=ing\\b)，匹配以ing结尾的单词的前面部分(除了ing以外的部分)，如查找I'm singing while you're dancing.时，它会匹配sing和danc。\n",
    "\n",
    "(?<=exp)也叫零宽度正回顾后发断言，它断言自身出现的位置的前面能匹配表达式exp。比如(?<=\\bre)\\w+\\b会匹配以re开头的单词的后半部分(除了re以外的部分)，例如在查找reading a book时，它匹配ading。\n",
    "\n",
    "假如你想要给一个很长的数字中每三位间加一个逗号(当然是从右边加起了)，你可以这样查找需要在前面和里面添加逗号的部分：((?<=\\d)\\d{3})+\\b，用它对1234567890进行查找时结果是234567890。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['sing', 'danc']\n"
     ]
    }
   ],
   "source": [
    "# (?=exp) 零宽度正预测先行断言\n",
    "text = r\"I'm singing while you're dancing.\"\n",
    "keyword = r'\\b\\w+(?=ing\\b)'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['ading']\n"
     ]
    }
   ],
   "source": [
    "# (?<=exp) 零宽度正回顾后发断言\n",
    "text = r\"reading a book\"\n",
    "keyword = r'(?<=\\bre)\\w+\\b'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "group 0 is: 234567890\n",
      "group 1 is: 890\n"
     ]
    }
   ],
   "source": [
    "text = r\"1234567890\"\n",
    "keyword = r'((?<=\\d)\\d{3})+\\b'\n",
    "matchlist = getlistiterbyregex(keyword, text)\n",
    "for match in matchlist:\n",
    "    print('group 0 is: {0}'.format(match.group(0)))\n",
    "    print('group 1 is: {0}'.format(match.group(1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 负向零宽断言\n",
    "前面我们提到过怎么查找不是某个字符或不在某个字符类里的字符的方法(反义)。但是如果我们只是想要确保某个字符没有出现，但并不想去匹配它时怎么办？例如，如果我们想查找这样的单词--它里面出现了字母q,但是q后面跟的不是字母u,我们可以尝试这样：\n",
    "```\n",
    "\\b\\w*q[^u]\\w*\\b\n",
    "```\n",
    "匹配包含后面不是字母u的字母q的单词。但是如果多做测试(或者你思维足够敏锐，直接就观察出来了)，你会发现，如果q出现在单词的结尾的话，像Iraq,Benq，这个表达式就会出错。这是因为[^u]总要匹配一个字符，所以如果q是单词的最后一个字符的话，后面的[^u]将会匹配q后面的单词分隔符(可能是空格，或者是句号或其它的什么)，后面的\\w*\\b将会匹配下一个单词，于是\n",
    "```\n",
    "\\b\\w*q[^u]\\w*\\b\n",
    "```\n",
    "就能匹配整个Iraq fighting。负向零宽断言能解决这样的问题，因为<b>它只匹配一个位置，并不消费任何字符。</b>现在，我们可以这样来解决这个问题：\n",
    "```\n",
    "\\b\\w*q(?!u)\\w*\\b。\n",
    "```\n",
    "零宽度负预测先行断言(?!exp)，断言此位置的后面不能匹配表达式exp。例如：\\d{3}(?!\\d)匹配三位数字，而且这三位数字的后面不能是数字；\\b((?!abc)\\w)+\\b匹配不包含连续字符串abc的单词。\n",
    "\n",
    "同理，我们可以用(?<!exp),零宽度负回顾后发断言来断言此位置的前面不能匹配表达式exp：(?<![a-z])\\d{7}匹配前面不是小写字母的七位数字。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Iraq Ben', 'Youqa', 'Linqep']\n"
     ]
    }
   ],
   "source": [
    "text = r'Iraq Ben sequence Youqa Linqep'\n",
    "keyword = r'\\b\\w*q[^u]\\w*\\b'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Iraq', 'Youqa', 'Linqep']\n"
     ]
    }
   ],
   "source": [
    "keyword = r'\\b\\w*q(?!u)\\w*\\b'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['123', '345', '888']\n"
     ]
    }
   ],
   "source": [
    "text = r'123q, 12345, 888good'\n",
    "keyword = r'\\d{3}(?!\\d)'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "group 0 is: higjeabwwc\n",
      "group 0 is: ubvsabpc\n"
     ]
    }
   ],
   "source": [
    "text = r'abcdefg higjeabwwc ubvsabpc cuabcwwe'\n",
    "keyword = r'\\b((?!abc)\\w)+\\b'\n",
    "matchlist = getlistiterbyregex(keyword, text)\n",
    "for match in matchlist:\n",
    "    print('group 0 is: {0}'.format(match.group(0)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "group 0 is: 1234568\n"
     ]
    }
   ],
   "source": [
    "# (?<!exp),零宽度负回顾后发断言\n",
    "# 匹配前面不是小写字母的七位数字\n",
    "text = r'abc1234567qww  ABC1234568'\n",
    "keyword = r'(?<![a-z])\\d{7}'\n",
    "pattern = re.compile(keyword)\n",
    "matchlist = re.finditer(pattern, text)\n",
    "for match in matchlist:\n",
    "    print('group 0 is: {0}'.format(match.group(0)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一个更复杂的例子：\n",
    "```\n",
    "(?<=<(\\w+)>).*(?=<\\/\\1>)\n",
    "匹配不包含属性的简单HTML标签内里的内容。\n",
    "\n",
    "(?<=<(\\w+)>)\n",
    "指定了这样的前缀：被尖括号括起来的单词(比如可能是<b>)，然后是.\\*(任意的字符串),最后是一个后缀。\n",
    "\n",
    "(?=<\\/\\1>)\n",
    "注意后缀里的\\/，它用到了前面提过的字符转义；\\1则是一个反向引用，引用的正是捕获的第一组，前面的(\\w+)匹配的内容，这样如果前缀实际上是<b>的话，后缀就是</b>了。整个表达式匹配的是<b>和</b>之间的内容(再次提醒，不包括前缀和后缀本身)。\n",
    "\n",
    "注：请详细分析表达式(?<=<(\\w+)>).*(?=<\\/\\1>)，这个表达式最能表现零宽断言的真正用途。\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "group 0 is: this is bold text \n",
      "group 1 is: b\n"
     ]
    }
   ],
   "source": [
    "text = '<b>this is bold text </b>'\n",
    "keyword = r'(?<=<(\\w)>).*(?=<\\/\\1>)'\n",
    "# 这里要注意的一个问题是：在python和Perl中两个反向界定的表达式exp只允许使用定长文本，\n",
    "# 譬如这个例子，如果正则表达式写成：(?<=<(\\w+)>).*(?=<\\/\\1>)'\n",
    "# python解释器会报错：error: look-behind requires fixed-width pattern。\n",
    "# keyword = r'(?<=<(\\w+)>).*(?=<\\/\\1>)'\n",
    "matchlist = getlistiterbyregex(keyword, text)\n",
    "for match in matchlist:\n",
    "    print('group 0 is: {0}'.format(match.group(0)))\n",
    "    print('group 1 is: {0}'.format(match.group(1)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 230,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "group 0 is: this is text in tag \n",
      "group 1 is: tag\n"
     ]
    }
   ],
   "source": [
    "# 因为在python中两个反向界定的表达式exp只允许使用定长文本\n",
    "# 所以对于tag什么的只能这样去写，非常的不方便，也不通用\n",
    "text = '<tag>this is text in tag </tag>'\n",
    "keyword = r'(?<=<(\\w{3})>).*(?=<\\/\\1>)|(?<=<(\\w{1})>).*(?=<\\/\\1>)'\n",
    "matchlist = getlistiterbyregex(keyword, text)\n",
    "for match in matchlist:\n",
    "    print('group 0 is: {0}'.format(match.group(0)))\n",
    "    print('group 1 is: {0}'.format(match.group(1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因为在python中两个反向界定的表达式exp只允许使用定长文本, 所以对于tag什么的只能这样去写，非常的不方便，也不通用。<br>\n",
    "但是我有办法，让其获取html的各种tag中的文本。其实就是活用分组的用法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 225,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "group 0 is: <td>this is text in tag </td>\n",
      "group 1 is: td\n",
      "group 0 is: <b>this is text in bold </b>\n",
      "group 1 is: b\n",
      "group 0 is: <div> this is in div </div>\n",
      "group 1 is: div\n"
     ]
    }
   ],
   "source": [
    "# 但是我有办法，让其获取html的各种tag中的文本\n",
    "# 这里的\\1代表获取分组编号为1的内容，即标签名称\n",
    "text = r'<td>this is text in tag </td> <b>this is text in bold </b> <div> this is in div </div>'\n",
    "keyword = r'<(?P<tagname>.*?)>(?P<text>.*?)</\\1>'\n",
    "matchlist = getlistiterbyregex(keyword, text)\n",
    "for match in matchlist:\n",
    "    print('group 0 is: {0}'.format(match.group(0)))\n",
    "    print('group 1 is: {0}'.format(match.group(1)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "group 0 is: <td>this is text in tag </td>\n",
      "group text is: this is text in tag \n",
      "group tagname is: td\n",
      "\n",
      "group 0 is: <b>this is text in bold </b>\n",
      "group text is: this is text in bold \n",
      "group tagname is: b\n",
      "\n",
      "group 0 is: <div> this is in div </div>\n",
      "group text is:  this is in div \n",
      "group tagname is: div\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 另外一种写法\n",
    "# 这里的(?P=tagname)代表获取分组名称为tagname的内容，更加直观与容易维护\n",
    "text = r'<td>this is text in tag </td> <b>this is text in bold </b> <div> this is in div </div>'\n",
    "# keyword = r'<(?P<tagname>.*?)>(?P<text>.*?)</(?P=tagname)>'\n",
    "keyword = r'<(?P<tagname>.*?)>(?P<text>.*?)</(?P=tagname)>'\n",
    "matchlist = getlistiterbyregex(keyword, text)\n",
    "for match in matchlist:\n",
    "    print('group 0 is: {0}'.format(match.group(0)))\n",
    "    print('group text is: {0}'.format(match.group('text')))\n",
    "    print('group tagname is: {0}'.format(match.group('tagname')))\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 注释\n",
    "小括号的另一种用途是通过语法(?#comment)来包含注释。例如：\n",
    "```2[0-4]\\d(?#200-249)|25[0-5](?#250-255)|[01]?\\d\\d?(?#0-199)```\n",
    "\n",
    "要包含注释的话，最好是启用\"忽略模式里的空白符\"选项，这样在编写表达式时能任意的添加空格，Tab，换行，而实际使用时这些都将被忽略。启用这个选项后，在#后面到这一行结束的所有文本都将被当成注释忽略掉。例如，我们可以前面的一个表达式写成这样："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    " (?<=          # 断言要匹配的文本的前缀\n",
    "<(\\w+)>     # 查找尖括号括起来的字母或数字(即HTML/XML标签)\n",
    ")                  # 前缀结束\n",
    ".*                # 匹配任意文本\n",
    "(?=             # 断言要匹配的文本的后缀\n",
    "<\\/\\1>       # 查找尖括号括起来的内容：前面是一个\"/\"，后面是先前捕获的标签\n",
    ")                 # 后缀结束\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "group 0 is: this is bold text \n",
      "group 1 is: b\n"
     ]
    }
   ],
   "source": [
    "text = '<b>this is bold text </b>'\n",
    "keyword = r'(?<=(?# 断言要匹配的文本的前缀)<(\\w)>(?#查找尖括号括起来的字母或数字))(?# 前缀结束).*(?# 匹配任意文本)(?=(?# 断言要匹配的文本的后缀)<\\/\\1>(?# 查找尖括号括起来的内容：前面是一个\"/\"，后面是先前捕获的标签))(?# 后缀结束)'\n",
    "matchlist = getlistiterbyregex(keyword, text)\n",
    "for match in matchlist:\n",
    "    print('group 0 is: {0}'.format(match.group(0)))\n",
    "    print('group 1 is: {0}'.format(match.group(1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 贪婪与懒惰\n",
    "当正则表达式中包含能接受重复的限定符时，通常的行为是（在使整个表达式能得到匹配的前提下）匹配尽可能多的字符。以这个表达式为例：a.*b，它将会匹配最长的以a开始，以b结束的字符串。如果用它来搜索aabab的话，它会匹配整个字符串aabab。这被称为贪婪匹配。\n",
    "\n",
    "有时，我们更需要懒惰匹配，也就是匹配尽可能少的字符。前面给出的限定符都可以被转化为懒惰匹配模式，只要在它后面加上一个问号?。这样.*?就意味着匹配任意数量的重复，但是在能使整个匹配成功的前提下使用最少的重复。现在看看懒惰版的例子吧：\n",
    "\n",
    "a.*?b匹配最短的，以a开始，以b结束的字符串。如果把它应用于aabab的话，它会匹配aab（第一到第三个字符）和ab（第四到第五个字符）。\n",
    "\n",
    "<b>注：为什么第一个匹配是aab（第一到第三个字符）而不是ab（第二到第三个字符）？简单地说，因为正则表达式有另一条规则，比懒惰／贪婪规则的优先级更高：最先开始的匹配拥有最高的优先权——The match that begins earliest wins。</b>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![懒惰限定符](./image/懒惰限定符.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['aa66ee']\n"
     ]
    }
   ],
   "source": [
    "text = r'aa66ee ddaaeeaa 6644ddee ddbbeeaa'\n",
    "keyword = r'aa\\d+ee'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['aabee', 'aaee', 'aa ccddee']\n"
     ]
    }
   ],
   "source": [
    "# aa与ee之间，可以没有字符\n",
    "keyword = r'aa.*?ee'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['aabee', 'aaeeaa ccddee']\n"
     ]
    }
   ],
   "source": [
    "# aa与ee之间至少要有一个字符\n",
    "keyword = r'aa.+?ee'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['aabee', 'aaee']\n"
     ]
    }
   ],
   "source": [
    "# aa与ee之间有0~1个字符\n",
    "keyword = r'aa.?ee'\n",
    "print(getretext(keyword, text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 处理选项\n",
    "上面介绍了几个选项如忽略大小写，处理多行等，这些选项能用来改变处理正则表达式的方式。下面是python中常用的正则表达式选项："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![修饰符](./image/修饰符.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一般来说，设置re.I其实就可以了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}