基本词匹配项目深入研究（二）- 分表思想的引入
关键词匹配项目深入研究（二）- 分表思想的引入
（二）分表思想的引入
近期的文章： 1）高并发数据采集的架构应用（Redis的应用）
2）高可用数据采集平台（如何玩转3门语言php+.net+aauto）
手把手教你做关键词匹配项目这块基本已经完成，深入研究是对系统的性能作为分析，在一些环境的刺激下所必需要做的一些改变。
手把手教你做关键词匹配项目: 手把手教你做关键词匹配项目（搜索引擎）---- 第一天~手把手教你做关键词匹配项目（搜索引擎）---- 第二十二天 (共22篇)
深入研究：上节讲到 关键词匹配项目深入研究-过滤器的引入。
每一篇会分为问题的前因、解决方案以及有些必要的实现方案。
本篇正文正式开始。
问题的前因
随着自动采集数据的爆炸式的增长，词库的容量蒸蒸日上，一下从几W数据猛增几百万数据，小帅帅看着数据库的查询越来越感到无能为力。
再加上小丁丁常对小帅帅说的最多的一句：何时那么选词能快一点，每次我都等好久都莫有反应，真是急死我了。
小帅帅也比较焦急，心力憔悴，真正的感觉到原来这就是挑战。小帅帅无可奈何的继续找到于老大，求于老大赏赐高招。
于老大拍拍小帅帅的肩膀：小伙子，知道项目的难度了吧！
小帅帅回答道：别挖苦我了，我已深深的感受到了，我想我心脏估计快承受不了了。
于老大：就这点你就承受不了，那估计以后有的是给你受的。
小帅帅：大哥，别说这些虚的行不，赶紧的解决方案丫。
于老大：急啥，事情是急不来的，过来，哥给你指条明路。
“每个宝贝是不是有类别的属性，那么这几百万数据真正属于这个类别的词能够有多少？假设我们只取这个类别的词库我们的项目是否可以继续稳定下来”。
解决方案
按照某种业务需要，我们可以对数据表实行分割，可以纵向或者横向分割，可以有效的进行性能优化。
纵向分割也称列分割，把不常用的列或者长字段分割来保证实体处于一个相对适用的状态，常见的有一对一关联。
横向分割也称行分割，按照某种业务拆分数据的记录来存放在不同的表，常见的有按日期分表操作。
本案例是使用横向分割，把数据按照类别的形式进行拆分。
实现方案
我们为了不更改数据表的结构，这样设计了，我们按照表名来区分项目使用那个数据表。这样一来的改动相对是非常少的。我们只需稍微改动下代码就可以解决了，这很心塞的一件事情。
修改Keyword的代码，增加获取数据源。
<span style="color: #000000;">php</span><span style="color: #008080;">define</span>('DATABASE_HOST','127.0.0.1'<span style="color: #000000;">);</span><span style="color: #008080;">define</span>('DATABASE_USER','xiaoshuaishuai'<span style="color: #000000;">);</span><span style="color: #008080;">define</span>('DATABASE__PASSWORD','xiaoshuaishuai'<span style="color: #000000;">);</span><span style="color: #008080;">define</span>('DATABASE_CHARSET','utf-8'<span style="color: #000000;">);</span><span style="color: #0000ff;">class</span><span style="color: #000000;"> Keyword { </span><span style="color: #0000ff;">public</span> <span style="color: #800080;">$word</span><span style="color: #000000;">; </span><span style="color: #0000ff;">public</span> <span style="color: #0000ff;">static</span> <span style="color: #800080;">$conn</span> = <span style="color: #0000ff;">null</span><span style="color: #000000;">; </span><span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span><span style="color: #000000;"> getDbConn(){ </span><span style="color: #0000ff;">if</span>(self::<span style="color: #800080;">$conn</span> == <span style="color: #0000ff;">null</span><span style="color: #000000;">){ self</span>::<span style="color: #800080;">$conn</span> = <span style="color: #008080;">mysql_connect</span>(DATABASE_HOST,DATABASE_USER,<span style="color: #000000;">DATABASE__PASSWORD); </span><span style="color: #008080;">mysql_query</span>("SET NAMES '".DATABASE_CHARSET."'",self::<span style="color: #800080;">$conn</span><span style="color: #000000;">); </span><span style="color: #008080;">mysql_select_db</span>("dict",self::<span style="color: #800080;">$conn</span><span style="color: #000000;">); </span><span style="color: #0000ff;">return</span> self::<span style="color: #800080;">$conn</span><span style="color: #000000;">; } </span><span style="color: #0000ff;">return</span> self::<span style="color: #800080;">$conn</span><span style="color: #000000;">; } </span><span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span><span style="color: #000000;"> save(){ </span><span style="color: #800080;">$sql</span> = "insert into keywords(word) values ('<span style="color: #800080;">$this</span>->word')"<span style="color: #000000;">; </span><span style="color: #0000ff;">return</span> <span style="color: #008080;">mysql_query</span>(<span style="color: #800080;">$sql</span>,<span style="color: #800080;">$this</span>-><span style="color: #000000;">getDbConn()); } </span><span style="color: #0000ff;">public</span> <span style="color: #0000ff;">static</span> <span style="color: #0000ff;">function</span> getWordsSource(<span style="color: #800080;">$cid</span>,<span style="color: #800080;">$limit</span>=0,<span style="color: #800080;">$offset</span>=40<span style="color: #000000;">){ </span><span style="color: #800080;">$sql</span> = "SELECT * FROM keywords_<span style="color: #800080;">$cid</span> LIMIT <span style="color: #800080;">$limit</span>,<span style="color: #800080;">$ffset</span>"<span style="color: #000000;">; </span><span style="color: #0000ff;">return</span> DB::MakeArray(<span style="color: #800080;">$sql</span><span style="color: #000000;">); } </span><span style="color: #0000ff;">public</span> <span style="color: #0000ff;">static</span> <span style="color: #0000ff;">function</span> getWordsCount(<span style="color: #800080;">$cid</span><span style="color: #000000;">){ </span><span style="color: #800080;">$sql</span> = "SELECT count(*) FROM keywords_<span style="color: #800080;">$cid</span>"<span style="color: #000000;">; </span><span style="color: #0000ff;">return</span> DB::QueryScalar(<span style="color: #800080;">$sql</span><span style="color: #000000;">); }}</span>DB类新增QueryScalar，用于算总量
<span style="color: #000000;">php</span><span style="color: #008000;">#</span><span style="color: #008000;">@author oShine</span><span style="color: #008080;">define</span>('DATABASE_HOST','127.0.0.1'<span style="color: #000000;">);</span><span style="color: #008080;">define</span>('DATABASE_USER','xiaoshuaishuai'<span style="color: #000000;">);</span><span style="color: #008080;">define</span>('DATABASE__PASSWORD','xiaoshuaishuai'<span style="color: #000000;">);</span><span style="color: #008080;">define</span>('DATABASE_CHARSET','utf-8'<span style="color: #000000;">);</span><span style="color: #0000ff;">class</span><span style="color: #000000;"> DB { </span><span style="color: #0000ff;">public</span> <span style="color: #0000ff;">static</span> <span style="color: #800080;">$conn</span> = <span style="color: #0000ff;">null</span><span style="color: #000000;">; </span><span style="color: #0000ff;">public</span> <span style="color: #0000ff;">static</span> <span style="color: #0000ff;">function</span><span style="color: #000000;"> Connect(){ </span><span style="color: #0000ff;">if</span>(self::<span style="color: #800080;">$conn</span> == <span style="color: #0000ff;">null</span><span style="color: #000000;">){ self</span>::<span style="color: #800080;">$conn</span> = <span style="color: #008080;">mysql_connect</span>(DATABASE_HOST,DATABASE_USER,<span style="color: #000000;">DATABASE__PASSWORD); </span><span style="color: #008080;">mysql_query</span>("SET NAMES '".DATABASE_CHARSET."'",self::<span style="color: #800080;">$conn</span><span style="color: #000000;">); </span><span style="color: #008080;">mysql_select_db</span>("dict",self::<span style="color: #800080;">$conn</span><span style="color: #000000;">); </span><span style="color: #0000ff;">return</span> self::<span style="color: #800080;">$conn</span><span style="color: #000000;">; } </span><span style="color: #0000ff;">return</span> self::<span style="color: #800080;">$conn</span><span style="color: #000000;">; } </span><span style="color: #0000ff;">public</span> <span style="color: #0000ff;">static</span> <span style="color: #0000ff;">function</span> Query(<span style="color: #800080;">$sql</span><span style="color: #000000;">){ </span><span style="color: #0000ff;">return</span> <span style="color: #008080;">mysql_query</span>(<span style="color: #800080;">$sql</span>,self::<span style="color: #000000;">Connect()); } </span><span style="color: #0000ff;">public</span> <span style="color: #0000ff;">static</span> <span style="color: #0000ff;">function</span> makeArray(<span style="color: #800080;">$sql</span><span style="color: #000000;">){ </span><span style="color: #800080;">$rs</span> = self::Query(<span style="color: #800080;">$sql</span><span style="color: #000000;">); </span><span style="color: #800080;">$result</span> = <span style="color: #0000ff;">array</span><span style="color: #000000;">(); </span><span style="color: #0000ff;">while</span>(<span style="color: #800080;">$data</span> = <span style="color: #008080;">mysql_fetch_assoc</span>(<span style="color: #800080;">$rs</span><span style="color: #000000;">)){ </span><span style="color: #800080;">$result</span>[] = <span style="color: #800080;">$data</span><span style="color: #000000;">; } </span><span style="color: #0000ff;">return</span> <span style="color: #800080;">$result</span><span style="color: #000000;">; } </span><span style="color: #0000ff;">public</span> <span style="color: #0000ff;">static</span> <span style="color: #0000ff;">function</span> QueryScalar(<span style="color: #800080;">$sql</span><span style="color: #000000;">){ </span><span style="color: #800080;">$rs</span> = self::Query(<span style="color: #800080;">$sql</span><span style="color: #000000;">); </span><span style="color: #800080;">$data</span> = <span style="color: #008080;">mysql_fetch_array</span>(<span style="color: #800080;">$rs</span><span style="color: #000000;">); </span><span style="color: #0000ff;">if</span>(<span style="color: #800080;">$data</span> == <span style="color: #0000ff;">false</span> || <span style="color: #0000ff;">empty</span>(<span style="color: #800080;">$data</span>) || !<span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$data</span>[1])) <span style="color: #0000ff;">return</span> 0<span style="color: #000000;">; </span><span style="color: #0000ff;">return</span> <span style="color: #800080;">$data</span>[1<span style="color: #000000;">]; }} </span>修改Selector的代码，用于选词：
<span style="color: #000000;">php</span><span style="color: #008000;">#</span><span style="color: #008000;">@Filename:selector/Selector.php</span><span style="color: #008000;">#</span><span style="color: #008000;">@Author:oshine</span><span style="color: #0000ff;">require_once</span> <span style="color: #008080;">dirname</span>(<span style="color: #ff00ff;">__FILE__</span>) . '/SelectorItem.php'<span style="color: #000000;">;</span><span style="color: #0000ff;">require_once</span> <span style="color: #008080;">dirname</span>(<span style="color: #ff00ff;">__FILE__</span>) . '/charlist/CharList.php'<span style="color: #000000;">;</span><span style="color: #0000ff;">require_once</span> <span style="color: #008080;">dirname</span>(<span style="color: #ff00ff;">__FILE__</span>) . '/charlist/CharlistHandle.php'<span style="color: #000000;">;</span><span style="color: #0000ff;">require_once</span> <span style="color: #008080;">dirname</span>(<span style="color: #008080;">dirname</span>(<span style="color: #ff00ff;">__FILE__</span>)) . '/lib/Logger.php'<span style="color: #000000;">;</span><span style="color: #0000ff;">class</span><span style="color: #000000;"> Selector{ </span><span style="color: #0000ff;">private</span> <span style="color: #0000ff;">static</span> <span style="color: #800080;">$charListHandle</span> = <span style="color: #0000ff;">array</span><span style="color: #000000;">( </span>"黑名单" => "BacklistCharListHandle", "近义词" => "LinklistCharListHandle"<span style="color: #000000;"> ); </span><span style="color: #0000ff;">public</span> <span style="color: #0000ff;">static</span> <span style="color: #0000ff;">function</span> select(<span style="color: #800080;">$num_iid</span><span style="color: #000000;">) { </span><span style="color: #800080;">$selectorItem</span> = SelectorItem::createFromApi(<span style="color: #800080;">$num_iid</span><span style="color: #000000;">); Logger</span>::trace(<span style="color: #800080;">$selectorItem</span>-><span style="color: #000000;">props_name); </span><span style="color: #800080;">$charlist</span> = <span style="color: #0000ff;">new</span><span style="color: #000000;"> CharList(); </span><span style="color: #0000ff;">foreach</span> (self::<span style="color: #800080;">$charListHandle</span> <span style="color: #0000ff;">as</span> <span style="color: #800080;">$matchKey</span> => <span style="color: #800080;">$className</span><span style="color: #000000;">) { </span><span style="color: #800080;">$handle</span> = self::createCharListHandle(<span style="color: #800080;">$className</span>, <span style="color: #800080;">$charlist</span>, <span style="color: #800080;">$selectorItem</span><span style="color: #000000;">); </span><span style="color: #800080;">$handle</span>-><span style="color: #008080;">exec</span><span style="color: #000000;">(); } </span><span style="color: #800080;">$selectWords</span> = <span style="color: #0000ff;">array</span><span style="color: #000000;">(); </span><span style="color: #800080;">$wordsCount</span> = Keyword::getWordsCount(selectorItem-><span style="color: #000000;">cid); </span><span style="color: #800080;">$offset</span> = 40<span style="color: #000000;">; </span><span style="color: #800080;">$page</span> = <span style="color: #008080;">ceil</span>(<span style="color: #800080;">$wordsCount</span>/<span style="color: #800080;">$offset</span><span style="color: #000000;">); </span><span style="color: #0000ff;">for</span>(<span style="color: #800080;">$i</span>=0;<span style="color: #800080;">$i</span>$page;<span style="color: #800080;">$i</span>++<span style="color: #000000;">){ </span><span style="color: #800080;">$limit</span> = <span style="color: #800080;">$i</span>*<span style="color: #800080;">$offset</span><span style="color: #000000;">; </span><span style="color: #800080;">$keywords</span> = Keyword::getWordsSource(selectorItem->cid,<span style="color: #800080;">$limit</span>,<span style="color: #800080;">$offset</span><span style="color: #000000;">); </span><span style="color: #0000ff;">foreach</span> (<span style="color: #800080;">$keywords</span> <span style="color: #0000ff;">as</span> <span style="color: #800080;">$val</span><span style="color: #000000;">) { </span><span style="color: #008000;">#</span><span style="color: #008000;"> code...</span> <span style="color: #800080;">$keywordEntity</span> = SplitterApp::<span style="color: #008080;">split</span>(<span style="color: #800080;">$val</span>["word"<span style="color: #000000;">]); </span><span style="color: #008000;">#</span><span style="color: #008000;"> code...</span> <span style="color: #0000ff;">if</span>(MacthExector::macth(<span style="color: #800080;">$keywordEntity</span>,<span style="color: #800080;">$charlist</span><span style="color: #000000;">)){ </span><span style="color: #800080;">$selectWords</span>[] = <span style="color: #800080;">$val</span>["word"<span style="color: #000000;">]; } } } </span><span style="color: #0000ff;">return</span> <span style="color: #800080;">$selectWords</span><span style="color: #000000;">; } </span><span style="color: #0000ff;">public</span> <span style="color: #0000ff;">static</span> <span style="color: #0000ff;">function</span> createCharListHandle(<span style="color: #800080;">$className</span>, <span style="color: #800080;">$charlist</span>, <span style="color: #800080;">$selectorItem</span><span style="color: #000000;">) { </span><span style="color: #0000ff;">if</span> (<span style="color: #008080;">class_exists</span>(<span style="color: #800080;">$className</span><span style="color: #000000;">)) { </span><span style="color: #0000ff;">return</span> <span style="color: #0000ff;">new</span> <span style="color: #800080;">$className</span>(<span style="color: #800080;">$charlist</span>, <span style="color: #800080;">$selectorItem</span><span style="color: #000000;">); } </span><span style="color: #0000ff;">throw</span> <span style="color: #0000ff;">new</span> <span style="color: #0000ff;">Exception</span>("class not exists", 0<span style="color: #000000;">); }}</span>总结
小帅帅又学到了新的知识点，这是要犒劳于老大的节奏吗？你们是否也要犒劳下我呢，求赞哈！
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
1387
52
Solution: Your organization requires you to change your PIN
Oct 04, 2023 pm 05:45 PM
The message "Your organization has asked you to change your PIN" will appear on the login screen. This happens when the PIN expiration limit is reached on a computer using organization-based account settings, where they have control over personal devices. However, if you set up Windows using a personal account, the error message should ideally not appear. Although this is not always the case. Most users who encounter errors report using their personal accounts. Why does my organization ask me to change my PIN on Windows 11? It's possible that your account is associated with an organization, and your primary approach should be to verify this. Contacting your domain administrator can help! Additionally, misconfigured local policy settings or incorrect registry keys can cause errors. Right now
How to adjust window border settings on Windows 11: Change color and size
Sep 22, 2023 am 11:37 AM
Windows 11 brings fresh and elegant design to the forefront; the modern interface allows you to personalize and change the finest details, such as window borders. In this guide, we'll discuss step-by-step instructions to help you create an environment that reflects your style in the Windows operating system. How to change window border settings? Press + to open the Settings app. WindowsI go to Personalization and click Color Settings. Color Change Window Borders Settings Window 11" Width="643" Height="500" > Find the Show accent color on title bar and window borders option, and toggle the switch next to it. To display accent colors on the Start menu and taskbar To display the theme color on the Start menu and taskbar, turn on Show theme on the Start menu and taskbar
How to change title bar color on Windows 11?
Sep 14, 2023 pm 03:33 PM
By default, the title bar color on Windows 11 depends on the dark/light theme you choose. However, you can change it to any color you want. In this guide, we'll discuss step-by-step instructions for three ways to change it and personalize your desktop experience to make it visually appealing. Is it possible to change the title bar color of active and inactive windows? Yes, you can change the title bar color of active windows using the Settings app, or you can change the title bar color of inactive windows using Registry Editor. To learn these steps, go to the next section. How to change title bar color in Windows 11? 1. Using the Settings app press + to open the settings window. WindowsI go to "Personalization" and then
OOBELANGUAGE Error Problems in Windows 11/10 Repair
Jul 16, 2023 pm 03:29 PM
Do you see "A problem occurred" along with the "OOBELANGUAGE" statement on the Windows Installer page? The installation of Windows sometimes stops due to such errors. OOBE means out-of-the-box experience. As the error message indicates, this is an issue related to OOBE language selection. There is nothing to worry about, you can solve this problem with nifty registry editing from the OOBE screen itself. Quick Fix – 1. Click the “Retry” button at the bottom of the OOBE app. This will continue the process without further hiccups. 2. Use the power button to force shut down the system. After the system restarts, OOBE should continue. 3. Disconnect the system from the Internet. Complete all aspects of OOBE in offline mode
How to enable or disable taskbar thumbnail previews on Windows 11
Sep 15, 2023 pm 03:57 PM
Taskbar thumbnails can be fun, but they can also be distracting or annoying. Considering how often you hover over this area, you may have inadvertently closed important windows a few times. Another disadvantage is that it uses more system resources, so if you've been looking for a way to be more resource efficient, we'll show you how to disable it. However, if your hardware specs can handle it and you like the preview, you can enable it. How to enable taskbar thumbnail preview in Windows 11? 1. Using the Settings app tap the key and click Settings. Windows click System and select About. Click Advanced system settings. Navigate to the Advanced tab and select Settings under Performance. Select "Visual Effects"
Display scaling guide on Windows 11
Sep 19, 2023 pm 06:45 PM
We all have different preferences when it comes to display scaling on Windows 11. Some people like big icons, some like small icons. However, we all agree that having the right scaling is important. Poor font scaling or over-scaling of images can be a real productivity killer when working, so you need to know how to customize it to get the most out of your system's capabilities. Advantages of Custom Zoom: This is a useful feature for people who have difficulty reading text on the screen. It helps you see more on the screen at one time. You can create custom extension profiles that apply only to certain monitors and applications. Can help improve the performance of low-end hardware. It gives you more control over what's on your screen. How to use Windows 11
10 Ways to Adjust Brightness on Windows 11
Dec 18, 2023 pm 02:21 PM
Screen brightness is an integral part of using modern computing devices, especially when you look at the screen for long periods of time. It helps you reduce eye strain, improve legibility, and view content easily and efficiently. However, depending on your settings, it can sometimes be difficult to manage brightness, especially on Windows 11 with the new UI changes. If you're having trouble adjusting brightness, here are all the ways to manage brightness on Windows 11. How to Change Brightness on Windows 11 [10 Ways Explained] Single monitor users can use the following methods to adjust brightness on Windows 11. This includes desktop systems using a single monitor as well as laptops. let's start. Method 1: Use the Action Center The Action Center is accessible
How to Fix Activation Error Code 0xc004f069 in Windows Server
Jul 22, 2023 am 09:49 AM
The activation process on Windows sometimes takes a sudden turn to display an error message containing this error code 0xc004f069. Although the activation process is online, some older systems running Windows Server may experience this issue. Go through these initial checks, and if they don't help you activate your system, jump to the main solution to resolve the issue. Workaround – close the error message and activation window. Then restart the computer. Retry the Windows activation process from scratch again. Fix 1 – Activate from Terminal Activate Windows Server Edition system from cmd terminal. Stage – 1 Check Windows Server Version You have to check which type of W you are using
