Published date: 2008/7/2 Original link: http:/ /gihyo.jp/dev/feature/01/memcached/0001 The link to this series of articles is here:
I am Nagano from the system operation team of the development department of mixi Co., Ltd. Responsible for day-to-day operations of the program. Starting today, we will focus on memcached, a hot topic recently in the field of web application scalability, in several sessions, and together with Maesaka of the research and development team of our company's development department, we will explain its internal structure and usage.
memcached is a software developed by Brad Fitzpatric of Danga Interactive, a subsidiary of LiveJournal. Now it has become an important factor in improving the scalability of web applications in many services such as mixi, hatena, Facebook, Vox, LiveJournal and so on.
Many web applications save data to an RDBMS, from which the application server reads the data and displays it in the browser. However, as the amount of data increases and access becomes concentrated, there will be significant impacts such as increased burden on the RDBMS, deterioration of database response, and delayed website display.
It’s time for memcached to show its talents. memcached is a high-performance distributed memory cache server. The general purpose of use is to reduce the number of database accesses by caching database query results to increase the speed and scalability of dynamic web applications.
Figure 1 General use of memcached
As a high-speed distributed cache server, memcached has the following characteristics.
Memcached’s server-client communication does not use complex XML and other formats, but uses a simple text line-based protocol. Therefore, data can be saved and obtained on memcached through telnet. Below are examples.
$ telnet localhost 11211 Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. set foo 0 0 3 (保存命令) bar (数据) STORED (结果) get foo (取得命令) VALUE foo 0 3 (数据) bar (数据)
The protocol documentation is located in the source code of memcached, you can also refer to the following URL.
libevent is a program library that encapsulates event processing functions such as Linux's epoll and BSD-like operating system's kqueue into a unified interface. Even if the number of connections to the server increases, O(1) performance can be achieved. Memcached uses this libevent library, so it can exert its high performance on operating systems such as Linux, BSD, and Solaris. Regarding event processing, we will not introduce it in detail here. You can refer to Dan Kegel's The C10K Problem.
In order to improve performance, the data saved in memcached is stored in the built-in memory storage space of memcached. Since data only exists in memory, restarting memcached and restarting the operating system will cause all data to disappear. In addition, after the content capacity reaches the specified value, unused caches are automatically deleted based on the LRU (Least Recently Used) algorithm. Memcached itself is a server designed for caching, so it does not consider the persistence of data too much. Regarding the detailed information of memory storage, Maesaka will introduce it after the second lecture of this series, so please refer to it at that time.
Although memcached is a "distributed" cache server, there is no distributed function on the server side. Individual memcacheds do not communicate with each other to share information. So, how to distribute it? It all depends on the client implementation. This serial will also introduce the distribution of memcached.
图2 memcached的分布式
接下来简单介绍一下memcached的使用方法。
memcached的安装比较简单,这里稍加说明。
memcached支持许多平台。
另外也能安装在Windows上。这里使用Fedora Core 8进行说明。
运行memcached需要本文开头介绍的libevent库。Fedora 8中有现成的rpm包, 通过yum命令安装即可。
$ sudo yum install libevent libevent-devel
memcached的源代码可以从memcached网站上下载。本文执笔时的最新版本为1.2.5。 Fedora 8虽然也包含了memcached的rpm,但版本比较老。因为源代码安装并不困难, 这里就不使用rpm了。
memcached安装与一般应用程序相同,configure、make、make install就行了。
$ wget http://www.danga.com/memcached/dist/memcached-1.2.5.tar.gz $ tar zxf memcached-1.2.5.tar.gz $ cd memcached-1.2.5 $ ./configure $ make $ sudo make install
默认情况下memcached安装到/usr/local/bin下。
从终端输入以下命令,启动memcached。
$ /usr/local/bin/memcached -p 11211 -m 64m -vv slab class 1: chunk size 88 perslab 11915 slab class 2: chunk size 112 perslab 9362 slab class 3: chunk size 144 perslab 7281 中间省略 slab class 38: chunk size 391224 perslab 2 slab class 39: chunk size 489032 perslab 2 <23 server listening <24 send buffer was 110592, now 268435456 <24 server listening (udp) <24 server listening (udp) <24 server listening (udp) <24 server listening (udp)
这里显示了调试信息。这样就在前台启动了memcached,监听TCP端口11211 最大内存使用量为64M。调试信息的内容大部分是关于存储的信息, 下次连载时具体说明。
作为daemon后台启动时,只需
$ /usr/local/bin/memcached -p 11211 -m 64m -d
这里使用的memcached启动选项的内容如下。
选项说明-p使用的TCP端口。默认为11211-m最大内存大小。默认为64M-vv用very vrebose模式启动,调试信息和错误输出到控制台-d作为daemon在后台启动上面四个是常用的启动选项,其他还有很多,通过
$ /usr/local/bin/memcached -h
命令可以显示。许多选项可以改变memcached的各种行为, 推荐读一读。
许多语言都实现了连接memcached的客户端,其中以Perl、PHP为主。 仅仅memcached网站上列出的语言就有
等等。
这里介绍通过mixi正在使用的Perl库链接memcached的方法。
Perl的memcached客户端有
等几个CPAN模块。这里介绍的Cache::Memcached是memcached的作者Brad Fitzpatric的作品, 应该算是memcached的客户端中应用最为广泛的模块了。
下面的源代码为通过Cache::Memcached连接刚才启动的memcached的例子。
#!/usr/bin/perl use strict; use warnings; use Cache::Memcached; my $key = "foo"; my $value = "bar"; my $expires = 3600; # 1 hour my $memcached = Cache::Memcached->new({ servers => ["127.0.0.1:11211"], compress_threshold => 10_000 }); $memcached->add($key, $value, $expires); my $ret = $memcached->get($key); print "$ret\n";
在这里,为Cache::Memcached指定了memcached服务器的IP地址和一个选项,以生成实例。 Cache::Memcached常用的选项如下所示。
选项说明servers用数组指定memcached服务器和端口compress_threshold数据压缩时使用的值namespace指定添加到键的前缀另外,Cache::Memcached通过Storable模块可以将Perl的复杂数据序列化之后再保存, 因此散列、数组、对象等都可以直接保存到memcached中。
向memcached保存数据的方法有
它们的使用方法都相同:
my $add = $memcached->add( '键', '值', '期限' ); my $replace = $memcached->replace( '键', '值', '期限' ); my $set = $memcached->set( '键', '值', '期限' );
向memcached保存数据时可以指定期限(秒)。不指定期限时,memcached按照LRU算法保存数据。 这三个方法的区别如下:
选项说明add仅当存储空间中不存在键相同的数据时才保存replace仅当存储空间中存在键相同的数据时才保存set与add和replace不同,无论何时都保存获取数据可以使用get和get_multi方法。
my $val = $memcached->get('键'); my $val = $memcached->get_multi('键1', '键2', '键3', '键4', '键5');
一次取得多条数据时使用get_multi。get_multi可以非同步地同时取得多个键值, 其速度要比循环调用get快数十倍。
删除数据使用delete方法,不过它有个独特的功能。
$memcached->delete('键', '阻塞时间(秒)');
删除第一个参数指定的键的数据。第二个参数指定一个时间值,可以禁止使用同样的键保存新数据。 此功能可以用于防止缓存数据的不完整。但是要注意,set函数忽视该阻塞,照常保存数据
可以将memcached上特定的键值作为计数器使用。
my $ret = $memcached->incr('键'); $memcached->add('键', 0) unless defined $ret;
增一和减一是原子操作,但未设置初始值时,不会自动赋成0。因此, 应当进行错误检查,必要时加入初始化操作。而且,服务器端也不会对 超过2 32时的行为进行检查。
这次简单介绍了memcached,以及它的安装方法、Perl客户端Cache::Memcached的用法。 只要知道,memcached的使用方法十分简单就足够了。
下次由前坂来说明memcached的内部结构。了解memcached的内部构造, 就能知道如何使用memcached才能使Web应用的速度更上一层楼。 欢迎继续阅读下一章。
版权声明:可以任意转载,但转载时必须标明原作者charlee、原始链接以及本声明。