Memcache初探(二)分布式集群

要探讨memcache 的集群,那么就要先理解其特征,伪分布式。

这个要作两层理解:1.服务端数据可以进行分布式部署,但是需要依靠客户端的将数据根据算法进行分布式存储。

2.伪,上面的分布式需要依靠客户端实现,这是它第一个伪,还一个是 memcache 本身做不到可靠性,因为服务端相互之间不通信,不能同步数据,所以如果某一点挂了,就该缓存就失效了,并且如果客户端的算法是直接HASH的话,可能还会 影响后续缓存的命中。

magent

如果让客户端去实现分布式,我觉得这个会麻烦,同时而magent 这个代理就解决了memcashe上述的不足,客户端就只需要存入缓存,取缓存就够了,把一切什么麻烦的分布式,数据同步,可靠性,可用性全都交给magent 。所以一般我们需要集群方式使用memcache 的时候,一般都会增加magent 这个代理节点。当然客户端是少不了,既然提到了那也就讲一下,一般的使用memcache 的流程:

客户端(我们的应用服务) 通过memcache 提供的客户端(SDK)与memcache服务端连接,进行缓存数据的读写操作,如果需要集群,那么我们一般连接到magent,通过magent进行缓存数据读写,具体读写到哪个memcache节点,magent替我们做主,可靠性,可用性等也通过magent帮我们解决,其分布式使用.

magent wiki 给我们的简单示意图,这样memcached 就有主备,可以解决单点问题。

magent is a proxy server sitting between memcache clients(such as php programs) and memcached servers. magent have several advantages over direct connections

                Client
                  |
                Magent
                  |
    |--|--|-------------------|--|--|
    |  |  |                   |  |  |
  Normal Servers Farm      Backup Servers Farm
magent安装

1.mkdir magent 
2.cd magent/ 
3.wget http://memagent.googlecode.com/files/magent-0.5.tar.gz
4.tar zxvf magent-0.5.tar.gz 
5./sbin/ldconfig 
6.sed -i “s#LIBS = -levent#LIBS = -levent -lm#g” Makefile 
7.make 
8.cp magent /usr/bin/magent 
9.cd ../

一步步来,当然安装还是要有root 权限,有啥问题就google 吧,貌似ketama.h

的头文件需要添加以下预定义宏

#ifndef SSIZE_MAX  
#define SSIZE_MAX      32767  
#endif

算上前面一篇这样子我们就安装好了magent,memcached。我们就来个最简单的集群吧:

1.启动memcashe与magent

memcached -m 1 -u weekcashe -d -l 127.0.0.1 -p 11222

memcached -m 1 -u weekcashe -d -l 127.0.0.1 -p 11223


memcached -m 1 -u weekcashe -d -l 127.0.0.1 -p 11224
magent -u weekcashe -n 51200 -l 127.0.0.1 -p 10000 -s 127.0.0.1:11222 -s 127.0.0.1:11223 -b 127.0.0.1:11224

magent -u weekcashe -n 51200 -l 127.0.0.1 -p 11000 -s 127.0.0.1:11222 -s 127.0.0.1:11223 -b 127.0.0.1:11224

2.可用性测试

[weekcashe@iZ28gxqlqfsZ ~]$ telnet 127.0.0.1 10000 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. set key 0 0 8 888888 STORED quit Connection closed by foreign host. [weekcashe@iZ28gxqlqfsZ ~]$ telnet 127.0.0.1 11222 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. get key VALUE key 0 8 888888 END quit Connection closed by foreign host. [weekcashe@iZ28gxqlqfsZ ~]$ telnet 127.0.0.1 11223 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. get key END quit Connection closed by foreign host. [weekcashe@iZ28gxqlqfsZ ~]$ telnet 127.0.0.1 11224 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. get key VALUE key 0 8 888888 END quit Connection closed by foreign host. [weekcashe@iZ28gxqlqfsZ ~]$ telnet 127.0.0.1 11000 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. get key VALUE key 0 8 888888 END quit Connection closed by foreign host.

3.可靠性测试

[weekcashe@iZ28gxqlqfsZ ~]$ ps x
  PID TTY      STAT   TIME COMMAND
 1600 ?        Ssl    0:00 memcached -d -u weekcashe -p 11222 -c 256 -P /tmp/memcached.pid
 1615 ?        Ssl    0:00 memcached -d -u weekcashe -p 11223 -c 256 -P /tmp/memcached.pid
 1630 ?        Ssl    0:00 memcached -d -u weekcashe -p 11224 -c 256 -P /tmp/memcached.pid
 1658 ?        Ss     0:00 magent -u weekcashe -n 51200 -l 127.0.0.1 -p 10000 -s 127.0.0.1:11222 -s 127.0.0.1:11223 -b 127.0.0.1:11224
 1660 ?        Ss     0:00 magent -u weekcashe -n 51200 -l 127.0.0.1 -p 11000 -s 127.0.0.1:11222 -s 127.0.0.1:11223 -b 127.0.0.1:11224
 1726 pts/2    R+     0:00 ps x
[weekcashe@iZ28gxqlqfsZ ~]$ kill 1600
[weekcashe@iZ28gxqlqfsZ ~]$ telnet 127.0.0.1 11222
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused

[weekcashe@iZ28gxqlqfsZ ~]$ telnet 127.0.0.1 10000
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
get key
VALUE key 0 8

888888
END
quit
Connection closed by foreign host.

[weekcashe@iZ28gxqlqfsZ ~]$ memcached -d -u weekcashe -p 11222 -c 256 -P /tmp/memcached.pid
[weekcashe@iZ28gxqlqfsZ ~]$ ps x
  PID TTY      STAT   TIME COMMAND
 1365 ?        S      0:00 sshd: weekcashe@pts/0
 1366 pts/0    Ss     0:00 -bash
 1487 ?        S      0:00 sshd: weekcashe@pts/2
 1488 pts/2    Ss     0:00 -bash
 1615 ?        Ssl    0:00 memcached -d -u weekcashe -p 11223 -c 256 -P /tmp/memcached.pid
 1630 ?        Ssl    0:00 memcached -d -u weekcashe -p 11224 -c 256 -P /tmp/memcached.pid
 1658 ?        Ss     0:00 magent -u weekcashe -n 51200 -l 127.0.0.1 -p 10000 -s 127.0.0.1:11222 -s 127.0.0.1:11223 -b 127.0.0.1:11224
 1660 ?        Ss     0:00 magent -u weekcashe -n 51200 -l 127.0.0.1 -p 11000 -s 127.0.0.1:11222 -s 127.0.0.1:11223 -b 127.0.0.1:11224
 1730 ?        Ssl    0:00 memcached -d -u weekcashe -p 11222 -c 256 -P /tmp/memcached.pid
 1744 pts/2    R+     0:00 ps x

[weekcashe@iZ28gxqlqfsZ ~]$ telnet 127.0.0.1 10000
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
get key
END

我可以看出,当11222挂掉的时候,magent依然可以从备机器中取到缓存数据。

但是当11222被重新启动后,因为数据丢失,但是magent 未更新hash 对,依然命中到 11222去取数据,导致取不到数据。备机数据失效。这个问题留待解决。

你好通过 magent 端口 10000 写入的数据存储在了 memcashe 端口11222及备份memcashe 11224上,并且可以在 magent 11000 上获取到。并且设置获取数据magent 与直接在memcashe 端口上的接口是一致的。

OK,现在后台模拟集群的memcashe环境已经搭建完毕,通过magent 也已经解决了伪分布式带来的一些麻烦,那么接下就可以直接开始使用用客户端接口进行实际编码工作了。

xmemcached 客户端SDK使用了 NIO 机制,可以方便实现大并发。如果一般玩票性质就直接上 memcached client for java ,虽然其通信方式是传统的阻塞IO模型但足够稳定,不是超大型的已经够用。

Memcache初探-安装

作为一种NOSQL内存式数据库,一般用作数据缓存,目前我们公司用来作为关系型数据库的内存的缓存库,当然做过一些封装与更改,难免在使用上都与原库有所差异,所以就安装下原库试下最原始的用法

既然提了就说一下其特性:伪分布式

 

服务端安装

安装
yum -y install memcached
授权

如果你的安装用户没有root授权,那么刚才的安装会提示错误,需要以下命令授权当前用户,安装不会出错。

su
帮助

使用如下蓝色命令可以查看memcached的使用帮助,参数的作用并附带校验安装是否成功。

# memcached -h
memcached 1.4.4
-p <num>      TCP port number to listen on (default: 11211)
-U <num>      UDP port number to listen on (default: 11211, 0 is off)
-s <file>     UNIX socket path to listen on (disables network support)
-a <mask>     access mask for UNIX socket, in octal (default: 0700)
-l <ip_addr>  interface to listen on (default: INADDR_ANY, all addresses)
-d            run as a daemon
-r            maximize core file limit
-u <username> assume identity of <username> (only when run as root)
-m <num>      max memory to use for items in megabytes (default: 64 MB)
-M            return error on memory exhausted (rather than removing items)
-c <num>      max simultaneous connections (default: 1024)
-k            lock down all paged memory.  Note that there is a
              limit on how much memory you may lock.  Trying to
              allocate more than that would fail, so be sure you
              set the limit correctly for the user you started
              the daemon with (not for -u <username> user;
              under sh this is done with 'ulimit -S -l NUM_KB').
-v            verbose (print errors/warnings while in event loop)
-vv           very verbose (also print client commands/reponses)
-vvv          extremely verbose (also print internal state transitions)
-h            print this help and exit
-i            print memcached and libevent license
-P <file>     save PID in <file>, only used with -d option
-f <factor>   chunk size growth factor (default: 1.25)
-n <bytes>    minimum space allocated for key+value+flags (default: 48)
-L            Try to use large memory pages (if available). Increasing
              the memory page size could reduce the number of TLB misses
              and improve the performance. In order to get large pages
              from the OS, memcached will allocate the total item-cache
              in one large chunk.
-D <char>     Use <char> as the delimiter between key prefixes and IDs.
              This is used for per-prefix stats reporting. The default is
              ":" (colon). If this option is specified, stats collection
              is turned on automatically; if not, then it may be turned on
              by sending the "stats detail on" command to the server.
-t <num>      number of threads to use (default: 4)
-R            Maximum number of requests per event, limits the number of
              requests process for a given connection to prevent 
              starvation (default: 20)
-C            Disable use of CAS
-b            Set the backlog queue limit (default: 1024)
-B            Binding protocol - one of ascii, binary, or auto (default)
-I            Override the size of each slab page. Adjusts max item size
              (default: 1mb, min: 1k, max: 128m)
启动

根据上面的帮助信息结合自己的实际情况启动memcashed服务,-d表示守护进程方式启动。

# memcached -d -u username -c 256 -P /tmp/memcached.pid

默认方式启动

/etc/init.d/memcached start
开机启动
chkconfig --level 2345 memcached on
检测

启动完成后,检测下memcached 服务是否已经OK,由于我们启动使用了默认端口,查看默认配置可以到如下路劲文件去看

vi /etc/sysconfig/memcached

PORT="11211"
USER="memcached"
MAXCONN="1024"
CACHESIZE="64"
OPTIONS=""

我们看到默认端口是11211,那么我们使用memcashe-tool工具去查看下当前服务状态:

memcached-tool 127.0.0.1:11211 stats
#127.0.0.1:11211   Field       Value
         accepting_conns           1
               auth_cmds           0
             auth_errors           0
                   bytes           0
              bytes_read           7
           bytes_written           0
              cas_badval           0
                cas_hits           0
              cas_misses           0
               cmd_flush           0
                 cmd_get           0
                 cmd_set           0
             conn_yields           0
   connection_structures           6
        curr_connections           5
              curr_items           0
               decr_hits           0
             decr_misses           0
             delete_hits           0
           delete_misses           0
               evictions           0
                get_hits           0
              get_misses           0
               incr_hits           0
             incr_misses           0
          limit_maxbytes    67108864
     listen_disabled_num           0
                     pid        6322
            pointer_size          64
           rusage_system    0.000000
             rusage_user    0.000999
                 threads           4
                    time  1453353845
       total_connections           6
             total_items           0
                  uptime          19
                 version       1.4.4
测试示例
telnet 127.0.0.1 11211
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
set foo 0 0 3
bar
STORED
get foo
VALUE foo 0 3
bar
END
使用示例
Cache Results

function get_foo(foo_id)
    foo = memcached_get("foo:" . foo_id)
    return foo if defined foo

    foo = fetch_foo_from_database(foo_id)
    memcached_set("foo:" . foo_id, foo)
    return foo
end
关闭
kill `cat /tmp/memcached.pid`
/etc/init.d/memcached stop

 

安装其实还涉及到一个分布式集群安装,因为还涉及到具体应用就放下一章节讲吧!