使用 workerman 做TCP client 做服务器的压力测试

aidong_wang 更新于2019-10-30

物联网业务单纯测试 TCP 连接数量已经没用意义，所以需要做压力测试的时候模拟真实业务，比如有5万台设备，每分钟内都上传一次 GPS 数据等。这时候测试的不再是 workerman 框架自身的性能，而是 Server 端整体的性能。

昨天开始尝试用 workerman 作为 TCP Client 与 server 对接（server 端也是 wokerman-gatewway）,通信协议是基于 TCP 的自定义协议，下面是示例代码：

<?php

    use Protocols\LindeProtocol;
    use Workerman\Lib\Timer;
    use \Workerman\Worker;
    use \Workerman\Connection\AsyncTcpConnection;

    require_once __DIR__ . '/Autoloader.php';

    $deviceMaxNumber = 10;

    for ($i = 0; $i < $deviceMaxNumber; $i++) {

        $task = new Worker();

        $deviceId = "99999888887777$i";

        $task->onWorkerStart = function ($task) use ($deviceId) {

            require_once __DIR__ . '/SmartLinkBusinessWorker.php';
            $busWorker     = new \Workerman\SmartLinkBusinessWorker($deviceId);
            $time_interval = 1;
            Timer::add($time_interval,
                function () use ($busWorker) {
                    $busWorker->timerTick();
                }
            );

            $connection = new AsyncTcpConnection('lindeProtocol://_._._._:*);
            // 当连接建立成功时
            $connection->onConnect    = ;
            $connection->onMessage    = ;
            $connection->onClose      = ;
            $connection->onError      = ;
            $connection->onWorkerStop = ;
            $connection->connect();
        };
    }

// 运行worker
    Worker::runAll();

目前是模拟10个客户端没有问题，如果想模拟一万个客户端，能否继续使用上面的方法（只把 deviceMaxNumber 从10改成10000）？

或者能否指点一下更优的方案？

 9497  4 0

4个回答

keytehu 2018-08-06

http://wenda.workerman.net/?/question/1453
直接new 多个 AasyncTcpConnection，类似这样估计可行。

暂无评论

phpcreeper 2018-08-06

个人认为作者这个方案不是很好，因为 deviceMaxNumber 的数量直接决定了启动的子进程数量，而且每个子进程只维持了一个AsyncTcpConnection连接，随着deviceMaxNumber 数量的增加，进程启动开销随之增加，其实可以将进程分组，每组进程内维持N个链接，类似负载均衡效果。

暂无评论

aidong_wang 2018-08-07

大家试过workerman/gateway-worker 做为 server 端，能保持多少 TCP 链接吗？
我的情况：

服务器：

Centos 6.8
4核心
16G 内存
已经完全按照 Linux 内核调优参数章节进行了调优

但成功建立 TCP 链接始终不超过两千，达到一千多以后，AsyncTcpConnection的 onError 函数频繁被触发：Error code:1 msg:connect 115.159.77.248:20073 fail after 127.281 seconds

    $task = new Worker();

    $task->count = 50;

    $task->onWorkerStart = function ($task) {

        $deviceMaxNumber = 100;
        for ($i = 0; $i < $deviceMaxNumber; $i  ) {

            $taskId = $task->id;

            $deviceId = "99999$taskId" . "888887777$i";

            require_once __DIR__ . '/SmartLinkBusinessWorker.php';
            $busWorker = new \Workerman\SmartLinkBusinessWorker($deviceId);

            $connection = new AsyncTcpConnection('lindeProtocol://_._._._:*');
            // 当连接建立成功时
            $connection->onConnect    = ;
            $connection->onMessage    = ;
            $connection->onClose      = ;
            $connection->onError      = ;
            $connection->onWorkerStop = ;
            $connection->connect();

            //sleep(1);

        }
    };

现在是开启50个进程，每个进程维护100个连接。

这是 “/etc/sysctl.conf” 配置

fs.file-max=6815744
net.ipv4.tcp_max_tw_buckets = 20000
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 16384 4194304
net.ipv4.tcp_max_syn_backlog = 262144
net.core.netdev_max_backlog = 32768
net.core.somaxconn = 65535
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_fin_timeout = 20
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 2
net.ipv4.tcp_syncookies = 1
#net.ipv4.tcp_tw_len = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle=0
net.ipv4.tcp_mem = 94500000 915000000 927000000
net.ipv4.tcp_max_orphans = 3276800
net.ipv4.ip_local_port_range = 1024 65000
net.nf_conntrack_max = 6553500
net.netfilter.nf_conntrack_max = 6553500
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_established = 3600
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0

客户端连接失败时，workerman/gateway-worker 端没有触发任何错误信息，看起来状态良好

----------------------------------------------GLOBAL STATUS----------------------------------------------------
Workerman version:3.5.11          PHP version:7.1.10
start time:2018-08-06 23:08:06   run 0 days 9 hours
load average: 0, 0, 0            event-loop:\Workerman\Events\Event
4 workers       83 processes
worker_name       exit_status      exit_count
linde_test_worker 0                0
linde_gateway     0                0
internalGateway   0                0
Register          0                0
----------------------------------------------PROCESS STATUS---------------------------------------------------
pid     memory  listening                     worker_name       connections send_fail timers  total_request qps    status
13458   6M      none                          linde_test_worker 53          0         0       5538          0      
13459   6M      none                          linde_test_worker 53          0         0       11943         0      
13460   6M      none                          linde_test_worker 53          0         0       10641         0      
13461   6M      none                          linde_test_worker 53          0         0       17033         0      
13462   6M      none                          linde_test_worker 53          0         0       14599         0      
13463   6M      none                          linde_test_worker 53          0         0       6763          0      
13464   6M      none                          linde_test_worker 53          0         0       13083         0      
13466   6M      none                          linde_test_worker 53          0         0       15920         0      
13467   6M      none                          linde_test_worker 53          0         0       15398         0      
13468   6M      none                          linde_test_worker 53          0         0       15713         0      
13469   6M      none                          linde_test_worker 53          0         0       11995         0      
13470   6M      none                          linde_test_worker 53          0         0       14709         0      
13471   6M      none                          linde_test_worker 53          0         0       18411         0      
13472   6M      none                          linde_test_worker 53          0         0       18084         0      
13473   6M      none                          linde_test_worker 53          0         0       13224         0      
13474   6M      none                          linde_test_worker 53          0         0       14478         0      
13475   6M      none                          linde_test_worker 53          0         0       10551         0      
13476   6M      none                          linde_test_worker 53          0         0       16148         0      
13479   6M      none                          linde_test_worker 53          0         0       26092         0      
13480   6M      none                          linde_test_worker 53          0         0       19470         0      
13481   6M      none                          linde_test_worker 53          0         0       26773         0      
13482   6M      none                          linde_test_worker 53          0         0       19910         0      
13483   6M      none                          linde_test_worker 53          0         0       14391         0      
13484   6M      none                          linde_test_worker 53          0         0       13781         0      
13486   6M      none                          linde_test_worker 53          0         0       7992          0      
13487   6M      none                          linde_test_worker 53          0         0       6793          0      
13488   6M      none                          linde_test_worker 53          0         0       11859         0      
13489   6M      none                          linde_test_worker 53          0         0       13144         0      
13490   6M      none                          linde_test_worker 53          0         0       12420         0      
13491   6M      none                          linde_test_worker 53          0         0       16900         0      
13492   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1730          0      
13493   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1740          0      
13494   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1760          0      
13495   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1740          0      
13497   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       4302          0      
13498   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       4330          0      
13499   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       4318          0      
13500   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1740          0      
13501   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1745          0      
13502   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1735          0      
13503   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       6904          0      
13505   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       6898          0      
13506   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1735          0      
13507   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1740          0      
13509   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1735          0      
13510   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1741          0      
13511   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1735          0      
13512   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1730          0      
13513   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1755          0      
13514   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1730          0      
13515   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1746          0      
13516   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       4322          0      
13517   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       4309          0      
13518   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1735          0      
13521   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1750          0      
13522   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1745          0      
13523   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       14851         0      
13524   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       4342          0      
13525   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       4322          0      
13526   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       4325          0      
13527   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       4335          0      
13528   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       4305          0      
13529   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       1735          0      
13530   2M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       4312          0      
13531   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       12079         0      
13532   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       6929          0      
13533   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     33          0         0       43721         0      
13534   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       9568          0      
13535   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       19957         0      
13536   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     33          0         0       38502         0      
13537   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       17961         0      
13538   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       66837         0      
13539   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     32          0         0       33455         0      
13540   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       85416         0      
13541   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       32825         0      
13542   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       12241         0      
13543   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     32          0         0       151696        0      
13544   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       95544         0      
13545   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     32          0         0       193222        0      
13546   4M      lindeProtocol://0.0.0.0:20073 linde_gateway     31          0         0       17377         0      
13547   2M      text://127.0.0.1:7274         internalGateway   31          0         0       1730          0      
13548   2M      text://127.0.0.1:7274         internalGateway   31          0         0       1730          0      
13549   4M      text://0.0.0.0:1410           Register          82          0         0       82            0      
----------------------------------------------PROCESS STATUS---------------------------------------------------
Summary 330M    -                             -                 3291        0         0       1385605       0

客户端维持每10秒发送一次心跳，烦请大家帮忙看看