java - 日均2.5个亿查询量的系统架构？

Question

负责的模块 每天要承接2.5个亿（3000QPS）的查询量（根据ID查找hbase数据 业务很简单）

目前是两台128g内存的物理服务器

ehcache单机缓存

数据存在hbase当中

还没有做压力测试

请问：
这种场景下 一般的tomcat能抗住多少QPS
我觉着两台不够 大概需要几台服务器
有什么优化的思路？

-我能想到的是
改用redis分布式缓存
tomcat jvm -gc优化
http和内核配置
代码优化
加机器不知道加几台现在

`

Answer 1

Find a stress testing tool to verify jmeter to find out.
For the SNS we did before, the performance indicator of a Tomcat application was 3000TPS.
The machine is a Huawei R3 blade machine with 16 cores.

At the same time, you can monitor the logic execution time through XRebel.

Answer 2

The QPS of 3k is not particularly high. If there is a cache, the database is fine and can handle it. It’s just that the QPS of tomcat is generally K level. Can we consider using other servers (such as nginx) to read and write cache? This reduces the load on tomcat, thus improving overall throughput.

Answer 3

Baidu’s bigtable-like storage rebuilt in C++ can achieve a single-machine random read and write of 30,000 qps. Although hbase has relatively poor performance, it should be able to achieve at least 10,000qps, so the database is not a problem.

In view of the machines and resources you mentioned, you can consider co-locating two machines, with hbase and tomcat on each machine. In addition, you can consider adjusting the number of threads appropriately based on tomcat's service mechanism and the time-consuming of your logic.

If the two machines are relatively high-end in all aspects, it is recommended to use some resource isolation and virtualization solutions

Answer 4

You don’t need tomcat, you can use something else, tomcat is a bit heavy