java - How to optimize the interface to return large amounts of data?

Question

Description of Requirement:

The peripheral platform calls the interface to query the user's song list recommendation information based on the mobile phone number. Each user will have about a thousand pieces of recommendation information. Each piece of recommendation information includes, Song ID, song name, copyright ID, Audition address field .

I need to query multiple tables. Each query takes about 4 seconds. After the query is completed, the data needs to be assembled before returning to the interface.

The return format is json. In this case, the interface return will be slower.

I thought about putting the data in the redis cluster in advance, but later rejected it because the number of users is about 5 million, and the size of recommended information for each user is about 200kb. Storing redis will consume a lot of memory, so I rejected it. But I can’t think of any other good solutions. Could you please help me find out if there are any good suggestions for how to deal with such a demand? grateful!

Answer 1

The bottleneck is that it takes 4 seconds to query many tables. Is there any point in the logic that can be optimized? If not, then these 4 seconds must be spent. With other data transmission formats, the network communication time cannot be less than 4 seconds no matter how optimized it is.
Either the client sends a recommendation request without a user being aware of it, or the query logic is optimized.

Answer 2

For linked list query, post your sql. Why not query separately? I guess you spend a lot of time in SQ

Answer 3

1. Return a thousand items at a time? Would it be faster to do 50 items at a time? What about multiple paging requests?
2. I feel that it is inappropriate to directly deny the caching solution. Not all users with more than 500 W are active users. Is redis acceptable for estimating the number of active users?
3

Answer 4

Add the ID attribute to [Recommended Information] and save it in redis. This amount should not be large.

The information recommended by each user is also stored in redis, but only 1,000 IDs of [recommended information] are saved.

In this case, each user’s recommended information will not be 200kb.