Testing to handle imprecise timing when canceling paginated queries

When performing database queries, we often encounter paging queries. However, when paginated queries are cancelled, timing inaccuracies can sometimes result. This issue is critical for applications that require precise timing. In this article, PHP editor Baicao will introduce to you how to deal with this problem to ensure the accuracy and precision of timing. We'll explore some possible causes and solutions to help you better understand and deal with this problem.

Question content

I have an object used to make a paginated sql query that allows the query to be run asynchronously:

type pagedquery[t any] struct {
    results   chan []*t
    errors    chan error
    done      chan error
    quit      chan error
    client    *sql.db
}

func newpagedquery[t any](client *sql.db) *pagedquery[t] {
    return &pagedquery[t]{
        results:   make(chan []*t, 1),
        errors:    make(chan error, 1),
        done:      make(chan error, 1),
        quit:      make(chan error, 1),
        client:    client,
    }
}

func (paged *pagedquery[t]) requestasync(ctx context.context, queries ...*query) {

    conn, err := client.conn(ctx)
    if err != nil {
        paged.errors <- err
        return
    }

    defer func() {
        conn.close()
        paged.done <- nil
    }()

    for i, query := range queries {
        select {
        case <-ctx.done():
            return
        case <-paged.quit:
            return
        default:
        }

        rows, err := conn.querycontext(ctx, query.string, query.arguments...)
        if err != nil {
            paged.errors <- err
            return
        }

        data, err := sql.readrows[t](rows)
        if err != nil {
            paged.errors <- err
            return
        }

        paged.results <- data
    }
}

I'm trying to test this code, specifically the cancellation part. My test code looks like this:

svc, mock := createServiceMock("TEST_DATABASE", "TEST_SCHEMA")

mock.ExpectQuery(regexp.QuoteMeta("TEST QUERY")).
    WithArgs(...).
    WillReturnRows(mock.NewRows([]string{"t", "v", "o", "c", "h", "l", "vw", "n"}))

ctx, cancel := context.WithCancel(context.Background())
go svc.requestAsync(ctx, query1, query2, query3, query4)

time.Sleep(50 * time.Millisecond)
cancel()

results := make([]data, 0)
loop:
for {
    select {
    case <-query.Done:
        break loop
    case err := <-query.Errors:
        Expect(err).ShouldNot(HaveOccurred())
    case r := <-query.Results:
        results = append(results, r...)
    }
}

Expect(results).Should(BeEmpty())
Expect(mock.ExpectationsWereMet()).ShouldNot(HaveOccurred()) // fails here

The problem I have is that this test occasionally fails at the line indicated by my comment, because when cancel() is called, there is no guarantee that when I check < <code>switch Execute at statement - ctx.done or <-exit. Execution can happen anywhere in the loop until I send the results to the results channel. But that doesn't make sense because the execution should block until I receive data from the results channel, which I won't do until after I call cancel(). Additionally, I rely on the sqlmock package for sql testing, which does not allow any kind of fuzzing of sql queries. Why do I get this glitch and how do I fix it?

Solution

My problem was caused by my own lack of understanding of go channels. I thought that by creating chan([]*t, 1) it meant that the channel would block when full (i.e. when it contains a single item), but this is not the case. Instead, when I try to send to the channel when the buffer is full, blocking occurs. So by modifying results like this:

func NewPagedQuery[T any](client *sql.DB) *PagedQuery[T] {
    return &PagedQuery[T]{
        Results:   make(chan []*T),    // Remove buffer here
        Errors:    make(chan error, 1),
        Done:      make(chan error, 1),
        Quit:      make(chan error, 1),
        client:    client,
    }
}

I can ensure that the channel blocks until the data it contains is received. This change resolves all issues with testing.

The above is the detailed content of Testing to handle imprecise timing when canceling paginated queries. For more information, please follow other related articles on the PHP Chinese website!