Handle HTTP 429 responses with a request backoff approach

Since it's difficult to wrap all possible go-mastodon API calls in a backoff
algorithm outside of the package itself, I decided to implement a simple
version of it in go-mastodon's doAPI itself.

This works nicely, but could be improved in two ways still:

- Abort sleeping when context gets cancelled
- Make backoff optional / configurable

Personally, I still think this is a good start and probably fits most of
go-mastodon's use-cases. It certainly beats string-grepping for status code
"429" in clients.
pull/97/head
Christian Muehlhaeuser 2018-11-26 04:26:27 +01:00 committed by mattn
parent e725c81450
commit bb2662b33c
1 changed files with 26 additions and 4 deletions

View File

@ -16,6 +16,7 @@ import (
"path/filepath" "path/filepath"
"strconv" "strconv"
"strings" "strings"
"time"
"github.com/tomnomnom/linkheader" "github.com/tomnomnom/linkheader"
) )
@ -118,12 +119,33 @@ func (c *Client) doAPI(ctx context.Context, method string, uri string, params in
req.Header.Set("Content-Type", ct) req.Header.Set("Content-Type", ct)
} }
resp, err := c.Do(req) var resp *http.Response
backoff := 1000 * time.Millisecond
for {
resp, err = c.Do(req)
if err != nil { if err != nil {
return err return err
} }
defer resp.Body.Close() defer resp.Body.Close()
// handle status code 429, which indicates the server is throttling
// our requests. Do an exponential backoff and retry the request.
if resp.StatusCode == 429 {
if backoff > time.Hour {
break
}
backoff *= 2
select {
case <-time.After(backoff):
case <-ctx.Done():
return ctx.Err()
}
continue
}
break
}
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
return parseAPIError("bad request", resp) return parseAPIError("bad request", resp)
} else if res == nil { } else if res == nil {