Rate Limiter

A simple in-process rate limiter for outbound API calls. Use it to avoid hitting external service rate limits (e.g. 10 requests per second). It is not for limiting incoming traffic to your server.

---

Behavior

• Allows up to 10 requests to proceed as fast as possible.
• After 10, it blocks for 1 second, then resets and allows the next 10.
• Thread-safe: safe to call Allow() from multiple goroutines.
• Call Allow() only when you are about to send a real request—skip it for cache hits or other non-network work.

---

Code

Put this in your own package (e.g. pkg/ratelimit or internal/ratelimit).

package ratelimit

import (
	"sync"
	"time"
)

// RateLimiter limits outbound API calls to 10 per second: up to 10 requests
// can be sent as fast as possible, then it blocks for 1 second before allowing
// the next batch of 10.
type RateLimiter struct {
	mu    sync.Mutex
	count int
}

// Allow blocks until the caller is allowed to send a request, then returns.
// Call Allow() immediately before each API request (and after any cache check).
func (r *RateLimiter) Allow() {
	r.mu.Lock()
	if r.count >= 10 {
		r.mu.Unlock()
		time.Sleep(time.Second)
		r.mu.Lock()
		r.count = 0
	}
	r.count++
	r.mu.Unlock()
}

---

Wiring It Up

1. Add a field to the struct that makes outbound HTTP calls (e.g. your API client):

   type MyAPIClient struct {
       baseURL string
       client  *http.Client
       limiter RateLimiter  // or *RateLimiter if you share one
   }
   

2. Call Allow() only when sending a real request—after any cache lookup, so cached responses are not rate-limited:

   func (c *MyAPIClient) Get(ctx context.Context, path string) ([]byte, error) {
       url := c.baseURL + path
   
       // 1. Optional: check cache first; return without calling Allow()
       if c.cache != nil {
           if body, ok := c.cache.Get(url); ok {
               return body, nil
           }
       }
   
       // 2. Rate limit only the actual outbound request
       c.limiter.Allow()
   
       req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
       if err != nil {
           return nil, err
       }
       resp, err := c.client.Do(req)
       // ... handle response, optionally cache, return
   }
   

3. Constructor: zero value is fine; no need to initialize the limiter:

   func NewMyAPIClient(baseURL string) *MyAPIClient {
       return &MyAPIClient{
           baseURL: baseURL,
           client:  &http.Client{Timeout: 10 * time.Second},
           // limiter is a value type; zero value works
       }
   }
   

---

Examples

Basic: single client

package main

import (
	"fmt"
	"net/http"
	"yourproject/pkg/ratelimit"
)

func main() {
	limiter := ratelimit.RateLimiter{}
	client := &http.Client{}

	for i := 0; i < 15; i++ {
		limiter.Allow()
		resp, err := client.Get("https://api.example.com/items?page=" + fmt.Sprint(i))
		if err != nil {
			fmt.Println("error:", err)
			continue
		}
		resp.Body.Close()
		fmt.Println("request", i+1, resp.Status)
	}
	// First 10 return quickly; next 5 wait ~1s then go.
}

With cache (rate limit only on miss)

func (c *MyAPIClient) Fetch(url string) ([]byte, error) {
	if c.cache != nil {
		if body, ok := c.cache.Get(url); ok {
			return body, nil
		}
	}
	c.limiter.Allow()
	resp, err := c.client.Get(url)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, fmt.Errorf("unexpected status %d", resp.StatusCode)
	}
	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, err
	}
	if c.cache != nil {
		_ = c.cache.Set(url, body)
	}
	return body, nil
}

Tuning (optional)

To change the limit (e.g. 5 per second), replace the constant and sleep:

const limit = 5

func (r *RateLimiter) Allow() {
	r.mu.Lock()
	if r.count >= limit {
		r.mu.Unlock()
		time.Sleep(time.Second)
		r.mu.Lock()
		r.count = 0
	}
	r.count++
	r.mu.Unlock()
}

---

Summary

Step	Action
1	Add the RateLimiter type and Allow() to your codebase.
2	Add a RateLimiter field to the client that talks to the external API.
3	Call Allow() once per real outbound request, after any cache check.
4	Do not call Allow() for cache hits or non-network work.