Local Storage (file-based HTTP cache)

A simple file-based cache for HTTP GET responses. Each request is keyed by the full URL (including query params); the response body is stored in a single directory. Useful for local/dev mode to avoid hitting external APIs repeatedly when data doesn’t change.

Behavior

Key: full URL (e.g. https://api.example.com/repos/foo/commits?page=1&per_page=100). No headers; URL + query params only.
Value: raw response body (bytes).
Storage: one file per URL. The key is base64-URL-encoded so it’s safe as a filename; all files live in one directory (no nested folders).
Cleanup: not built-in; rely on OS cleanup of the directory (e.g. /tmp) or delete the directory when you want to clear the cache.

Use this only when “local mode” (or similar) is enabled—e.g. you have more disk space locally but want to avoid heavy API usage in production.

Code

Put this in your own package (e.g. pkg/localstorage or internal/cache).

package localstorage

import (
	"encoding/base64"
	"fmt"
	"os"
	"path/filepath"
)

// Client caches HTTP GET response bodies keyed by full URL (with query params).
type Client struct {
	dir string
}

// New creates a client and ensures the storage directory exists.
// dir is the directory for cache files, e.g. "/tmp/myapp_cache".
func New(dir string) (*Client, error) {
	if err := os.MkdirAll(dir, 0755); err != nil {
		return nil, fmt.Errorf("localstorage: create dir %s: %w", dir, err)
	}
	return &Client{dir: dir}, nil
}

// key returns a filesystem-safe filename for the given URL.
func (c *Client) key(url string) string {
	return base64.URLEncoding.EncodeToString([]byte(url))
}

// Get returns the cached response body for url and true if found.
func (c *Client) Get(url string) ([]byte, bool) {
	path := filepath.Join(c.dir, c.key(url))
	data, err := os.ReadFile(path)
	if err != nil {
		return nil, false
	}
	return data, true
}

// Set stores the response body for url in the cache.
func (c *Client) Set(url string, body []byte) error {
	path := filepath.Join(c.dir, c.key(url))
	return os.WriteFile(path, body, 0644)
}

Wiring It Up

Config: add a flag or setting to enable “local mode” (or “use local cache”):
```
type Config struct {
    LocalMode bool   `json:"localMode"`
    // ...
}
```

Create the cache only when enabled (e.g. when building your app’s clients):

var cache *localstorage.Client
if config.LocalMode {
    var err error
    cache, err = localstorage.New("/tmp/myapp_cache")
    if err != nil {
        return nil, fmt.Errorf("local storage: %w", err)
    }
}
// cache stays nil when LocalMode is false

Pass the cache into clients that do HTTP GETs (same pattern for each client):

type MyAPIClient struct {
    baseURL string
    client  *http.Client
    cache   *localstorage.Client  // nil when local mode off
}

func NewMyAPIClient(baseURL string, cache *localstorage.Client) *MyAPIClient {
    return &MyAPIClient{baseURL: baseURL, client: &http.Client{}, cache: cache}
}

In your fetch logic: check cache first; only on miss do the request, then store success response:

Use the full URL (base URL + path + query) as the key.
Only cache successful responses (e.g. 2xx); do not cache error bodies if you want to retry later.

func (c *MyAPIClient) Get(ctx context.Context, path string) ([]byte, error) {
    url := c.baseURL + path

    if c.cache != nil {
        if body, ok := c.cache.Get(url); ok {
            return body, nil
        }
    }

    // ... rate limit here if you use one, then ...
    resp, err := c.client.Do(req)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()

    if resp.StatusCode < 200 || resp.StatusCode > 299 {
        return nil, fmt.Errorf("status %d", resp.StatusCode)
    }
    body, err := io.ReadAll(resp.Body)
    if err != nil {
        return nil, err
    }
    if c.cache != nil {
        _ = c.cache.Set(url, body)
    }
    return body, nil
}

Examples

Minimal: create and use

package main

import (
	"fmt"
	"yourproject/pkg/localstorage"
)

func main() {
	cache, err := localstorage.New("/tmp/myapp_cache")
	if err != nil {
		panic(err)
	}

	url := "https://api.example.com/users/1"
	cache.Set(url, []byte(`{"id":1,"name":"alice"}`))

	body, ok := cache.Get(url)
	if !ok {
		panic("expected cache hit")
	}
	fmt.Println(string(body))
}

In an HTTP client with cache + rate limiter

func (c *MyAPIClient) Fetch(url string) ([]byte, error) {
	// 1. Cache lookup — no rate limit
	if c.cache != nil {
		if body, ok := c.cache.Get(url); ok {
			return body, nil
		}
	}
	// 2. Rate limit only the real request
	c.limiter.Allow()

	req, _ := http.NewRequest(http.MethodGet, url, nil)
	resp, err := c.client.Do(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, fmt.Errorf("status %d", resp.StatusCode)
	}
	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, err
	}
	if c.cache != nil {
		_ = c.cache.Set(url, body)
	}
	return body, nil
}

Config-driven (enable only in local mode)

// In your bootstrap/wiring code:
func NewClients(cfg *Config) (*Clients, error) {
	var cache *localstorage.Client
	if cfg.LocalMode {
		var err error
		cache, err = localstorage.New("/tmp/myapp_cache")
		if err != nil {
			return nil, err
		}
	}

	return &Clients{
		API:   NewAPIClient(cfg.APIURL, cache),
		Other: NewOtherClient(cfg.Other, cache),
	}, nil
}

Directory choice

Linux/macOS: /tmp/myapp_cache or /tmp/myapp_local_storage is fine; OS often clears /tmp on reboot.
Windows: use os.TempDir() + subdir, e.g. filepath.Join(os.TempDir(), "myapp_cache").
Configurable: pass the directory from config so different environments can use different paths.

dir := cfg.CacheDir
if dir == "" {
    dir = filepath.Join(os.TempDir(), "myapp_cache")
}
cache, err := localstorage.New(dir)

Summary

Step	Action
1	Add the `Client`, `New`, `Get`, and `Set` code to your project.
2	Add a config flag (e.g. `LocalMode`) and create the client only when it’s true.
3	Pass the cache (or `nil`) into each HTTP client that should use it.
4	In each client: before the request, call `Get(fullURL)` and return if found.
5	After a successful response, call `Set(fullURL, body)`.
6	Use full URL (including query params) as the key; don’t cache error responses if you want to retry.