Immutable Go

A year ago I landed a staff engineer role building cloud-based lending solutions in Go, and I fell hard for the language. The simplicity is intoxicating: one loop syntax, errors as structs, the elegant (value, error) two-return pattern (deal with errors now!), and a single deployable binary. It gave me C vibes, and I mean that as a compliment.

(Side note: I've shipped everything from .Net to Java to TypeScript. My first love was C, I could flow state it before LSPs and autocomplete. Go does the same, it is becoming my next C)

At work, I was tasked with performance improvements on some chatty microservices that were drowning in redundant database calls and inter-service requests. Classic gravitational pull to distributed monolith situation. So we built a request-based caching layer middleware that injects a request ID into the context, initializes a cache group for the request, then passes it through handlers → services → repositories, then cleans up the cache group at the end of the request. It has no distributed side effects, each replica data is not corrupted or stale, a very short lived cache. It works great. We're talking tens of thousands round trips savings per day!

I was letting it simmer on a few rebases and rerunning integrations tests on each merge to master to see if anything would break. Everything was looking great, then a rebase came in and broke it :( a few tests started to fail, WTF!

The Problem

The cache is in-memory and runs in-process, so the structs are shared. The following code is for example purposes only, it sux, here's the bug:

type Mortgage struct {
    ID      int
    Product *Product
}

// Repository caches a pointer to the mortgage
func (r *Repository) GetMortgage(ctx context.Context, tenant, mortgageID int) (*Mortgage, error) {
    cacheKey := fmt.Sprintf(GetMortgageCacheKey, tenant, mortgageID)
    // Try to get from cache
    cachedValue, err := r.cache.Get(ctx, cacheKey)
    if err == nil && cachedValue != nil {
        if val, ok := cachedValue.(*Mortgage); ok {
            log.G(ctx).Debug("cache hit: ", cacheKey, " method: ", "GetByID")
            return val, nil
        }
    }
    // Call decorated repository
    result, err := r.decorated.GetMortgage(ctx, tenant, mortgageID)
    if err != nil {
    	return mp1, err
    }
    // Cache the result
    r.cache.Set(ctx, cacheKey, result)
    return result, nil
}

Then in the service layer, someone mutates it:

func (m *MortgageService) GetMortgageWithoutInvestor(ctx context.Context, tenant, mortgageID int) (*Mortgage, error) {
		// we get with Investor
    mortgage, _ := m.Repository.GetMortgage(ctx, tenant, mortgageID)

    if mortgage.Product.Investor != nil {
        // do stuff
    }
    // nil the investor because the function says so
    mortgage.Product.Investor = nil  // whoops, corrupted the cache
    return mortgage
}

Next time someone calls the repository's GetMortgage and expects the investor to be there? Nope. It's nil because we mutated the cached pointer. shared ... mutable ... state. Cache corrupted.

One thing to remember, this solution needs to be drop in, general, and not intrusive to the current code. Also we did not want to introduce the complexity of distributed caching. So don't say it ...

Go Doesn't Care About Immutability

My first instinct was obvious: just dereference everything. But then you've got nested pointers, which means dereferencing all the way down the object graph, becomes a dereferencer jig. Deep copy? Sure, except:

JSON marshal/unmarshal is messy (legacy code with ignore JSON annotations means non-deterministic deep copy)
Gob is worse it doesn't handle interface{}, and pointers well and requires type registration
Deep copying complex types generically is hard.

I spent way too much time arguing with Go about this. Other languages have an option to handle Immutability. Go just... doesn't.

The Fix

We ended up with a pragmatic solution: generate a hash when setting a cache value, check it on retrieval. If the object changed, the hash changes, return an error, invalidate the key, and repopulate. Worked like a charm. No overhead with copying.

I will post blog on the actual solution used. TBD!

Conclusion

So how do you get immutability in go? The only answer is you can't, not truly!

The Real Problem (Another Post)

Look, let's be honest, the actual issue is architectural. This "microservice" is really a monolith. Proper service boundaries, less anemic shared models, minimal cross-service dependencies? That would eliminate most of this chattiness. We wouldn't be making seven calls to fetch a single domain object. We wouldn't have repositories/services scattered across layers sharing mutable state.

But that's a massive refactoring. It takes time, buy-in, and it doesn't ship performance improvements today. So we took the pragmatic path: drop in caching, detect mutations, invalidate, and move on. It's a win. This is the reality of scaling software. Architecture decisions that were fine at 10k LOC become disasters at 500k LOC. By then you're patching symptoms instead of fixing roots. That's a different post though.

Appendix Still Curious Though

Another approach to get immutable cache would be to deep copy into the cache and deep copy out of the cache, or always use value types, but the latter is not a drop in solution.

Lets benchmark different approaches to deep copying structs and see what is the overhead cost?

Wether this bench mark is optimized or 100% accurate, I don't care, it is just to give me a mental frame to measure by:

https://github.com/latebit-io/deepcopybenchmark

This was run on M1 mac, with 24 gigs of ram.

DEEPCOPY BENCHMARK COMPARISON

TOKEN (Simple struct with map):
  Manual:      223 ns/op (2.2385ms total for 10000 ops)
  Reflection:  390 ns/op (3.900209ms total for 10000 ops)
  JSON:        4944 ns/op (49.4425ms total for 10000 ops)
  GOB:         20869 ns/op (208.69125ms total for 10000 ops)

USER (Complex struct with pointers, slice, map):
  Manual:      165 ns/op (1.653459ms total for 10000 ops)
  Reflection:  285 ns/op (2.850375ms total for 10000 ops)
  JSON:        5493 ns/op (54.931125ms total for 10000 ops)
  GOB:         23838 ns/op (238.382416ms total for 10000 ops)

DEVICE (Simple struct, no pointers):
  Manual:      1 ns/op (15.459µs total for 10000 ops)
  Reflection:  155 ns/op (1.551625ms total for 10000 ops)
  JSON:        1854 ns/op (18.549ms total for 10000 ops)
  GOB:         14558 ns/op (145.588416ms total for 10000 ops)

Next steps, I may implement an immutable in memory request cache just for fun. In general I think it might be still faster than redis.

If you would like to add a comment in the comment section do a pull request on file: immutable-go.html