Golang Nugget - January 6, 2024

Go's concurrency, upgrade strategies, and internals of memory management. Plus, tools and tips for Gophers.

Author

Liam Manesh
January 06, 2025

Hi Gophers,


I’ve got some material for the week to keep your Go skills sharp – let’s dive into the latest nuggets! And if you enjoy it, please share it with your friends and colleagues. After all, nothing beats a friend who shares nuggets!

  1. Concurrency in Go: A Deep Dive — Hiten wrote a comprehensive post about Go’s way of handling concurrent programming. It starts from the basics of goroutines and channels and covers more advanced topics such as worker pools, sync.Cond and atomic operations. Plus, it talks about common mistakes, debugging tips, and real-world uses in web servers and microservices.

  2. Go upgrade checklist — Go 1.24 is coming out soon (check out What to expect from Go 1.24 - Part 1) and it’s a good practice to have a checklist to keep our systems updated. Hakan, a former engineer at Lyft, shares their tried-and-true checklist for upgrading Go services to new major versions. The guide covers everything from initial compatibility checks and linter updates to dealing with potential issues using tools like golangci-lint.

  3. Go Protobuf: The new Opaque API — Google’s Go team has released a new API for PB in Go which I love! If you have worked with PB in Go, you know how Open Struct API looks. A PB message gets translated to a struct with ton of pointers. Opaque API is going to change it and expose accessor methods. This let them implement lazy decoding (faster processing) and better memory usage. The best part? Some benchmarks show up to 58% better performance and 87% fewer allocations in certain scenarios.

  4. Errors, Errors Everywhere: How We Centralized and Structured Error Handling — When a codebase and team grow, you might end up in dealing with inconsistent messages in errors which makes it difficult to understand logs and debug. In this post, Oliver explains how they solved this issue by creating a centralised system for defining error codes and wrapping errors with more context.

  5. Tuning Go Application, which has GC issues with a few steps — In high throughput application, GC can become bottleneck itself. This post covers method for reducing heap allocations and reducing GC pause frequency, GC pause time and GC time per minute.

Tool Time

  1. 🏗️ github.com/Melkeydev/go-blueprint —  go-blueprint is a cool CLI tool that helps you set up Go projects with the proper structure in seconds. If you’re starting a greenfield project, take a look! You can choose from popular frameworks like Gin, Chi, or Fiber, and it supports various databases, including Postgres, MySQL, and MongoDB. There’s also a web UI at go-blueprint.dev where you can preview your project structure before generating it.

  2. 🔧 github.com/bitfield/script — The script is a Go library for doing the kind of tasks that shell scripts are good at: reading files, executing subprocesses, counting lines, matching strings, and so on. It provides a clean, pipe-based API that lets you chain operations

  3. 🔍 github.com/carapace-sh/carapace-bin — Carapace-bin is a handy command-line completion tool that works across multiple shells (Bash, Fish, Powershell, and Zsh). It’s written in Go and provides argument completion for many CLI commands.

  4. 🍦 github.com/charmbracelet/soft-serve — Soft-serve is a self-hosted Git server written in Go. It’s not something we would add as a dependency, but I think it’s fun to have some tools written in our favourite language.

🦉 Architecture Nugget is my other newsletter where I share my favourite weekly reads and the most interesting insights I’ve come across about software architecture. Give it a try—I think you’ll love it!

Architecture Nugget5-min software architecture newsletter packed with bite-sized nuggets. Trusted by engineers at Google and Amazon.

Deep Dive

Go uses a mix of stack and heap memory, with a garbage collector to handle memory cleanup automatically.

The stack is where Go stores function calls and local variables, starting small (2-4KB) and growing as needed per goroutine. The heap, on the other hand, is slower, larger, and requires GC to clean it up.

GC runs concurrently and uses a mark-and-sweep approach. It interrupts code execution during two brief “stop-the-world” phases: before marking objects for deletion and after the mark phase. The trigger is the ratio between new allocations and remaining live data after previous collections.

In this must-watch talk, Liam Hampton gives a comprehensive overview of Go memory management and how to harness its power to write highly performant and cost-efficient code.

Go’s concurrency is based on the M-P-G scheduler model. It manages goroutines (G) by scheduling them onto logical processors (P), which then run on OS threads (M). That said, you can lock/unlock a goroutine on a thread with runtime.LockOSThread() and runtime.UnlockOSThread().

If you enjoyed Liam Hampton’s talk, you might find this article interesting. It discusses the work-stealing approach, local and global queues, and the Go scheduler flow. It also covers escape analysis, where it’s decided whether a variable should go on the stack or heap.

func createCounter() *int {
    counter := 0  // This escapes to heap
    return &counter
}

This is not particularly about Go, but it’s interesting to learn about Michael Pratt’s setup and workflow, as he works on Golang. He shared how he uses jj (a Git-compatible VCS) for his contributions to Go. The Go project uses Gerrit for code review, which isn’t supported by jj.

Here’s the snippet I borrowed, as I want to give jj a try in the coming weeks. But I’d recommend you check out the post.

  • Clone the repo with colocated repository setup:

jj git clone --colocate https://go.googlesource.com/go

  • Configure your identity:

jj config set --user user.name "Your Name"
jj config set --user user.email "[email protected]"

  • Set up a custom commit message template to automatically add Change-Id lines by adding this to ~/.config/jj/config.toml:

[templates]
draft_commit_description = '''
concat(
  description,
  if(
 !description.contains("Change-Id: I"),
 "\n\n" ++ "Change-Id: I6a6a636c" ++ change_id.normal_hex() ++ "\n",
  ),
  surround(
 "\nJJ: This commit contains the following changes:\n", "",
 indent("JJ:     ", diff.summary()),
  ),
)
'''

How much did you love this read?

Login or Subscribe to participate in polls.

Reply

or to participate.