Testing on Ricky

Use Go Fuzzing to Write More Complete Unit Tests

Thu, 03 Nov 2022 16:57:29 +0800

Use Go Fuzzing to Write More Complete Unit Tests

func Pow(base uint, exponent uint) uint {
 if exponent == 0 {
 return 1
 }

 return base * Pow(base, exponent-1)
}

func FuzzPow(f *testing.F) {
 f.Fuzz(func(t *testing.T, x, y uint) {
 assert := assert.New(t)

 assert.Equal(Pow(x, 0), uint(1))
 assert.Equal(Pow(x, 1), x)
 assert.Equal(Pow(x, 2), x*x)

 if x > 0 && y > 0 {
 assert.Equal(Pow(x, y)/x, Pow(x, y-1))
 }
 })
}

go test -fuzz=Fuzz -fuzztime 20s

When a fuzz test fails, Go records the input in testcase/
You will find that when x=6 and y=30, the assert fails, i.e., pow(6, 30)/6 is not equal to pow(6, 29). That seems odd, but after testing you will see it is because pow(6, 30) overflows.
The max.MaxUint in Go is about 18 * 10^¹⁸, while 6^²⁹ is about 7 * 10^¹⁸. If you multiply 6^²⁹ by 6, it overflows and yields 8 * 10^¹⁸. It is like running two laps and ending up near the starting point.

var ErrOverflow = fmt.Errorf("overflow")

func Pow(base uint, exponent uint) (uint, error) {
 if exponent == 0 {
 return 1
 }

 prevResult, err := Pow(base, exponent-1)
 if math.MaxUint/base < prevResult {
 return 0, ErrOverflow
 }

 return base * prevResult, nil
}

func FuzzPow(f *testing.F) {
 f.Fuzz(func(t *testing.T, x, y uint) {
 assert := assert.New(t)
 if result, err := Pow(x, 1); err != ErrOverflow {
 assert.Equal(result, x)
 }

 if result, err := Pow(x, y); x > 0 && y > 0 && err != ErrOverflow {
 resultDivX, _ := Pow(x, y-1)
 assert.Equal(result/x, resultDivX)
 }
 })
}

Simple A/B Testing with Nginx split_clients

Mon, 04 Jul 2022 14:36:23 +0800

Simple A/B Testing with Nginx split_clients

ngx_http_split_clients_module

Configure

For example, suppose we want 20% of users to be redirected to https://example.com/, 30% to https://example.org/, and the rest to https://example.edu/.

split_clients "${remote_addr}AAA" $variant {
 20% https://example.com/;
 30% https://example.org/;
 * https://example.edu/;
}

server {
 listen 80;
 listen [::]:80;
 server_name _;

 return 302 ${variant};
}

In the example above, the visitor’s IP address plus the string AAA is hashed with MurmurHash2 into a number. If the number falls in the first 20%, $variant is https://example.com/. If it falls in the middle 30%, $variant is https://example.org/. Otherwise it is https://example.edu/.

Dockertest: A Fast Way to Set Up Integration Test Environments

Thu, 12 May 2022 16:43:31 +0800

Golang benchmarks

Thu, 02 Sep 2021 12:50:33 +0800

Golang benchmarks

Basics

Benchmarks are used for performance testing. Functions should import the testing package and define functions that start with Benchmark. The parameter type is testing.B, and the target function is called repeatedly inside the benchmark loop.

➜ go test -bench=. -run=none
goos: darwin
goarch: amd64
pkg: pkg06
cpu: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
BenchmarkFib-12 250 4682682 ns/op
PASS
ok pkg06 1.875s
➜ go test -bench=. -benchmem -run=none
goos: darwin
goarch: amd64
pkg: pkg06
cpu: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
BenchmarkFib-12 249 4686452 ns/op 0 B/op 0 allocs/op
PASS
ok pkg06 1.854s

How bench works

The benchmark function keeps running until b.N is no longer valid; it is the number of iterations.
b.N starts at 1. If the benchmark completes within 1 second (default), b.N increases and the benchmark runs again.
b.N increases in the sequence 1,2,5,10,20,50,... and the benchmark reruns.
The result above means it ran 250 times in 1 second, with each run taking 4682682 ns.
The -12 suffix relates to GOMAXPROCS. The default value is the number of CPUs visible to the Go process at startup. You can change it with -cpu, and pass multiple values to run multiple benchmarks.

Run with multiple CPU counts

➜ go test -bench=. -cpu=1,2,4 -benchmem -run=none
goos: darwin
goarch: amd64
pkg: pkg06
cpu: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
BenchmarkFib 244 4694667 ns/op 0 B/op 0 allocs/op
BenchmarkFib-2 255 4721201 ns/op 0 B/op 0 allocs/op
BenchmarkFib-4 256 4756392 ns/op 0 B/op 0 allocs/op
PASS
ok pkg06 5.826s

Run benchmarks multiple times with count

Due to CPU throttling, memory locality, background work, GC activity, etc., a single run may be noisy. It is common to run benchmarks multiple times.