Hero Image
Top 25 Nginx Tips and Tricks From Practical Experience

Top 25 Nginx Tips and Tricks From Practical Experience server_tokens off; ssl_protocols TLSv1.2 TLSv1.3; Disable any undesirable HTTP methods location / { limit_except GET HEAD POST { deny all; } } Enable sysctl based protection net.ipv4.conf.all.rp_filter = 1 net.ipv4.tcp_syncookies = 1 Stop image hotlinking location /images/ { valid_referers none blocked www.domain.com domain.com; if ($invalid_referer) { return 403; } } add_header X-Content-Type-Options nosniff; add_header X-XSS-Protection "1; mode=block"; add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;

Wednesday, August 10, 2022 閱讀
Hero Image
為你的Go應用建立輕量級Docker映象？ | IT人

為你的 Go 應用建立輕量級 Docker 映象？ | IT 人 go build # default $ go build -o test1 main.go $ du -sh test1 14M test1 # 在程式編譯的時候可以加上 `-ldflags "-s -w"` 引數來優化編譯，原理是通過去除部分連結和除錯等資訊來減小編譯生成的可執行程式體積，具體引數如下： # -a：強制編譯所有依賴包 # -s：去掉符號表資訊，不過 panic 的時候 stack trace 就沒有任何檔名/行號資訊了 # -w：去掉 DWARF 除錯資訊，不過得到的程式就不能使用 gdb 進行除錯了 # 若對符號表無需求，-ldflags 直接新增 "-s" 即可 # 注：不建議-w和-s同時使用 $ go build -ldflags "-s -w" -o test2 main.go $ du -sh test2 11M test2 upx(brew/yum install upx) $ upx test2 Ultimate Packer for eXecutables Copyright (C) 1996 - 2020 UPX 3.96 Markus Oberhumer, Laszlo Molnar & John Reiser Jan 23rd 2020 File size Ratio Format Name -------------------- ------ ----------- ----------- 11490768 -> 4063248 35.36% macho/amd64 test2 Packed 1 file. $ upx --brute test2 $ du -sh test2 4.6M test2 upx 的壓縮選項

Monday, July 25, 2022 閱讀
Hero Image
[Golang] 利用build tags達到不同的build config

[Golang] 利用 build tags 達到不同的 build config Go build -ldflags package main import ( "fmt" ) var flagString string func main() { fmt.Println("This build with ldflag:", flagString) } $ go build -ldflags '-X main.flagString "test"' $ ./main This build with ldflag: test Go build -tags debug_config.go //+build debug package main var TestString string = "test debug" var TestString2 string = " and it will run every module with debug tag." func GetConfigString() string { return "it is debug....." } //+build debug 前後需要一個空行(除非你在第一行) release_config.go //+build !debug package main var TestString string = "test release" var TestString2 string = " and it will run every module as no debug tag." func GetConfigString() string { return "it is release....." } main package main import ( "fmt" ) func main() { fmt.Println("This build is ", TestString, TestString2, GetConfigString()) } $ go build -tags debug . $ ./main This build is test debug and it will run every module with debug tag. it is debug..... $ go build . $ ./main This build is test release and it will run every module as no debug tag. it is release.....

Monday, July 18, 2022 閱讀
Hero Image
Go embed 简明教程

Go embed 简明教程 嵌入 对于单个的文件，支持嵌入为string和 []byre 对于多个文件和文件夹，支持嵌入为新的文件系统 FS 比如导入 “embed” 包，即使无显式的使用 go:embed 指令用来嵌入，必须紧跟着嵌入后的变量名 只支持嵌入为 string, []byte 和 embed.FS 三种类型，这三种类型的别名(alias)和命名类型(如 type S string)都不可以 使用的是相对路径 string/[]byte package main import ( _ "embed" "fmt" ) //go:embed hello.txt var s string //go:embed hello.txt var b []byte func main() { fmt.Println(s) fmt.Println(b) } embed.FS package main import ( "embed" "fmt" ) //go:embed hello.txt hello2.txt //go:embed "p/he llo.txt" //go:embed `p/hello-2.txt` //go:embed dir var f embed.FS func main() { data, _ := f.ReadFile("hello.txt") fmt.Println(string(data)) data, _ = f.ReadFile("hello2.txt") fmt.Println(string(data)) data, _ := f.ReadFile("p/he llo.txt") fmt.Println(string(data)) data, _ = f.ReadFile("p/hello-2.txt") fmt.Println(string(data)) } 匹配模式 go:embed 指令中可以只写文件夹名，此文件夹中除了.和_开头的文件和文件夹都会被嵌入，并且子文件夹也会被递归的嵌入，形成一个此文件夹的文件系统。

Tuesday, July 12, 2022 閱讀
Hero Image
Using Vim as a Diff Tool

Using Vim as a Diff Tool command vim -d file1.txt file2.txt or vimdiff file1.txt file2.txt Switching Between Diff Windows: Ctrl-w + Ctrl-w Changing Split Window Orientation ctrl-w + K key combination to change window orientation from vertical to horizontal. ctrl-w + H shortcut to switch back window orientation to a vertical split. Jumping Between Changes ]c key combination to go to the next change. [c shortcut to jump to the previous change. Applying Changes From the Diff Window: :diffget, :diffput Export Diff to the HTML Web Page: :TOhtml | w ~/diff.html

Thursday, July 7, 2022 閱讀
Hero Image
OIDC(OpenID Connect) 简介

OIDC(OpenID Connect) 简介 Authentication vs. authorization Authentication 通常是指校验是否是用户本人的这个过程，而 Authorization 则更多的是指用户是否有权限。通常我们都是先校验 是否是用户本人，然后再校验用户是否有权限。也就是先开始 Authentication，再开始 Authorization。 Authentication Authorization Determines whether users are who they claim to be Determines what users can and cannot access Challenges the user to validate credentials (for example, through passwords, answers to security questions, or facial recognition) Verifies whether access is allowed through policies and rules Usually done before authorization Usually done after successful authentication Generally, transmits info through an ID Token Generally, transmits info through an Access Token Generally governed by the OpenID Connect (OIDC) protocol Generally governed by the OAuth 2.0 framework Example: Employees in a company are required to authenticate through the network before accessing their company email Example: After an employee successfully authenticates, the system determines what information the employees are allowed to access OAuth 2 Client Credentials Grant 这种模式是最简单的，其实就是客户端告诉服务端自己是哪个客户端，服务器就将 access_token 下发

Wednesday, July 6, 2022 閱讀
Hero Image
Bash Bitwise Operators

Bash Bitwise Operators Bash’s Tools for Base Conversion Bash’s arithmetic expansion allows us to write numbers in any base by prefixing the numeric digits of the base followed by the hash symbol. $ echo $((2#1001)) 9 bc allows us to convert effortlessly between bases, using the parameters ibase (for the input base) and obase (for the output base). So, let’s convert the decimal number 9 to base 2 $ echo "ibase=10;obase=2;9" | bc 1001

Monday, July 4, 2022 閱讀
Hero Image
Nginx 使用 split_clients 进行简易 A/B 测试

Nginx 使用 split_clients 进行简易 A/B 测试 ngx_http_split_clients_module configure 这里举例，我们想要 20% 的用户跳转到网址 https://example.com/，30% 的用户跳转到网址 https://example.org/，剩下的跳转到网址 https://examle.edu/ split_clients "${remote_addr}AAA" $variant { 20% https://example.com/; 30% https://example.org/; * https://example.edu/; } server { listen 80; listen [::]:80; server_name _; return 302 ${variant}; } 上述例子中，按照访客请求的 IP 地址 加上 AAA 字符串 会使用 MurmurHash2 转换成数字，如果得出的数字在前 20%，那么 $variant 值为 https://example.com/，相应的在中间 30% 区间的值为 https://example.org/，其他的为 https://example.edu/。 指定不同的目录 root /var/www/${variant}; 指定不同的首页 index index-${variant}.html;

Monday, July 4, 2022 閱讀
Hero Image
Monitoring HTTP Requests on a Network Interface in Real Time

Monitoring HTTP Requests on a Network Interface in Real Time tcpflow apt/dnf install tcpflow $ sudo tcpflow -p -c -i wlp0s20f3 port 80 | grep -oE '(GET|POST) .* HTTP/1.[01]|Host: .*' reportfilename: ./report.xml tcpflow: listening on wlp0s20f3 GET /alexlarsson/flatpak/ubuntu/dists/focal/InRelease HTTP/1.1 GET /mirrors.txt HTTP/1.1 -p disables promiscuous mode -c means only print the output to the console and don’t create files -i specifies the network interface grep receives the output of tcpflow -o means show only the matching parts of the lines that match the pattern -E means the pattern is an extended regular expression (ERE) httpry https://github.com/jbittel/httpry.git

Thursday, June 23, 2022 閱讀
Hero Image
golang pprof 实战

golang pprof 实战 炸弹程序 package main import ( // 略 _ "net/http/pprof" // 会自动注册 handler 到 http server，方便通过 http 接口获取程序运行采样报告 // 略 ) func main() { // 略 runtime.GOMAXPROCS(1) // 限制 CPU 使用数，避免过载 runtime.SetMutexProfileFraction(1) // 开启对锁调用的跟踪 runtime.SetBlockProfileRate(1) // 开启对阻塞操作的跟踪 go func() { // 启动一个 http server，注意 pprof 相关的 handler 已经自动注册过了 // /debug/pprof/ if err := http.ListenAndServe(":6060", nil); err != nil { log.Fatal(err) } os.Exit(0) }() // 略 } http://localhost:6060/debug/pprof/ 类型 描述 备注 allocs 内存分配情况的采样信息 可以用浏览器打开，但可读性不高 blocks 阻塞操作情况的采样信息 可以用浏览器打开，但可读性不高 cmdline 显示程序启动命令及参数 可以用浏览器打开，这里会显示 ./go-pprof-practice goroutine 当前所有协程的堆栈信息 可以用浏览器打开，但可读性不高 heap 堆上内存使用情况的采样信息 可以用浏览器打开，但可读性不高 mutex 锁争用情况的采样信息 可以用浏览器打开，但可读性不高 profile CPU 占用情况的采样信息 浏览器打开会下载文件 threadcreate 系统线程创建情况的采样信息 可以用浏览器打开，但可读性不高 trace 程序运行跟踪信息 浏览器打开会下载文件，本文不涉及，可另行参阅《深入浅出 Go trace》 排查 CPU 占用过高

Wednesday, June 22, 2022 閱讀
Hero Image
Parse Command Line Arguments in Bash

Parse Command Line Arguments in Bash getopts getopts optstring opt [arg ...] #!/bin/bash while getopts 'abc:h' opt; do case "$opt" in a) echo "Processing option 'a'" ;; b) echo "Processing option 'b'" ;; c) arg="$OPTARG" echo "Processing option 'c' with '${OPTARG}' argument" ;; ?|h) echo "Usage: $(basename $0) [-a] [-b] [-c arg]" exit 1 ;; esac done shift "$(($OPTIND -1))" optstring represents the supported options. The option expects an argument if there is a colon (:) after it. For instance, if option c expects an argument, then it would be represented as c: in the optstring When an option has an associated argument, then getopts stores the argument as a string in the OPTARG shell variable. For instance, the argument passed to option c would be stored in the OPTARG variable. opt contains the parsed option. #!/bin/bash while getopts ':abc:h' opt; do case "$opt" in a) echo "Processing option 'a'" ;; b) echo "Processing option 'b'" ;; c) arg="$OPTARG" echo "Processing option 'c' with '${OPTARG}' argument" ;; h) echo "Usage: $(basename $0) [-a] [-b] [-c arg]" exit 0 ;; :) echo -e "option requires an argument.\nUsage: $(basename $0) [-a] [-b] [-c arg]" exit 1 ;; ?) echo -e "Invalid command option.\nUsage: $(basename $0) [-a] [-b] [-c arg]" exit 1 ;; esac done shift "$(($OPTIND -1))" Note that we’ve updated optstring as well. Now it starts with the colon(:) character, which suppresses the default error message. The getopts function disables error reporting when the OPTERR variable is set to zero. Parsing Long Command-Line Options With getopt #!/bin/bash VALID_ARGS=$(getopt -o abg:d: --long alpha,beta,gamma:,delta: -- "$@") if [[ $? -ne 0 ]]; then exit 1; fi eval set -- "$VALID_ARGS" while [ : ]; do case "$1" in -a | --alpha) echo "Processing 'alpha' option" shift ;; -b | --beta) echo "Processing 'beta' option" shift ;; -g | --gamma) echo "Processing 'gamma' option. Input argument is '$2'" shift 2 ;; -d | --delta) echo "Processing 'delta' option. Input argument is '$2'" shift 2 ;; --) shift; break ;; esac done -o option represents the short command-line options --long option represents the long command-line options

Tuesday, June 7, 2022 閱讀
Hero Image
Go code refactoring : the 23x performance hunt

Go code refactoring : the 23x performance hunt $ go test -bench=. -cpuprofile cpu.prof $ go tool pprof -svg cpu.prof > cpu.svg $ go test -bench=. -trace trace.out $ go tool trace trace.out

Tuesday, June 7, 2022 閱讀