Hero Image
Fix Nginx 500 errors (too many open files, connection)

Fix Nginx 500 errors (too many open files, connection) Nginx 500 errors only show up in logs. Two common cases: socket() failed (24: Too many open files) while connecting to upstream $ sudo su - www-data $ ulimit -n # check current limit (ulimit -a shows all params) 1024 # vim /etc/security/limits.conf # set nofile (max number of open files) # add/modify the following two lines * soft nofile 655360 * hard nofile 655360 ulimit -n # log out and log back in to see the new value 655360 # If ulimit -n is not 655360, run ulimit -n 655360 to force set it # Then verify with ulimit -n or ulimit -Sn (soft) and ulimit -Hn (hard) (or ulimit -a). # Calculate and set from system level lsof | wc -l # count open files sudo vim /etc/sysctl.conf fs.file-max = 3268890 sudo sysctl -p 512 worker_connections are not enough while connecting to upstream # /etc/nginx/nginx.conf worker_connections 10240; # Refer to Nginx CoreModule # worker_processes 2; # worker_rlimit_nofile 10240; # events { # # worker_connections 10240; # } # Increasing Nginx connections can slow down overall speed because php-cgi is not enough. # Adjust as follows. # php-cgi was started with phpfcgid_children="10" and phpfcgid_requests="500" # ab was run on another server, connect via a switch using GBit ethernet # http://till.klampaeckel.de/blog/archives/30-PHP-performance-III-Running-nginx.html # vim /etc/nginx/nginx.conf worker_connections 10240; worker_rlimit_nofile # vim /etc/init.d/php-fcgi PHP_FCGI_CHILDREN=15 PHP_FCGI_MAX_REQUESTS=1000 change to PHP_FCGI_CHILDREN=512 # or 150 and increase gradually, watch MySQL connections PHP_FCGI_MAX_REQUESTS=10240 # The article's phpfcgid_stop() function is good and can be used if needed. # phpfcgid_stop() { # echo "Stopping $name." # pids=`pgrep php-cgi` # pkill php-cgi # wait_for_pids $pids # }

Hero Image
Gitlab-CI Introduction

Gitlab CI Concept Gitlab DevOps GitOps Workflow code push -> pipeline -> stage -> job Design plan -> code -> build -> test -> release -> deploy -> operate -> monitor -> plan Runner Executors Shell VirtualBox Docker Docker Machine Kubernetes Else… References Gitlab CI/CD Gitlab Runner .gitlab-ci.yaml Runner Register gitlab-runner register After register concurrent = 1 check_interval = 0 [session_server] session_timeout = 1800 [[runners]] name = "public-shell" url = "https://gitlab.go2cloudten.com/" token = "-mdH9OAOzG5yPsf_AVnW" executor = "shell" [[runners]] name = "public-docker" url = "https://gitlab.go2cloudten.com/" token = "AcEGPPKTS1uuQ_A_qpWy" executor = "docker" [runners.docker] dns = ["192.168.185.5", "192.168.185.6"] tls_verify = false image = "registry.go2cloudten.com/it/office_sop/node:12.13.0" privileged = true disable_entrypoint_overwrite = false oom_kill_disable = false disable_cache = false shm_size = 0 pull_policy = "if-not-present" volumes = ["/cache"] Repository .gitlab-ci.yaml stages: - domain check-icp: stage: domain image: registry.go2cloudten.com/it/office_sop/icp tags: - docker script: - domains=$(awk -F '|' '{if($6 ~ "Y" && ($7 ~ "West" || $7 ~ "Yuqu")) print $3}' domains-info.md | sed 's/ //g' | sort | uniq) - if [[ "${domains}" == "" ]]; then telegram.sh 'There is no domain in list' ; else telegram.sh 'Start checking ICP.' ; fi - for i in ${domains}; do result=$(checkicp ${i}); if [[ "${result}" == "未备案" ]];then telegram.sh "${i} 未备案"; sleep 1 ;fi;done - telegram.sh 'ICP check completed.' only: - schedules

Hero Image
Docker Introduction

Docker Concept VM vs Container VM - Base on OS Container - Base on Application (Linux Kernel: Namespace and Cgroup) Client to Server Docker daemon - containerd, docker-containerd-shim, docker-runc Docker client - cli command docker cli -> docker daemon -> containerd -> runc -> namespace & cgroup Image Snapshots Container Read-Only processes on image Hub / Registry Store images References Docker —— 從入門到實踐 docker docs Docker commands Dockerfile ARG dist="/tmp/password" ARG projectDir="/password" FROM golang:1.16-alpine3.14 AS builder RUN apk add build-base upx ARG dist ARG projectDir WORKDIR ${projectDir} COPY . . RUN go build -trimpath -o main cmd/main.go RUN upx -9 -o ${dist} main FROM scratch ARG dist ENV TZ=Asia/Taipei COPY --from=builder ${dist} /usr/local/bin/password Dockerfile1 FROM alpine CMD ["nc","-l","12345"] Dockerfile2 FROM alpine CMD ["echo","DOCKER"] docker build command docker build . -t program docker build . -f Dockerfile -t test_mysql docker build . -t hello:v1.1 --build-arg dist=/tmp/hello --build-arg projectDir=/hello docker build . docker/status echo -e "${GREEN}Before build${RESET}" docker image ls docker build . -f docker/Dockerfile1 -t test1 docker build . -f docker/Dockerfile2 -t test2 docker image . docker/status echo -e "${GREEN}After build${RESET}" docker image ls docker run AND rm . docker/status echo -e "${GREEN}Run container1${RESET}" docker run -d --name container1 test1 echo -e "${GREEN}Run container2${RESET}" docker run -d --name container2 test2 echo -e "${GREEN}List alive containers${RESET}" docker ps echo -e "${GREEN}List all containers${RESET}" docker ps -a echo -e "${GREEN}Remove alive container${RESET}" docker rm -f container1 echo -e "${GREEN}List all containers${RESET}" docker ps -a echo -e "${GREEN}Remove exit container${RESET}" docker rm container2 echo -e "${GREEN}List all containers${RESET}" docker ps -a docker pull AND rmi . docker/status echo -e "${GREEN}List all image${RESET}" docker image ls echo -e "${GREEN}Pull alpine image${RESET}" docker pull alpine echo -e "${GREEN}List all image${RESET}" docker image ls docker rmi . docker/status echo -e "${GREEN}Remove alpine image${RESET}" docker rmi alpine echo -e "${GREEN}List all image${RESET}" docker image ls prune docker system prune -f --volumes docker history . docker/status echo -e "${GREEN}History of test1${RESET}" docker history test1 echo -e "${GREEN}History of mysql:8${RESET}" docker history mysql:8 Docker remote Edit service file # /lib/systemd/system/docker.service ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock -H tcp://0.0.0.0:2375 Restart service systemctl daemon-reload systemctl restart docker Specify DOCKER_HOST . docker/status echo -e "${GREEN}List images on 192.168.185.9${RESET}" DOCKER_HOST=192.168.185.9:2375 docker images Docker-compose version: "3" services: svn: image: zeyanlin/svn environment: - LDAP_HOSTS=${LDAP_HOSTS} - LDAP_BASE_DN=${LDAP_BASE_DN} - LDAP_BIND_DN=${LDAP_BIND_DN} - LDAP_ADMIN_PASS=${LDAP_ADMIN_PASS} ports: - 8000:80 - 3690:3690 depends_on: - ldap ldap: image: zeyanlin/openldap environment: - LDAP_DOMAIN=${LDAP_DOMAIN} - LDAP_ADMIN_PASS=${LDAP_ADMIN_PASS} ports: - 389:389 - 636:636 php: image: zeyanlin/phpldapadmin environment: - LDAP_HOSTS=${LDAP_HOSTS} ports: - 80:80 depends_on: - ldap Env LDAP_HOSTS=ldap LDAP_DOMAIN="knowhow.fun" LDAP_BASE_DN="dc=knowhow,dc=fun" LDAP_BIND_DN="cn=admin" LDAP_ADMIN_PASS="123qwe"

Hero Image
Golang benchmarks

Golang benchmarks Basics Benchmarks are used for performance testing. Functions should import the testing package and define functions that start with Benchmark. The parameter type is testing.B, and the target function is called repeatedly inside the benchmark loop. ➜ go test -bench=. -run=none goos: darwin goarch: amd64 pkg: pkg06 cpu: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz BenchmarkFib-12 250 4682682 ns/op PASS ok pkg06 1.875s ➜ go test -bench=. -benchmem -run=none goos: darwin goarch: amd64 pkg: pkg06 cpu: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz BenchmarkFib-12 249 4686452 ns/op 0 B/op 0 allocs/op PASS ok pkg06 1.854s How bench works The benchmark function keeps running until b.N is no longer valid; it is the number of iterations. b.N starts at 1. If the benchmark completes within 1 second (default), b.N increases and the benchmark runs again. b.N increases in the sequence 1,2,5,10,20,50,... and the benchmark reruns. The result above means it ran 250 times in 1 second, with each run taking 4682682 ns. The -12 suffix relates to GOMAXPROCS. The default value is the number of CPUs visible to the Go process at startup. You can change it with -cpu, and pass multiple values to run multiple benchmarks. Run with multiple CPU counts ➜ go test -bench=. -cpu=1,2,4 -benchmem -run=none goos: darwin goarch: amd64 pkg: pkg06 cpu: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz BenchmarkFib 244 4694667 ns/op 0 B/op 0 allocs/op BenchmarkFib-2 255 4721201 ns/op 0 B/op 0 allocs/op BenchmarkFib-4 256 4756392 ns/op 0 B/op 0 allocs/op PASS ok pkg06 5.826s Run benchmarks multiple times with count Due to CPU throttling, memory locality, background work, GC activity, etc., a single run may be noisy. It is common to run benchmarks multiple times.