📃Articles 的子部分
Cheat Sheet
Cheat Sheet 的子部分
Aliyun Related
Aliyun Related 的子部分
OSSutil
阿里版本的 Minio(https://min.io/)
下载 ossutil
首先，你需要下载 ossutil 二进制文件
OS:
curl https://gosspublic.alicdn.com/ossutil/install.sh | sudo bashcurl -o ossutil-v1.7.19-windows-386.zip https://gosspublic.alicdn.com/ossutil/1.7.19/ossutil-v1.7.19-windows-386.zip配置 ossutil
./ossutil config| Params | Description | Instruction |
|---|---|---|
| endpoint | the Endpoint of the region where the Bucket is located | 通过OSS页面找到endpoin 地址 |
| accessKeyID | OSS AccessKey | 通过用户中心找到accessKey |
| accessKeySecret | OSS AccessKeySecret | 通过用户中心找到accessKeySecret |
| stsToken | token for sts service | 可以为空 |
信息
您还可以直接修改 /home/<$user>/.ossutilconfig 文件来配置ossutil。
展示文件
ossutil ls oss://<$PATH>下载文件/文件夹
你能用 cp 上传或者下载文件
ossutil cp -r oss://<$PATH> <$PTHER_PATH>上传文件/文件夹
ossutil cp -r <$SOURCE_PATH> oss://<$PATH>ECS
Apsara Stack (Aliyun Directed Cloud)
Append content in /etc/resolv.conf
nameserver 172.27.205.79And then restart kube-system.coredns-xxxx
OS Mirrors
Fedora
CentOS
CentOS 7 located in
/etc/yum.repos.d/CentOS 8 stream located in
/etc/yum.repos.d/
Ubuntu
- Ubuntu 18.04 located in
/etc/apt/sources.list - Ubuntu 20.04 located in
/etc/apt/sources.list - Ubuntu 22.04 located in
/etc/apt/sources.list
Refresh DNS
OS:
dnf clean all && dnf makecacheyum clean all && yum makecacheApp Related
Git Related
Linux Related
Linux Related 的子部分
Shell脚本示例
初始化ES备份设置
在S3存储上初始化ES备份设置，并在设置完成之后，进行一个创建快照操作
#!/bin/bash
ES_HOST="http://192.168.58.2:30910"
ES_BACKUP_REPO_NAME="s3_fs_repository"
S3_CLIENT="default"
ES_BACKUP_BUCKET_IN_S3="es-snapshot"
ES_SNAPSHOT_TAG="auto"
CHECK_RESPONSE=$(curl -s -k -X POST "$ES_HOST/_snapshot/$ES_BACKUP_REPO_NAME/_verify?pretty" )
CHECKED_NODES=$(echo "$CHECK_RESPONSE" | jq -r '.nodes')
if [ "$CHECKED_NODES" == null ]; then
echo "Doesn't exist an ES backup setting..."
echo "A default backup setting will be generated. (using '$S3_CLIENT' s3 client and all backup files will be saved in a bucket : '$ES_BACKUP_BUCKET_IN_S3'"
CREATE_RESPONSE=$(curl -s -k -X PUT "$ES_HOST/_snapshot/$ES_BACKUP_REPO_NAME?pretty" -H 'Content-Type: application/json' -d "{\"type\":\"s3\",\"settings\":{\"bucket\":\"$ES_BACKUP_BUCKET_IN_S3\",\"client\":\"$S3_CLIENT\"}}")
CREATE_ACKNOWLEDGED_FLAG=$(echo "$CREATE_RESPONSE" | jq -r '.acknowledged')
if [ "$CREATE_ACKNOWLEDGED_FLAG" == true ]; then
echo "Buckup setting '$ES_BACKUP_REPO_NAME' has been created successfully!"
else
echo "Failed to create backup setting '$ES_BACKUP_REPO_NAME', since $$CREATE_RESPONSE"
fi
else
echo "Already exist an ES backup setting '$ES_BACKUP_REPO_NAME'"
fi
CHECK_RESPONSE=$(curl -s -k -X POST "$ES_HOST/_snapshot/$ES_BACKUP_REPO_NAME/_verify?pretty" )
CHECKED_NODES=$(echo "$CHECK_RESPONSE" | jq -r '.nodes')
if [ "$CHECKED_NODES" != null ]; then
SNAPSHOT_NAME="meta-data-$ES_SNAPSHOT_TAG-snapshot-$(date +%s)"
SNAPSHOT_CREATION=$(curl -s -k -X PUT "$ES_HOST/_snapshot/$ES_BACKUP_REPO_NAME/$SNAPSHOT_NAME")
echo "Snapshot $SNAPSHOT_NAME has been created."
else
echo "Failed to create snapshot $SNAPSHOT_NAME ."
fiStorage Related
Command
Command 的子部分
Git
Init global config
git config --global user.name "AaronYang"
git config --global user.email aaron19940628@gmail.com
git config --global pager.branch false
git config --global pull.ff only
git --no-pager diffGet specific file from remote
git archive --remote=git@github.com:<$user>/<$repo>.git <$branch>:<$source_file_path> -o <$target_source_path>Clone specific branch
git clone --single-branch --branch v2.4.0 https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner.gitLinux
telnet
a command line interface for communication with a remote device or serve
telnet <$ip> <$port>lsof (list as open files)
everything is a file
lsof <$option:value>awk (Aho, Weinberger, and Kernighan [Names])
awk is a scripting language used for manipulating data and generating reports.
# awk [params] 'script'
awk <$params> <$string_content>filter bigger than 3
echo -e "1\n2\n3\n4\n5\n" | awk '$1>3'
ss (socket statistics)
view detailed information about your system’s network connections, including TCP/IP, UDP, and Unix domain sockets
# awk [params] 'script' var=value file(s)
awk <$params> -f scriptfile var=value file(s)clean files 3 days ago
find /aaa/bbb/ccc/*.gz -mtime +3 -exec rm {} \;ssh without affect $HOME/.ssh/known_hosts
ssh -o "UserKnownHostsFile /dev/null" root@aaa.domain.com
ssh -o "UserKnownHostsFile /dev/null" -o "StrictHostKeyChecking=no" root@aaa.domain.comsync clock
[yum|dnf] install -y chrony \
&& systemctl enable chronyd \
&& (systemctl is-active chronyd || systemctl start chronyd) \
&& chronyc sources \
&& chronyc tracking \
&& timedatectl set-timezone 'Asia/Shanghai'set hostname
hostnamectl set-hostname developadd remote key
ssh -o "UserKnownHostsFile /dev/null" \
root@aaa.bbb.ccc \
"mkdir -p /root/.ssh && chmod 700 /root/.ssh && echo '$SOME_PUBLIC_KEY' \
>> /root/.ssh/authorized_keys && chmod 600 /root/.ssh/authorized_keys"set -x
This will print each command to the standard error before executing it, which is useful for debugging scripts.
set -xsed (Stream Editor)
sed <$option> <$file_path>fdisk
list all disk
fdisk -lcreate CFS file system
Use mkfs.xfs command to create xfs file system and internal log on the same disk, example is shown below:
mkfs.xfs <$path>modprobe
program to add and remove modules from the Linux Kernel
modprobe nfs && modprobe nfsdMaven
1. build from submodule
You dont need to build from the head of project.
./mvnw clean package -DskipTests -rf :<$submodule-name>you can find the <$submodule-name> from submodule ’s pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.flink</groupId>
<artifactId>flink-formats</artifactId>
<version>1.20-SNAPSHOT</version>
</parent>
<artifactId>flink-avro</artifactId>
<name>Flink : Formats : Avro</name>Then you can modify the command as
./mvnw clean package -DskipTests -rf :flink-avro2. skip some other test
For example, you can skip RAT test by doing this:
./mvnw clean package -DskipTests '-Drat.skip=true'Gradle
1. spotless
keep your code spotless, check more detail in https://github.com/diffplug/spotless
And the, you can execute follwoing command to format your code.
./gradlew spotlessApply./mvnw spotless:apply2. shadowJar
shadowjar could combine a project’s dependency classes and resources into a single jar. check https://imperceptiblethoughts.com/shadow/
./gradlew shadowJar3. check dependency
list your project’s dependencies in tree view
./gradlew dependencies --configuration compileClasspath./gradlew :<$module_name>:dependencies --configuration compileClasspath