Subsections of Cheat Sheet
Aliyun Related
Subsections of Aliyun Related
OSSutil
Ali version of Minio
(https://min.io/)
download ossutil
first, you need to download ossutil
first
curl https://gosspublic.alicdn.com/ossutil/install.sh | sudo bash
curl -o ossutil-v1.7.19-windows-386.zip https://gosspublic.alicdn.com/ossutil/1.7.19/ossutil-v1.7.19-windows-386.zip
config ossutil
./ossutil config
Params | Description | Instruction |
---|---|---|
endpoint | the Endpoint of the region where the Bucket is located | |
accessKeyID | OSS AccessKey | get from user info panel |
accessKeySecret | OSS AccessKeySecret | get from user info panel |
stsToken | token for sts service | could be empty |
and you can also modify /home/<$user>/.ossutilconfig
file directly to change the configuration.
list files
ossutil ls oss://<$PATH>
download file/dir
you can use cp
to download or upload file
ossutil cp -r oss://<$PATH> <$PTHER_PATH>
upload file/dir
ossutil cp -r <$SOURCE_PATH> oss://<$PATH>
ECS DNS
ZJADC (Aliyun Directed Cloud)
Append content in /etc/resolv.conf
options timeout:2 attempts:3 rotate
nameserver 10.255.9.2
nameserver 10.200.12.5
And then you probably need to modify yum.repo.d
as well, check link
YQGCY (Aliyun Directed Cloud)
Append content in /etc/resolv.conf
nameserver 172.27.205.79
And then restart kube-system
.coredns-xxxx
Google DNS
nameserver 8.8.8.8
nameserver 4.4.4.4
nameserver 223.5.5.5
nameserver 223.6.6.6
Restart DNS
vim /etc/NetworkManager/NetworkManager.conf
vim /etc/NetworkManager/NetworkManager.conf
add "dns=none"
under '[main]'
part
systemctl restart NetworkManager
Modify ifcfg-ethX
[Optional]
if you cannot get ipv4 address, you can try to modify ifcfg-ethX
vim /etc/sysconfig/network-scripts/ifcfg-ens33
set ONBOOT=yes
OS Mirrors
Fedora
- Fedora 40 located in
/etc/yum.repos.d/
CentOS
CentOS 7 located in
/etc/yum.repos.d/
CentOS 8 stream located in
/etc/yum.repos.d/
Ubuntu
Ubuntu 18.04 located in
/etc/apt/sources.list
Ubuntu 20.04 located in
/etc/apt/sources.list
Ubuntu 22.04 located in
/etc/apt/sources.list
Debian
Debian Buster located in
/etc/apt/sources.list
Debian Bullseye located in
/etc/apt/sources.list
Anolis
Anolis 3 located in
/etc/yum.repos.d/
Anolis 2 located in
/etc/yum.repos.d/
Refresh Repo
dnf clean all && dnf makecache
yum clean all && yum makecache
apt-get clean all
App Related
Subsections of App Related
Mirrors [Aliyun, Tsinghua]
Gradle Tencent Mirror
https://mirrors.cloud.tencent.com/gradle/gradle-8.0-bin.zip
PIP Tuna Mirror -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple some-package
Maven Mirror
<mirror>
<id>aliyunmaven</id>
<mirrorOf>*</mirrorOf>
<name>阿里云公共仓库</name>
<url>https://maven.aliyun.com/repository/public</url>
</mirror>
Git Related
Subsections of Git Related
Not Allow Push
Cannot push to your own branch
Edit
.git/config
file under your repo directory.Find
url
=entry under section[remote "origin"]
.Change it from:
url=https://gitlab.com/AaronYang2333/ska-src-dm-local-data-preparer.git/
url=ssh://git@gitlab.com/AaronYang2333/ska-src-dm-local-data-preparer.git
try push again
Linux Related
Subsections of Linux Related
Disable Service
Disable firewall、selinux、dnsmasq、swap service
systemctl disable --now firewalld
systemctl disable --now dnsmasq
systemctl disable --now NetworkManager
setenforce 0
sed -i 's#SELINUX=permissive#SELINUX=disabled#g' /etc/sysconfig/selinux
sed -i 's#SELINUX=permissive#SELINUX=disabled#g' /etc/selinux/config
reboot
getenforce
swapoff -a && sysctl -w vm.swappiness=0
sed -ri '/^[^#]*swap/s@^@#@' /etc/fstab
Example Shell Script
Init ES Backup Setting
create an ES backup setting in s3, and make an snapshot after creation
#!/bin/bash
ES_HOST="http://192.168.58.2:30910"
ES_BACKUP_REPO_NAME="s3_fs_repository"
S3_CLIENT="default"
ES_BACKUP_BUCKET_IN_S3="es-snapshot"
ES_SNAPSHOT_TAG="auto"
CHECK_RESPONSE=$(curl -s -k -X POST "$ES_HOST/_snapshot/$ES_BACKUP_REPO_NAME/_verify?pretty" )
CHECKED_NODES=$(echo "$CHECK_RESPONSE" | jq -r '.nodes')
if [ "$CHECKED_NODES" == null ]; then
echo "Doesn't exist an ES backup setting..."
echo "A default backup setting will be generated. (using '$S3_CLIENT' s3 client and all backup files will be saved in a bucket : '$ES_BACKUP_BUCKET_IN_S3'"
CREATE_RESPONSE=$(curl -s -k -X PUT "$ES_HOST/_snapshot/$ES_BACKUP_REPO_NAME?pretty" -H 'Content-Type: application/json' -d "{\"type\":\"s3\",\"settings\":{\"bucket\":\"$ES_BACKUP_BUCKET_IN_S3\",\"client\":\"$S3_CLIENT\"}}")
CREATE_ACKNOWLEDGED_FLAG=$(echo "$CREATE_RESPONSE" | jq -r '.acknowledged')
if [ "$CREATE_ACKNOWLEDGED_FLAG" == true ]; then
echo "Buckup setting '$ES_BACKUP_REPO_NAME' has been created successfully!"
else
echo "Failed to create backup setting '$ES_BACKUP_REPO_NAME', since $$CREATE_RESPONSE"
fi
else
echo "Already exist an ES backup setting '$ES_BACKUP_REPO_NAME'"
fi
CHECK_RESPONSE=$(curl -s -k -X POST "$ES_HOST/_snapshot/$ES_BACKUP_REPO_NAME/_verify?pretty" )
CHECKED_NODES=$(echo "$CHECK_RESPONSE" | jq -r '.nodes')
if [ "$CHECKED_NODES" != null ]; then
SNAPSHOT_NAME="meta-data-$ES_SNAPSHOT_TAG-snapshot-$(date +%s)"
SNAPSHOT_CREATION=$(curl -s -k -X PUT "$ES_HOST/_snapshot/$ES_BACKUP_REPO_NAME/$SNAPSHOT_NAME")
echo "Snapshot $SNAPSHOT_NAME has been created."
else
echo "Failed to create snapshot $SNAPSHOT_NAME ."
fi
Login Without Pwd
copy id_rsa
to other nodes
yum install sshpass -y
mkdir -p /extend/shell
cat >>/extend/shell/fenfa_pub.sh<< EOF
#!/bin/bash
ROOT_PASS=root123
ssh-keygen -t rsa -f ~/.ssh/id_rsa -P ''
for ip in 101 102 103
do
sshpass -p\$ROOT_PASS ssh-copy-id -o StrictHostKeyChecking=no 192.168.29.\$ip
done
EOF
cd /extend/shell
chmod +x fenfa_pub.sh
./fenfa_pub.sh
Set Http Proxy
set http proxy
export https_proxy=http://localhost:20171
Storage Related
Subsections of Storage Related
User Based Policy
User Based Policy
you can change <$bucket>
to control the permission
${aws:username}
is a build-in variable, indicating the logined user name.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowUserToSeeBucketListInTheConsole",
"Action": [
"s3:ListAllMyBuckets",
"s3:GetBucketLocation"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::*"
]
},
{
"Sid": "AllowRootAndHomeListingOfCompanyBucket",
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::<$bucket>"
],
"Condition": {
"StringEquals": {
"s3:prefix": [
"",
"<$path>/",
"<$path>/${aws:username}"
],
"s3:delimiter": [
"/"
]
}
}
},
{
"Sid": "AllowListingOfUserFolder",
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::<$bucket>"
],
"Condition": {
"StringLike": {
"s3:prefix": [
"<$path>/${aws:username}/*"
]
}
}
},
{
"Sid": "AllowAllS3ActionsInUserFolder",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::<$bucket>/<$path>/${aws:username}/*"
]
}
]
}
<$uid>
is Aliyun UID
{
"Version": "1",
"Statement": [{
"Effect": "Allow",
"Action": [
"oss:*"
],
"Principal": [
"<$uid>"
],
"Resource": [
"acs:oss:*:<$oss_id>:<$bucket>/<$path>/*"
]
}, {
"Effect": "Allow",
"Action": [
"oss:ListObjects",
"oss:GetObject"
],
"Principal": [
"<$uid>"
],
"Resource": [
"acs:oss:*:<$oss_id>:<$bucket>"
],
"Condition": {
"StringLike": {
"oss:Prefix": [
"<$path>/*"
]
}
}
}]
}
Example:
{
"Version": "1",
"Statement": [{
"Effect": "Allow",
"Action": [
"oss:*"
],
"Principal": [
"203415213249511533"
],
"Resource": [
"acs:oss:*:1007296819402486:conti-csst/test/*"
]
}, {
"Effect": "Allow",
"Action": [
"oss:ListObjects",
"oss:GetObject"
],
"Principal": [
"203415213249511533"
],
"Resource": [
"acs:oss:*:1007296819402486:conti-csst"
],
"Condition": {
"StringLike": {
"oss:Prefix": [
"test/*"
]
}
}
}]
}