reference:
- SSH port forwarding - Example, command, server config
- 透过代理连接SSH
- SSH ProxyCommand example: Going through one host to reach another server
- OpenSSH/Cookbook/Proxies and Jump Hosts
- Tunneling SSH over PageKite
- Transparent Multi-hop SSH
- SSH via HTTP proxy in OSX
- ssh key
- 6 ssh authentication methods to secure connection (sshd_config)
- 8 ways to prevent brute force SSH attacks in Linux (CentOS/RHEL 7)
ssh key
generate ssh key
$ keyname='marslo@china'
# rsa4096
$ ssh-keygen -t rsa -b 4096 -f ~/.ssh/${keyname} -C "${keyname}" -P "" -q
# ed25519
$ ssh-keygen -t ed25519 -C "${keyname}" -f ~/.ssh/${keyname} -P '' -q
$ ssh-keygen -t ed25519 -o -a 100 -C "${keyname}" -f ~/.ssh/${keyname} -P '' -q
get servers public key
$ ssh-keyscan -H www.server.com
# or
$ ssh-keyscan -p 29418 -t rsa www.server.com
$ ssh-keyscan -p 29418 -t rsa www.server.com >> ~/.ssh/known_hosts
upload the local ~/.ssh/know_hosts
[!NOTE|label:references:]
$ ssh-keygen -R github.com # Host github.com found: line 63 /home/marslo/.ssh/known_hosts updated. Original contents retained as /home/marslo/.ssh/known_hosts.old # or $ curl -sL https://api.github.com/meta | jq -r '.ssh_keys | .[]' | sed -e 's/^/github.com /' >> ~/.ssh/know_hosts # more details $ curl -sL https://api.github.com/meta | jq -r '.ssh_keys | .[]' ssh-ed25519 AAAA***9GKJl ecdsa-sha2-nistp256 AAAA***ockg= ssh-rsa AAAA***wsjk= $ curl -sL https://api.github.com/meta | jq -r '.ssh_keys | .[]' | sed -e 's/^/github.com /' github.com ssh-ed25519 AAAA***9GKJl github.com ecdsa-sha2-nistp256 AAAA***ockg= github.com ssh-rsa AAAA***wsjk=
add ssh key into agent
$ ssh-add ~/.ssh/${keyname}
# e.g.:
$ ssh-add ~/.ssh/id_ed25519
Identity added: /Users/marslo/.ssh/id_ed25519 (marslo@devops)
$ ssh-agent -s
SSH_AUTH_SOCK=/var/folders/s3/mg_f3cv54nn7y758j_t46zt40000gn/T//ssh-MgCrKA3ZS06N/agent.25376; export SSH_AUTH_SOCK;
SSH_AGENT_PID=25377; export SSH_AGENT_PID;
echo Agent pid 25377;
get public key from private key
$ ssh-keygen -y -f ~/.ssh/id_rsa > ~/.ssh/id_rsa.pub
verify public and private key pair
$ diff [-qs] <( ssh-keygen -y -e -f /path/to/private ) \
<( ssh-keygen -y -e -f /path/to/public.pub )
generate new passphrase
$ ssh-keygen -p -f /path/to/private
get fingerprinter
sha256:
$ ssh-keygen -l -f ~/.ssh/id_rsa # or $ ssh-keygen -l -f ~/.ssh/id_rsa.pubmd5:
$ ssh-keygen -E md5 -l -f ~/.ssh/id_rsa # or $ ssh-keygen -E md5 -l -f ~/.ssh/id_rsa.pub
with openssl
$ openssl pkey -in ~/.ssh/ec2/primary.pem -pubout -outform DER | openssl md5 -c
keys performance
$ openssl speed rsa1024 rsa2048 dsa1024 dsa2048
Doing 1024 bit private rsa's for 10s: 91211 1024 bit private RSA's in 9.97s
Doing 1024 bit public rsa's for 10s: 1161461 1024 bit public RSA's in 9.93s
Doing 2048 bit private rsa's for 10s: 12305 2048 bit private RSA's in 9.94s
Doing 2048 bit public rsa's for 10s: 403455 2048 bit public RSA's in 9.98s
Doing 1024 bit sign dsa's for 10s: 84873 1024 bit DSA signs in 10.00s
Doing 1024 bit verify dsa's for 10s: 109544 1024 bit DSA verify in 9.99s
Doing 2048 bit sign dsa's for 10s: 30010 2048 bit DSA signs in 9.99s
Doing 2048 bit verify dsa's for 10s: 33202 2048 bit DSA verify in 9.98s
OpenSSL 1.0.2t 10 Sep 2019
built on: reproducible build, date unspecified
options:bn(64,64) rc4(ptr,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang -I. -I.. -I../include -fPIC -fno-common -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -arch x86_64 -O3 -DL_ENDIAN -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
sign verify sign/s verify/s
rsa 1024 bits 0.000109s 0.000009s 9148.5 116964.9
rsa 2048 bits 0.000808s 0.000025s 1237.9 40426.4
sign verify sign/s verify/s
dsa 1024 bits 0.000118s 0.000091s 8487.3 10965.4
dsa 2048 bits 0.000333s 0.000301s 3004.0 3326.9
ssh
force use password
$ ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no user@target.server
ssh and tar
copy multiple files to remote server
$ tar cvzf - -T list_of_filenames | ssh hostname tar xzf -
find && tar
backup all
config.xmlin JENKINS_HOME$ find ${JENKINS_HOME}/jobs \ -maxdepth 2 \ -name config\.xml \ -type f -print | tar czf ~/config.xml.tar.gz --files-from -back build history
$ find ${JENKINS_HOME}/jobs \ -name builds \ -prune -o \ -type f \ -print | tar czf ~/m.tar.gz --files-from -
tar all and extra in remote
# ssh -C
# no `-z` `-C`
# | |
# v v
$ tar cf - . | ssh -C hostname "cd ~/.marslo/test/; tar xvf -"
Warning: Permanently added '10.69.78.40' (ECDSA) to the list of known hosts.
./
./temp/
./a/
./a/a.txt
./c/
./b/
# tar z-flag
# `-z` no `-C`
# | |
# v v
$ tar cfz - . | ssh hostname "cd ~/.marslo/test/; tar xvzf -"
with proxy
using command directly
Linux:
$ ssh -o 'ProxyCommand nc -x proxy.url.com proxy-port %h %p' -vT user@target.server # or $ ssh -o 'ProxyCommand corkscrew proxy.url.com proxy-port %h %p' -vT user@target.serverwindows:
git for windows
$ ssh -o 'ProxyCommand connect -H http://proxy.url.com:proxy-port %h %p' user@target.servercygwin
$ apt-cyg install corkscrew $ ssh -o 'ProxyCommand corkscrew proxy.url.com proxy-port %h %p' user@target.server # or $ apt-cyg install nc $ ssh -o 'ProxyCommand nc -X connect -x proxy.url.com:proxy-port %h %p' -vT git@github.com
ssh certificate
[!NOTE|label:references:]
- * OpenSSH/Cookbook/Certificate-based Authentication
- * Configure ssh certificate based authentication
- 14.3.3. Creating SSH CA Certificate Signing Keys
- Files Associated with SSH Host Certificates
- Certificate Authority (ca) - a private key generated for signing other keys
- Certificate public key - the public component of the certificate authority
- Host Public Key - the actual key that the SSH daemon uses to identify itself to the clients
- Host Certificate - the signature made for the Host Public Key using the Certificate Authority
- How to Generate and Configure SSH Certificate-Based Authentication
- If you’re not using SSH certificates you’re doing SSH wrong
- How to configure and setup SSH certificates for SSH authentication
- How To Configure SSH Key-Based Authentication on a Linux Server
- Tightening SSH access using short-lived SSH certificates
- How to setup SSH certificates for SSH authentication
SSH User Certificates
ca
remote $ ssh-keygen -t ed25519 -b 4096 -f devops@ca -C 'devops@ca' -P '' -q
remote $ sudo cp devops@ca* /etc/ssh/
remote $ echo 'TrustedUserCAKeys /etc/ssh/devops@ca.pub' | sudo tee -a /etc/ssh/sshd_config
remote $ sudo grep TrustedUserCAKeys /etc/ssh/sshd_config
TrustedUserCAKeys /etc/ssh/devops@ca.pub
sign and generate cert key
# download devops@ca
# key id principals
# v v
local $ ssh-keygen -s devops@ca -I marslo -n marslo -V +52w ~/.ssh/marslo.pub
$ ls ~/.ssh/marslo*
/Users/marslo/.ssh/marslo /Users/marslo/.ssh/marslo-cert.pub /Users/marslo/.ssh/marslo.pub
verify
$ ssh-keygen -Lf ~/.ssh/marslo-cert.pub /Users/marslo/.ssh/marslo-cert.pub: Type: ssh-ed25519-cert-v01@openssh.com user certificate Public key: ED25519-CERT SHA256:JfJnCDxjWhLwW3BcBX3XycBr3dT3JtHTwD1M4H3828E Signing CA: ED25519 SHA256:5dNlpIIjX/pdoNSpmtfcGQijSrl3W87TByA9KeCe2M0 (using ssh-ed25519) Key ID: "marslo" Serial: 0 Valid: from 2023-08-17T17:41:00 to 2024-08-15T17:42:52 Principals: marslo Critical Options: (none) Extensions: permit-X11-forwarding permit-agent-forwarding permit-port-forwarding permit-pty permit-user-rc $ ssh -i ~/.ssh/marslo marslo@remote "cat /home/marslo/.ssh/authorized_keys; du -hs /home/marslo/.ssh/authorized_keys; hostname" 0 /home/marslo/.ssh/authorized_keys remote
update for existing key
# serial
# v
$ ssh-keygen -s devops@ca -I 'edcba' -z '0002' -n marslo marslo.pub
Signed user key marslo-cert.pub: id "edcba" serial 2 for marslo valid forever
# verify
$ ssh-keygen -f marslo-cert.pub -L
marslo-cert.pub:
Type: ssh-ed25519-cert-v01@openssh.com user certificate
Public key: ED25519-CERT SHA256:JfJnCDxjWhLwW3BcBX3XycBr3dT3JtHTwD1M4H3828E
Signing CA: ED25519 SHA256:5dNlpIIjX/pdoNSpmtfcGQijSrl3W87TByA9KeCe2M0 (using ssh-ed25519)
Key ID: "edcba"
Serial: 2
Valid: forever
Principals:
marslo
Critical Options: (none)
Extensions:
permit-X11-forwarding
permit-agent-forwarding
permit-port-forwarding
permit-pty
permit-user-rc
login via specific cert
[!NOTE|label:references:]
- OpenSSH/Cookbook/Certificate-based Authentication
$ ssh -o CertificateFile=server01.ed25519-cert.pub -i server01.ed25519 \ fred@server01.example.org
$ ssh marslo@sample.server.com
marslo@sample.server.com's password:
$ ssh -o CertificateFile=marslo-cert.pub marslo@sample.server.com "du -hs ~/.ssh/authorized_keys"
0 /home/marslo/.ssh/authorized_keys
# via config
$ cat ~/.ssh/confg
Host example sample.server.com
Hostname sample.server.com
User marslo
IdentitiesOnly yes
IdentityFile /home/marslo/.ssh/marslo
CertificateFile /home/marslo/.ssh/marslo-cert.pub
SSH Host Certificates
ssh tunnel
references:
[!TIP]
key point:
-L:<---R:-->basic command line
$ ssh -Nf -L <localhost/localip>:<local_port>:<target>:<target_port> <to-be-mapping/localhost>- usage:
1 -> [2 ->] 3:ssh host2:port2:host3:port3 host1:port1- if ignore
host2. default using local.host
two servers
-L
[!TIP]
- purpose:
local:22 --> jumper:6666
# -L : <--
$ ssh user@jumper.server
$ ssh -Nf -L [jumper.server:]6666:my.server:22 root@my.server
# verify
$ sudo netstat -an | grep 6666
tcp 0 0 jumper.ip:6666 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:6666 0.0.0.0:* LISTEN
tcp 0 0 ::1:6666 :::* LISTEN
-R
[!TIP]
status:
current status | v mymac:22 <--------- jumper:22 purpose: | ^ + ---------------> +
# login to jumper first via GUI
destination
|
v
<jumper> $ ssh -Nf -R 6666:<mymac>:22 marslo@<mymac>
<jumper> $ ps auxfww | grep ssh | grep -v grep
marslo 2281 0.0 0.0 11716 4064 ? Ss 19:55 0:00 \_ ssh -Nf -R 6666:<my.pc.ip>:22 marslo@<my.pc.ip>
<jumper> $ sudo netstat -anp | grep ssh
tcp 0 0 <jumper>:46176 <mymac>:22 ESTABLISHED 2281/ssh
# verify in <mymac>
<mymac> $ netstat -an | grep 6666
tcp4 0 0 127.0.0.1.6666 *.* LISTEN
tcp6 0 0 ::1.6666 *.* LISTEN
<mymac> $ ssh -p 7777 localhost
## or
<mymac> $ cat ~/.ssh/config
Host jumper
Hostname localhost
Port 7777
<mymac> $ ssh jumper
three serves
scenario 1
- purpose:
local:6666 <--- jumper:6666 <--- remote:6666
# -R : -->
$ ssh user@jumper
$ ssh -Nf -R [jumper:]6666:local:6666 root@remote
# verify
## in remote
$ ssh root@remote
$ scp -P 6666 root@localhost:/remote/path/file /local/path
## in local
$ scp -P 6666 /local/path/file root@localhost:/remote/path/file
$ ssh user@jumper
<jumper> $ ps awwx | grep ssh | grep 6666
17549 ? Ss 0:00 ssh -Nf -L 6666:remote:22 root@remote
18740 ? Ss 3:50 ssh -Nf -R 6666:local:6666 root@remote
scenario 2
[!TIP]
details :
current status -------------- | | mymac <---- jumper ----> destination | ^ wants: v | -------------->-------------- visit directly
# login to jumper
<jumper> $ ssh -Nf -R7777:<destination>:22 marslo@<mymac>
<jumper> $ ps auxfww | grep ssh | grep -v grep
marslo 41 0.0 0.0 11716 5832 ? Ss 21:51 0:00 \_ ssh -Nf -R7777:<destination>:22 marslo@<my.mac.ip>
# verify in mymac
<mymac> $ ssh -p 7777 localhost # localhost in <mymac> == <destination>
<mymac> $ netstat -an | grep 7777
tcp6 0 0 ::1.7777 ::1.65371 ESTABLISHED
tcp6 0 0 ::1.65371 ::1.7777 ESTABLISHED
tcp4 0 0 127.0.0.1.7777 *.* LISTEN
tcp6 0 0 ::1.7777 *.* LISTEN
config
ssh config
$ cat ~/.ssh/config
HOST *
LogLevel ERROR
HostkeyAlgorithms +ssh-rsa
GSSAPIAuthentication no
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
IdentityFile ~/.ssh/id_ed25519
IdentityFile ~/.ssh/id_rsa # keep the older key if necessary
# PubkeyAcceptedAlgorithms +ssh-rsa
Include config.d/*
Host github.com
User marslo.jiao@gmail.com
Hostname ssh.github.com
IdentityFile /C/Marslo/my@key
IdentitiesOnly yes
Port 443
# ProxyCommand nc -X connect -x proxy.com:8080 %h %p
# ProxyCommand /usr/bin/nc -X 5 -x 127.0.0.1:1087 %h %p
# ProxyCommand /usr/local/bin/corkscrew 127.0.0.1 1087 %h %p
sshd_config
[!TIP]
- disable Root Login :
PermitRootLogin- allow only specific users or groups :
AllowUsers,AllowGroups- deny specific users or groups :
DenyUsers,DenyGroups- change sshd port number :
Port- change login grace time :
LoginGraceTime- Restrict the Interface (IP Address) to Login :
ListenAddress- disconnect ssh when no activity :
ClientAliveInterval
banner and motd
[!TIP] files:
/etc/pam.d/sshd:session optional pam_motd.so:/usr/lib64/security/pam_motd.so/etc/motd/etc/ssh/sshd_config:Banner /path/to/banner
- force disable all banner or motd
$ ssh user@host $ touch ~/.hushlogin
disable login password
$ cat /etc/ssh/sshd_config
ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no
scripts:
TIMESTAMPE=$(date +"%Y%m%d%H%M%S") SSHDFILE="/etc/ssh/sshd_config" sudo cp "${SSHDFILE}{,.org.${TIMESTAMPE}}" sudo bash -c '/bin/sed -i -e "s:^\(UsePAM.*$\):# \1:" ${SSHDFILE}' sudo bash -c '/bin/sed -i -e "s:^\(PermitRootLogin.*$\):# \1:" ${SSHDFILE}' sudo bash -c '/bin/sed -i -e "s:^\(ChallengeResponseAuthentication.*$\):# \1:" ${SSHDFILE}' sudo bash -c '/bin/sed -i -e "s:^\(PasswordAuthentication.*$\):# \1:" ${SSHDFILE}' if ! grep 'Add my marslo' ${SSHDFILE} > /dev/null 2>&1; then sudo bash -c "cat >> ${SSHDFILE}" << EOF # Add my marslo PermitRootLogin no UsePAM no ChallengeResponseAuthentication no PasswordAuthentication no PrintMotd yes Banner /etc/ssh/server.banner EOF fi
disallow group to use password
[!TIP|label:references:]
- Directive 'UsePAM' is not allowed within a Match block
- Directive 'ChallengeResponseAuthentication' is not allowed within a Match block
- Directive 'PrintMotd' is not allowed within a Match block
- Directive 'LoginGraceTime' is not allowed within a Match block
if ! grep 'Add my marslo' ${SSHDFILE} > /dev/null 2>&1; then
sudo bash -c "cat >> ${SSHDFILE}" << EOF
# Add my marslo
Match Group devops
PasswordAuthentication no
Banner /etc/ssh/server.banner
MaxAuthTries 3
ClientAliveInterval 600 # 60*10 secs
EOF
fi
disallowing user to use tcp forwarding
Match User testuser
AllowTcpForwarding no
displaying a special banner for users not in the staff group
Match Group *,!staff
Banner /etc/banner.text
allowing root login from host rootallowed.example.com
Match Host rootallowed.example.com
PermitRootLogin yes
allowing anyone to use gatewayports from the local net
Match Address 192.168.0.0/24
GatewayPorts yes
duebug
debug git
GIT_SSH_COMMAND
$ GIT_SSH_COMMAND="ssh -vvT" git clone git@github.com:Marslo/myblog.git Cloning into 'myblog'... OpenSSH_7.9p1, LibreSSL 2.7.3 debug1: Reading configuration data /Users/marslo/.ssh/config debug1: /Users/marslo/.ssh/config line 1: Applying options for * debug1: /Users/marslo/.ssh/config line 13: Applying options for github.com debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 48: Applying options for * debug1: Connecting to github.com port 22. ^CGIT_TRACE
$ GIT_TRACE=1 git st 00:30:44.772137 git.c:703 trace: exec: git-st 00:30:44.772540 run-command.c:663 trace: run_command: git-st 00:30:44.772894 git.c:384 trace: alias expansion: st => status 00:30:44.772903 git.c:764 trace: exec: git status 00:30:44.772907 run-command.c:663 trace: run_command: git status 00:30:44.777379 git.c:440 trace: built-in: git status On branch master Your branch is up to date with 'origin/master'. 00:30:44.782714 run-command.c:663 trace: run_command: GIT_INDEX_FILE=.git/index git submodule summary --cached --for-status --summary-limit -1 HEAD 00:30:44.787490 git.c:703 trace: exec: git-submodule summary --cached --for-status --summary-limit -1 HEAD 00:30:44.788038 run-command.c:663 trace: run_command: git-submodule summary --cached --for-status --summary-limit -1 HEAD 00:30:44.838222 git.c:440 trace: built-in: git rev-parse --git-dir 00:30:44.845054 git.c:440 trace: built-in: git rev-parse --git-path objects 00:30:44.852811 git.c:440 trace: built-in: git rev-parse -q --git-dir 00:30:44.870362 git.c:440 trace: built-in: git rev-parse --show-prefix 00:30:44.878755 git.c:440 trace: built-in: git rev-parse --show-toplevel 00:30:44.893984 git.c:440 trace: built-in: git rev-parse -q --verify --default HEAD HEAD 00:30:44.899709 git.c:440 trace: built-in: git rev-parse --show-toplevel 00:30:44.905200 git.c:440 trace: built-in: git rev-parse --sq --prefix -- 00:30:44.911762 git.c:440 trace: built-in: git diff-index --cached --ignore-submodules=dirty --raw 52c94664ffc09cde2308c6bf9824ca0355ff5ff7 -- 00:30:44.917374 run-command.c:663 trace: run_command: GIT_INDEX_FILE=.git/index git submodule summary --files --for-status --summary-limit -1 00:30:44.922165 git.c:703 trace: exec: git-submodule summary --files --for-status --summary-limit -1 00:30:44.922568 run-command.c:663 trace: run_command: git-submodule summary --files --for-status --summary-limit -1 00:30:44.965375 git.c:440 trace: built-in: git rev-parse --git-dir 00:30:44.972784 git.c:440 trace: built-in: git rev-parse --git-path objects 00:30:44.979117 git.c:440 trace: built-in: git rev-parse -q --git-dir 00:30:44.991077 git.c:440 trace: built-in: git rev-parse --show-prefix 00:30:44.997718 git.c:440 trace: built-in: git rev-parse --show-toplevel 00:30:45.012365 git.c:440 trace: built-in: git rev-parse -q --verify --default HEAD 00:30:45.018759 git.c:440 trace: built-in: git rev-parse --show-toplevel 00:30:45.024687 git.c:440 trace: built-in: git rev-parse --sq --prefix -- 00:30:45.031664 git.c:440 trace: built-in: git diff-files --ignore-submodules=dirty --raw -- nothing to commit, working tree clean
debug ssh
- debug mode
$ sudo /usr/sbin/sshd -d [-d] [-d] test mode
# -T : extended test mode $ sudo /usr/sbin/sshd -T [-f /path/to/sshd_config] # -t : test mode $ sudo /usr/sbin/sshd -t [-f /path/to/sshd_config]
tips
disconnect
[!TIP] Enter + ~ + . + Enter
references:
$ <Enter>
$ ~?
Supported escape sequences:
~. - terminate connection (and any multiplexed sessions)
~B - send a BREAK to the remote system
~C - open a command line
~R - request rekey
~V/v - decrease/increase verbosity (LogLevel)
~^Z - suspend ssh
~# - list forwarded connections
~& - background ssh (when waiting for connections to terminate)
~? - this message
~~ - send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)