Difference between revisions of "Setup - Kali Linux"
(→ZFS (server)) |
(→pam) |
||
(19 intermediate revisions by 2 users not shown) | |||
Line 25: | Line 25: | ||
systemctl enable ntp.service | systemctl enable ntp.service | ||
systemctl start ntp.service | systemctl start ntp.service | ||
+ | |||
+ | |||
+ | If you have several ethernets (private network for server) | ||
+ | # /etc/network/interfaces | ||
+ | |||
+ | # eth0 | ||
+ | auto eth0 | ||
+ | iface eth0 inet dhcp | ||
+ | |||
+ | # eht1 | ||
+ | auto eth1 | ||
+ | iface eth1 inet static | ||
+ | address 192.168.3.105 | ||
+ | netmask 255.255.255.0 | ||
+ | |||
+ | Restart network if needed | ||
+ | systemctl restart networking | ||
== Firewall Setup== | == Firewall Setup== | ||
Line 33: | Line 50: | ||
ufw reject auth | ufw reject auth | ||
ufw limit ssh/tcp | ufw limit ssh/tcp | ||
+ | ufw allow from 192.168.3.0/24 | ||
ufw logging off | ufw logging off | ||
Line 43: | Line 61: | ||
Finally, query the rules being applied via the status command. | Finally, query the rules being applied via the status command. | ||
ufw status | ufw status | ||
+ | |||
+ | === autofs === | ||
+ | |||
+ | To mount the mada shared disk. Use 192.168.3.100 if server, mada0.cse.ucsc.edu otherwise | ||
+ | |||
+ | Install autofs | ||
+ | apt install autofs | ||
+ | systemctl enable autofs.service | ||
+ | |||
+ | |||
+ | Add to /etc/auto.master | ||
+ | echo "/mada /etc/auto.mada" >>/etc/auto.master | ||
+ | |||
+ | Create the /etc/auto.mada | ||
+ | software -ro,intr,soft,noatime,fstype=nfs4 192.168.3.100:/srv/nfs/software | ||
+ | users -rw,intr,soft,noatime,fstype=nfs4 192.168.3.100:/srv/nfs/users | ||
+ | |||
+ | If on Ubuntu, append the following to /etc/default/nfs-common | ||
+ | echo "NEED_IDMAPD=yes" | sudo tee -a /etc/default/nfs-common | ||
== Determinism for QEMU/ESESC/LiveHD/.... == | == Determinism for QEMU/ESESC/LiveHD/.... == | ||
Line 115: | Line 152: | ||
Use default options of this two packages | Use default options of this two packages | ||
− | apt install libnss- | + | apt install libnss-ldapd |
− | + | ||
− | + | Check the /etc/nsswitch.conf to have ldap | |
passwd: files systemd ldap | passwd: files systemd ldap | ||
group: files systemd ldap | group: files systemd ldap | ||
shadow: files ldap | shadow: files ldap | ||
− | Update the | + | Update the /etc/nslcd.conf with: |
− | + | ||
− | + | uid nslcd | |
− | + | gid nslcd | |
+ | uri ldap://ldap-99.soe.ucsc.edu/ | ||
+ | base dc=soe,dc=ucsc,dc=edu | ||
+ | ssl start_tls | ||
+ | tls_reqcert never | ||
+ | tls_cacertfile /etc/ssl/certs/ca-certificates.crt | ||
+ | |||
+ | |||
+ | nslcd is the daemon. To test in debug/interactive | ||
+ | nslcd -n -d | ||
After this, you should have a getent | After this, you should have a getent | ||
Line 142: | Line 186: | ||
+ : (vama) : ALL | + : (vama) : ALL | ||
-:ALL :ALL | -:ALL :ALL | ||
+ | |||
+ | == Server Disk == | ||
+ | |||
+ | |||
+ | === MDADM RAID === | ||
+ | |||
+ | Get mdad | ||
+ | apt install mdadm | ||
+ | |||
+ | Make sure that the disks have GPT partition | ||
+ | |||
+ | parted /dev/XXX | ||
+ | (parted) mklabel gpt | ||
+ | (parted) quit | ||
+ | |||
+ | Create a single partition for full disk type RAID (29) | ||
+ | |||
+ | fdisk /dev/XXX | ||
+ | |||
+ | Sample sequence | ||
+ | fdisk /dev/disk/by-id/nvme-Micron_9300_MTFDHAL6T4TDR_1943249XXXX | ||
+ | Command (m for help): p | ||
+ | Disk /dev/disk/by-id/nvme-Micron_9300_MTFDHAL6T4TDR_194324955FD6: 5.84 TiB, 6401252745216 bytes, 12502446768 sectors | ||
+ | Disk model: Micron_9300_MTFDHAL6T4TDR | ||
+ | Units: sectors of 1 * 512 = 512 bytes | ||
+ | Sector size (logical/physical): 512 bytes / 512 bytes | ||
+ | I/O size (minimum/optimal): 512 bytes / 512 bytes | ||
+ | Disklabel type: gpt | ||
+ | Disk identifier: A80D3A07-8B7A-47F6-96AC-D470B50E5BDE | ||
+ | |||
+ | Command (m for help): n | ||
+ | Partition number (1-128, default 1): | ||
+ | First sector (34-12502446734, default 2048): | ||
+ | Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-12502446734, default 12502446734): | ||
+ | |||
+ | Created a new partition 1 of type 'Linux filesystem' and of size 5.8 TiB. | ||
+ | |||
+ | Command (m for help): t | ||
+ | Selected partition 1 | ||
+ | Partition type (type L to list all types): 29 | ||
+ | Changed type of partition 'Linux filesystem' to 'Linux RAID'. | ||
+ | |||
+ | Command (m for help): w | ||
+ | The partition table has been altered. | ||
+ | Calling ioctl() to re-read partition table. | ||
+ | Syncing disks. | ||
+ | |||
+ | |||
+ | Create the /dev/md0 | ||
+ | mdadm --create /dev/md0 --level=5 --raid-devices=7 /dev/disk/by-id/nvme-Micron_9300_MTFDHAL6T4TDR_1943249?????-part1 | ||
+ | |||
+ | Wait (or not) until the RAID is created | ||
+ | watch -n1 cat /proc/mdstat | ||
+ | |||
+ | Check the RAID status | ||
+ | mdadm --detail /dev/md0 | ||
+ | |||
+ | Add mdadm conf | ||
+ | mdadm --detail --scan >>/etc/mdadm/mdadm.conf | ||
+ | |||
+ | Update initrd to include mdadm | ||
+ | update-initramfs -u | ||
+ | |||
+ | === BTRFS === | ||
+ | |||
+ | Install BTRFS package | ||
+ | apt install btrfs-progs | ||
+ | |||
+ | Format disk | ||
+ | mkfs.btrfs /dev/md0 | ||
+ | |||
+ | Create a module (full disk) | ||
+ | btrfs subvolume create /local/users | ||
+ | |||
+ | Create a snapshot | ||
+ | btrfs subvolume snapshot /local/users /local/users_snap-1 | ||
+ | |||
+ | Delete a snapshot | ||
+ | btrfs subvolume delete /local/users_snap-1 | ||
+ | |||
+ | List volumes/snapshots | ||
+ | btrfs subvolume list /local/ | ||
+ | |||
+ | Add to /etc/fstab | ||
+ | /dev/md0 /local btrfs defaults,nofail,noatime,autodefrag,compress=lzo,noauto,x-systemd.automount 0 0 | ||
+ | |||
+ | Update initrd to include mdadm | ||
+ | update-initramfs -u | ||
+ | |||
+ | * NOTE; btrfs quota is per subvolume. Maybe something to enable? It will require a subvolume per user | ||
+ | |||
+ | Enable quota in /local/users | ||
+ | |||
+ | btrfs quota enable /local/users/xxxx | ||
+ | |||
+ | To see the quota usage (re-scan may be in progress if just created) | ||
+ | |||
+ | btrfs qgroup show -pcre /local/users/xxxx | ||
+ | |||
+ | === Setup local disk === | ||
+ | |||
+ | All the machines should have a /local directory. A RAID server may use a mdadm, but even local machines should have it to avoid NFS for benchmarking | ||
+ | and for trace generation. | ||
+ | |||
+ | mkdir -p /local/scrap/ | ||
+ | cd /local/scrap | ||
+ | mkdir masc vlsi vama | ||
+ | chmod 775 ???? | ||
+ | chgrp masc masc | ||
+ | chgrp vlsi vlsi | ||
+ | chgrp vama vama | ||
== ZFS (server) == | == ZFS (server) == |
Latest revision as of 01:12, 30 October 2021
Contents
Create Install Setup
Create a bootable USB from Kali linux
https://www.kali.org/docs/usb/kali-linux-live-usb-install/
Use default partition (single partition and swap around same as memory)
GUI install works fine. Default options
Account Setup
Create a local account that does not match in name the SOE LDAP. E.g: jrenau vs renau
Basic Setup
SSH Server (not for laptop, just desktop and servers)
apt-get install openssh-server systemctl enable ssh.service systemctl start ssh.service
Get the time in sync with NTP
systemctl enable ntp.service systemctl start ntp.service
If you have several ethernets (private network for server)
# /etc/network/interfaces
# eth0 auto eth0 iface eth0 inet dhcp
# eht1 auto eth1 iface eth1 inet static address 192.168.3.105 netmask 255.255.255.0
Restart network if needed
systemctl restart networking
Firewall Setup
Setup the firewall:
apt-get install ufw ufw default deny ufw reject auth ufw limit ssh/tcp ufw allow from 192.168.3.0/24 ufw logging off
The next line is only needed the first time you install the package.
ufw enable
Follow it by enabling ufw with systemctl.
systemctl enable ufw.service
Finally, query the rules being applied via the status command.
ufw status
autofs
To mount the mada shared disk. Use 192.168.3.100 if server, mada0.cse.ucsc.edu otherwise
Install autofs
apt install autofs systemctl enable autofs.service
Add to /etc/auto.master
echo "/mada /etc/auto.mada" >>/etc/auto.master
Create the /etc/auto.mada
software -ro,intr,soft,noatime,fstype=nfs4 192.168.3.100:/srv/nfs/software users -rw,intr,soft,noatime,fstype=nfs4 192.168.3.100:/srv/nfs/users
If on Ubuntu, append the following to /etc/default/nfs-common
echo "NEED_IDMAPD=yes" | sudo tee -a /etc/default/nfs-common
Determinism for QEMU/ESESC/LiveHD/....
mmap problem for qemu, enable perf monitoring for everyone, no randomization (determinism)
echo "vm.mmap_min_addr = 4096" >>/etc/sysctl.d/10-masc.conf echo "fs.inotify.max_user_watches = 65535" >>/etc/sysctl.d/10-masc.conf echo "kernel.randomize_va_space = 0" >>/etc/sysctl.d/10-masc.conf echo "kernel.perf_event_paranoid = -1" >>/etc/sysctl.d/10-masc.conf
restart sysctl or wait for reboot
systemctl restart systemd-sysctl.service
Packages
Get Atom/Bazel repos
Get Atom package
curl -L https://packagecloud.io/AtomEditor/atom/gpgkey | apt-key add - echo "deb [arch=amd64] https://packagecloud.io/AtomEditor/atom/any/ any main" > /etc/apt/sources.list.d/atom.list apt update apt install atom
Get Bazel package
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add - echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list apt update apt install bazel
Get Basic Packages
Go to a Kali Linux machine (mada4?), dump the packages installed
dpkg --get-selections | grep -v deinstall > installed_packages.txt
Go to your new machine, and install the missing packages
cut -f 1 installed_packages.txt | xargs apt-get install -y
NOTE: There may be some conflicts with the older kali machine. My suggestion is to upgrade to the latest version before. At the end, there may be also issues for packages like atom
- vi /etc/apt/sources.list.d/buster-backports.list
deb http://deb.debian.org/debian buster-backports main contrib deb-src http://deb.debian.org/debian buster-backports main contrib
LDAP
ldap directory access
Copy the cacerts from SOE (firedance in example)
scp -r renau@firedance:/etc/openldap/cacerts /etc/ldap/ scp -r renau@firedance:/etc/openldap/ldap.conf /etc/ldap/ldap.conf.soe sed -s/openldap/ldap/g /etc/ldap/ldap.conf.soe >/etc/ldap/ldap.conf
Now you should be able to execute the following command.
ldapsearch -x LLL
pam
Use default options of this two packages
apt install libnss-ldapd
Check the /etc/nsswitch.conf to have ldap
passwd: files systemd ldap group: files systemd ldap shadow: files ldap
Update the /etc/nslcd.conf with:
uid nslcd gid nslcd uri ldap://ldap-99.soe.ucsc.edu/ base dc=soe,dc=ucsc,dc=edu ssl start_tls tls_reqcert never tls_cacertfile /etc/ssl/certs/ca-certificates.crt
nslcd is the daemon. To test in debug/interactive
nslcd -n -d
After this, you should have a getent
getent passwd | grep -i renau
security
Restrict access only to masc/vlsi/vama groups Add to /etc/security/access.conf
+ : root : ALL + : (masc) : ALL + : (vlsi) : ALL + : (vama) : ALL -:ALL :ALL
Server Disk
MDADM RAID
Get mdad
apt install mdadm
Make sure that the disks have GPT partition
parted /dev/XXX (parted) mklabel gpt (parted) quit
Create a single partition for full disk type RAID (29)
fdisk /dev/XXX
Sample sequence
fdisk /dev/disk/by-id/nvme-Micron_9300_MTFDHAL6T4TDR_1943249XXXX Command (m for help): p Disk /dev/disk/by-id/nvme-Micron_9300_MTFDHAL6T4TDR_194324955FD6: 5.84 TiB, 6401252745216 bytes, 12502446768 sectors Disk model: Micron_9300_MTFDHAL6T4TDR Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: A80D3A07-8B7A-47F6-96AC-D470B50E5BDE
Command (m for help): n Partition number (1-128, default 1): First sector (34-12502446734, default 2048): Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-12502446734, default 12502446734):
Created a new partition 1 of type 'Linux filesystem' and of size 5.8 TiB.
Command (m for help): t Selected partition 1 Partition type (type L to list all types): 29 Changed type of partition 'Linux filesystem' to 'Linux RAID'.
Command (m for help): w The partition table has been altered. Calling ioctl() to re-read partition table. Syncing disks.
Create the /dev/md0
mdadm --create /dev/md0 --level=5 --raid-devices=7 /dev/disk/by-id/nvme-Micron_9300_MTFDHAL6T4TDR_1943249?????-part1
Wait (or not) until the RAID is created
watch -n1 cat /proc/mdstat
Check the RAID status
mdadm --detail /dev/md0
Add mdadm conf
mdadm --detail --scan >>/etc/mdadm/mdadm.conf
Update initrd to include mdadm
update-initramfs -u
BTRFS
Install BTRFS package
apt install btrfs-progs
Format disk
mkfs.btrfs /dev/md0
Create a module (full disk)
btrfs subvolume create /local/users
Create a snapshot
btrfs subvolume snapshot /local/users /local/users_snap-1
Delete a snapshot
btrfs subvolume delete /local/users_snap-1
List volumes/snapshots
btrfs subvolume list /local/
Add to /etc/fstab
/dev/md0 /local btrfs defaults,nofail,noatime,autodefrag,compress=lzo,noauto,x-systemd.automount 0 0
Update initrd to include mdadm
update-initramfs -u
- NOTE; btrfs quota is per subvolume. Maybe something to enable? It will require a subvolume per user
Enable quota in /local/users
btrfs quota enable /local/users/xxxx
To see the quota usage (re-scan may be in progress if just created)
btrfs qgroup show -pcre /local/users/xxxx
Setup local disk
All the machines should have a /local directory. A RAID server may use a mdadm, but even local machines should have it to avoid NFS for benchmarking and for trace generation.
mkdir -p /local/scrap/ cd /local/scrap mkdir masc vlsi vama chmod 775 ???? chgrp masc masc chgrp vlsi vlsi chgrp vama vama
ZFS (server)
WARNING DO NOT INSTALL THIS. It has issues upgrading kernel. Kept for documentation reasons
This explains the ZFS setup for large filesystem server (not for desktop)
Get the latest kernel and headers
apt update apt install linux-headers-`uname -r`
Set backports for ZFS
# vi /etc/apt/preferences.d/90_zfs Package: libnvpair1linux libuutil1linux libzfs2linux libzpool2linux spl-dkms zfs-dkms zfs-test zfsutils-linux zfsutils-linux-dev zfs-zed Pin: release n=buster-backports Pin-Priority: 990
Install zfs packages
apt install zfsutils-linux zfs-dkms spl-dkms