Difference between revisions of "Desktop Setup - Ubuntu 12.04"
(→Installing PBS Torque (server)) |
|||
Line 65: | Line 65: | ||
The current apt-get/dpkg releases have problem. So we need to work around the problems that is going to happen in this process. | The current apt-get/dpkg releases have problem. So we need to work around the problems that is going to happen in this process. | ||
− | ssh <username>@ | + | ssh <username>@mascd9 dpkg --get-selections > dpkg.list |
dpkg --set-selections < dpkg.list | dpkg --set-selections < dpkg.list | ||
apt-get update -y | apt-get update -y |
Revision as of 22:27, 9 June 2013
Currently still testing the distribution, so please report any problems to Ehsan and Jose!!
- 0 - Install Ubuntu 12.04 x86_64
NOTE: ONLY if you have trouble with Step 4 of the installation process, the partition selection, open a terminal in the trial mode and run:
sudo dmraid -r -E /dev/sda
NOTE: When prompted for a username, choose one that is NOT your SOE login! For example "ian_local" rather than my SOE login of "ianlee1521". Otherwise you will not be able to login to your computer after you setup LDAP.
NOTE: When prompted for a machine name, enter the name that machine is supposed to be (mascd1, omelette, quiche, etc)
Installation should complete as normal.
Contents
Plain Ubuntu Installation
- 1 - Enable root account
Login to the GUI with your local login, then open a terminal (Applications -> Accessories -> Terminal):
sudo su - passwd
Enter your new root password for the machine.
Add your local and SOE logins to the sudoers list:
vi /etc/sudoers
- 2 - Install NVIDIA Drivers
Blacklist some kernel modules
vi /etc/modprobe.d/blacklist.conf
Add the following lines to the file
blacklist vga16fb blacklist nouveau blacklist rivafb blacklist nvidiafb blacklist rivatv
Get rid of any installed NVIDIA drivers (you are already root, if not use with sudo):
apt-get --purge remove nvidia-*
apt-get update apt-get install nvidia-current
Reboot the machine and X should start normally. Login at the GUI login as root and open a terminal.
- 3 - Setup the new repo
scp <username>@mada0:/etc/apt/sources.list /etc/apt/sources.list apt-get update -y apt-get dist-upgrade -y
- 4 - Adjust sysctl options
mmap problem for qemu
vi /etc/sysctl.d/10-zeropage.conf
Edit the following value:
vm.mmap_min_addr = 4096
increase (add) the inotify watches in /etc/sysctl.conf
echo -e "\nfs.inotify.max_user_watches = 65535 \nkernel.randomize_va_space = 0" | tee -a /etc/sysctl.conf
- 5 - Replicate the installation on Master machine (mascd3)
The current apt-get/dpkg releases have problem. So we need to work around the problems that is going to happen in this process.
ssh <username>@mascd9 dpkg --get-selections > dpkg.list dpkg --set-selections < dpkg.list apt-get update -y
echo "libdb5.1-java:i386 deinstall" | dpkg --set-selections echo "libsensors4:i386 deinstall" | dpkg --set-selections echo "libapt-pkg4.12:i386 deinstall" | dpkg --set-selections
"apt-get dselect-upgrade -y" Will give many options and take a long time for the upgrade (about an hour for me). Select all default options.
apt-get dselect-upgrade -y
It most likely will fail, complaining unable to install libapt-pkg4.12:i386, etc. You might need to repeat this until the apt-get dselect-upgrade finished gracefully. Pay attention to the error message. If it asks you to do 'apt-get install -f', then do so!
finalized the installed packages so far:
dpkg --configure -a
then remove the problematic packages from the package selecting list, and run the apt-get dselect-upgrade again :
echo "libdb5.1-java:i386 deinstall" | dpkg --set-selections echo "libsensors4:i386 deinstall" | dpkg --set-selections echo "libapt-pkg4.12:i386 deinstall" | dpkg --set-selection apt-get dselect-upgrade -y
Ruby Related Stuff
We use RVM to support multiple ruby versions
- Install the base versions shared by all the users
sudo apt-get install curl
Log as user (foo) and install the latest rvm
curl -L https://get.rvm.io | sudo bash -s stable
Add yourself (foo) to to the rvm group (vi /etc/group)
Install the following ruby options
rvm install ruby-1.8.7 rvm install ruby-1.9.3 rvm install rbx rvm install rbx-2.0.0-rc1 # This may be better in the future, but rbx 2 has several advantages rvm install ruby-1.9.3 rvm install jruby
- Install the packages needed. These should work across ruby versions:
rvm use 1.9.3 gem install rspec parslet treetop trollop rgl json gherkin cucumber
rvm use 1.8.7 rvm rubygems 1.3.5 gem install rspec-expectations gem install rspec parslet treetop trollop rgl json gherkin cucumber gem install reek flay flog
rvm use rbx gem install rspec parslet treetop trollop rgl json gherkin cucumber
... # any ruby version that you may want to use
- This are version specific
rvm use 1.8.7 gem install rvm use 1.9.3 gem install
rvm use rbx-2.0.0-rc1 gem install
MADA Related Stuff
- 6 - Install autofs
Create mada directory
mkdir /mada
Get autofs files from another machine
apt-get install -y autofs scp <username>@mada7:/etc/auto.* /etc/ ln -s /mada/users /soe initctl reload autofs
/etc/auto.mada should use nfs4. Something like this:
users -rw,intr,soft,noquota,noatime,fstype=nfs4 mada0.cse.ucsc.edu:/mada/users software -ro,intr,soft,noquota,noatime,fstype=nfs4 mada0.cse.ucsc.edu:/mada/software
edit /etc/default/nfs-common. If you have a NEED_IDMAPD variable, it must be set to true
NEED_IDMAPD=yes
- 7 - awk for cadence
apt-get install -y ksh csh original-awk ln -s /usr/bin/awk /bin/awk
- 8 - Set the firewall
ufw enable ufw default deny ufw reject auth ufw limit ssh/tcp ufw logging off
- 9 - Get tempcap from another ubuntu machine (mascd3, mascd9, ...), and some lib preparation:
scp <username>@mada0:/usr/lib/libtermcap.so.2.0.8 /usr/lib/ ln -s /usr/lib/libtermcap.so.2.0.8 /usr/lib/libtermcap.so.2
ln -sf /usr/lib/x86_64-linux-gnu/libtiff.so.4.3.4 /usr/lib/libtiff.so.3
- 10 - LDAP
Install
apt-get install -y ldap-auth-client libpam-krb5 krb5-user libsasl2-modules-gssapi-mit
Leave the kerberos config entry blank. You will copy it in the next step.
Create /ect/krb5.conf
scp <username>@mada7:/etc/krb5.conf /etc/
Copy the ldap certificate
scp <username>@mada7:/etc/ssl/certs/* /etc/ssl/certs/
Create "/etc/auth-client-config/profile.d/krb-ldap-config"
scp <username>@mada7:/etc/auth-client-config/profile.d/krb-ldap-config /etc/auth-client-config/profile.d/krb-ldap-config auth-client-config -a -p krb_ldap
Update ldap.conf
scp <username>@mada7:/etc/ldap.conf /etc/ldap.conf scp <username>@mada7:/etc/ldap/ldap.conf /etc/ldap/ldap.conf
Now you should be able to execute:
ldapsearch -x LLL
- 11 - Lock SOE people out of your machine :)
scp <username>@mada7:/etc/security/access.conf /etc/security/access.conf
You will need to add your local login to the machine, or you will lock yourself out:
vim /etc/security/access.conf
Go to very bottom of file and add your local user account to the list below root Example of the last lines of my file (Replace "ian_local" with your local login):
+ : root : ALL + : nx : ALL + : (masc) : ALL + : (vlsi) : ALL -:ALL :ALL
Now, you should see people with "getent passwd" but, calling "su - cormac" should say "permission denied", but, calling "su - renau" should work.
- 12 - Since the LDAP has timeouts, setup the local backup
apt-get install nss-updatedb nss_updatedb ldap cd /etc/cron.hourly/ scp <username>@mada7:/etc/cron.hourly/nss_updatedb_ldap . scp <username>@mada7:/etc/nsswitch.conf /etc
- 13 - Edit lines passwd and group in /etc/nsswitch.conf so that they are as follows (involves adding db before ldap)
sudo vi /etc/nsswitch.conf passwd: files db [NOTFOUND=return] ldap group: files db [NOTFOUND=return] ldap
- 14 - Allow other accounts to be shown at the login screen, and disable Guest user.
sudo vi /etc/lightdm/lightdm.conf
add the following lines to lightdm.conf
greeter-show-manual-login=true allow-guest=false
- 16 - Edit the /etc/idmapd.conf and make sure that you use the correct idmapd for nfs4. It should
set:
Domain = localdomain
- 15 - Restart your computer:
shutdown -r now
- 16 - Login to the GUI with your SOE login information. If you get a prompt asking for an "LDAP Password" this means that you typed your password incorrectly.
- 17 - Remove old desktop configuration files (KDE / GNOME / browser)
NOTE: Removing ".config" will delete any saved passwords in your browser, but will fix other problems. NOTE: You can use kubuntu(KDE), gnome, unity, or awesome as desktop/windows manager.
Logout of / kill any open X sessions. Switch to tty1 (CTRL + ALT + F1), and login with your SOE account.
rm -rf .config .kde* .gconf* .gnome* .awesome*
- 18 - Some Minor Patches for things that don't work properly
Disable apparmor
sudo invoke-rc.d apparmor stop sudo update-rc.d -f apparmor remove
- 19 - Add bash as default shell
cd /bin sudo ln -sf bash sh
- 20 - Using QEMU and ARM in 32 bit mode. We need to install 32 bit libraries for this (as root).
sudo apt-get install g++-multilib libxext-dev:i386 -y
- 21 - Install some fonts
sudo apt-get install libtiff4 t1-xfree86-nonfree ttf-xfree86-nonfree ttf-xfree86-nonfree-syriac xfonts-75dpi xfonts-100dpi
- 22 - Make sure that there is no file indexing (beagle)
sudo apt-get purge beagle
- 23 - Add/create to /etc/cron.daily/ntpdate for keep time adjusted (chmod 755 ntpdate)
#!/bin/sh ntpdate time.ucsc.edu
- 24 - Add a freenx server to your desktop
sudo apt-get install python-software-properties sudo add-apt-repository ppa:freenx-team sudo apt-get update sudo apt-get install freenx-server
- 25 - Add sendmail (no external forwarding)
sudo apt-get purge exim4 sudo apt-get install sendmail cd /etc/mail/ sudo scp <username>@mada0:/etc/mail/alias* . sudo scp <username>@mada0:/etc/mail/sendmail.* . sudo scp <username>@mada0:/etc/mail/generi* . sudo chmod 777 /var/spool/mqueue sudo /etc/init.d/sendmail restart
- 26- Install extra tools (should have been installed through dpkg --set-selection)
sudo apt-get install -y git-core libjpeg62 sudo apt-get install -y libglib2.0-dev zlib1g-dev build-essential sudo apt-get install -y bison flex sudo apt-get install -y libboost-dev sudo apt-get install -y ack-grep sudo apt-get install -y cmake sudo apt-get install -y gridengine-client libtool mutrace sudo apt-get install -y texlive-latex-base texlive-latex-extra texlive-latex-recommended preview-latex-style latex-xcolor latex-beamer texlive-science texlive-fonts-recommended
sudo apt-get install -y okular
- 27 - Copy libtermcap needed by some synopsys aps
sudo scp <username>@mada0:/usr/lib/libtermcap.so.2.0.8 /usr/lib/. sudo ln -s /usr/lib/libtermcap.so.2.0.8 /usr/lib/libtermcap.so.2
- 28 - Manually install glib for i386, and get some more 32 bit libs.
scp mascd9:/lib/i386-linux-gnu/libg.tar /tmp/ cd /lib/i386-linux-gnu sudo tar zxvf /tmp/libg.tar
Upgrade from Ubuntu 10.04
- 1- sudo apt-get dist-upgrade -y
- 2- Upgrade to Ubuntu 1.10:
https://help.ubuntu.com/community/MaverickUpgrades
Installing PBS Torque (server)
- mada0 will be the headnode (pbsserver). Install the following packages on mada0:
apt-get install torque-server torque-scheduler torque-client
- on other mada machines
apt-get install torque-client torque-mom
- Torque will use TCP ports 15000-15004. Either open the ports or allow them for mada machines based on their IP address.
ufw allow proto tcp from EACH_MADA_IP to any port 15000:15004
- Make sure server_name is properly set
vi /var/spool/torque/server_name
mada0
- on mada0, add the other mada machines as compute node:
vi /var/spool/torque/server_priv/nodes
- and add the following
mada8 np=16 mada1 np=48 mada2 np=4 mada3 np=4 mada4 np=4 mada5 np=4 mada6 np=4 mada7 np=4 mada0 np=10
on the all mada machines, set the pbsserver
vi /var/spool/torque/mom_priv/config
- add the following lines:
$pbsserver mada0 $logevent 255
- Now create a queue and run the server on mada0
qmgr -c "set server scheduling=true" qmgr -c "create queue batch queue_type=execution" qmgr -c "set queue batch started=true" qmgr -c "set queue batch enabled=true" qmgr -c "set queue batch resources_default.walltime=240:00:00" qmgr -c "set server default_queue=batch" qmgr -c "set server operators = root@mada0" qmgr -c "set server keep_completed = 0" qmgr -c "set queue batch resources_default.neednodes = 1:ppn=1"
- restart the pbs_server, pbs_client and pbs_sched on mada0, and pbs_mom and pbs_client on others
- pbsnode -a should show the status of all nodes. If nodes are down, check the firewall log to make sure it does not block the requests.
- check the queue
echo "sleep 10" | qsub -l nodes=1:ppn=1 qstat
Configuring mdadm RAID (servers)
- For mada0, we use mdadm raid 5.
- To assemble the hard drives in a raid array:
mdadm --assemble /dev/md0 /dev/disk/by-id/ID-OF-DISK /dev/disk/by-id/ID-OF-DISK2 ...
You can preserve the same alphabetic order in which the list of disk IDs appear.
Note that this assembles the raid /dev/md0 from the devices already prepared for raid. You might need to stop the raid if it is already running to be able to reassemble it:
mdadm --stop /dev/md0
- If the disk have not been prepared for raid before, then you need to create the raid partition tables first. We use the whole disk for raids.
mdadm --create --verbose /dev/md0 --level=5 --raid-devices=6 /dev/disk/by-id/ID-OF-DISK1 /dev/disk/by-id/ID-OF-DISK2 ...
You can preserve the same alphabetic order in which the list of disk IDs appear.
To see the preparation that will happen in the background:
watch cat /proc/mdstat
To see the details of the array:
mdadm --detail /dev/md0
- Then you need to format the disks:
mkfs.ext4 /dev/md0
- update the /etc/fstab.
Lets say we want to mount the raid at /mirror
mkdir /mirror vi /etc/fstab
and make sure you specify the mount point for the raind.
/dev/md0 /mirror ext4 rw,intr,soft,noquota,relatime 0 0
then mount the raid:
mount -a
Configuring zfs RAID (servers)
- Install the zfs kernel module (not fuse)
sudo add-apt-repository ppa:zfs-native/stable
- Format the SDDs to have the following partitions:
1 /boot : 4GB 2 cache : 80GB 3 log: 4GB 4 root : 32GB 5 swap : 4GB
- create the radz1 zpool
zpool create tank raidz1 sdc sdd sde sdf sdg sdh zpool status tank
zpool iostat -v
- add read and write caches. To make sure you add the right partition, use "lsblk" to ensure the right partition.
zppol add tank cache /dev/disk/by-id/ata-FOO_disk1_part2 /dev/disk/by-id/ata-FOO_disk2_part2 log mirror /dev/disk/by-id/ata-FOO_disk1_part3 /dev/disk/by-id/ata-FOO_disk2_part3
zpool iostat -v
- Now prepare the folders
zfs create tank/users zfs create tank/software zfs create tank/servers
Setting up CUPS for Printing
- 1. Make sure cups and cups-bsd are installed:
sudo apt-get install cups
- 2. Go to /etc/cups directory:
cd /etc/cups
- 3. Add file called "client.conf" with this in it (you may need sudo access to create new file):
ServerName cups-01.cse.ucsc.edu ServerName spooler4.cse.ucsc.edu spooler4.cse.ucsc.edu
- 4. Save the client.conf file and restart the cups server:
/etc/init.d/cups restart
- You should now be able to print. If you are not seeing a list of printer or are getting an error saying something like "client error: forbidden", then your machine does not have access to the VLAN printer network. Contact the IT tech staff and this issue should be resolved (Eric Shell helped with this).
Setting up Cadence Virtuoso on your Ubuntu 12.04 Desktop
1. cd /tmp
2. sudo apt-get update
3. sudo apt-get install nscd csh ksh gawk libelf1 libelf-dev ia32-libs libelf1:i386
This installs the library dependencies.
4. sudo apt-get update
5. sudo ln -s /usr/bin/awk /bin/awk
This ensures the awk utility is also available at the locations typically found on Solaris, CentOS, SuSE flavors.