My Basic Setup on OSX Lion
Sunday, July 24, 2011 at 10:39PM On this blog post I will covered what I learned to setup my basic environment that use from terminal to do my development and research on my Macbook, in addition to the steps you will see here I also. In addition to this I download and install VMware Fusion, Nessus, Netbeans, Colloquy, Chrome, Firefox, Adium, Skype, WebSecurify just to mention a few. But the major pain point has always been not the pre-package apps but the terminal environment so here I will share my basic setup of that environment.
The first thing is to do a software update and make sure you have any new patches that there may be from Apple for the OS, then go in to the App Store and download and install the latest version of Xcode for Lion once this has been done we can install the latest version of Homebrew, an alternative package manager for OSX similar to MacPorts.
Install Homebrew
ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"
Before installing any packages I recommend you modify that shell profile to add environment variables for the new compiler in version 4 of Xcode so we do not get any errors compiling Ruby Gems, Python Eggs and software found in the Homebrew Formulas. To be able to modify the system wide profile we must first make it writable and then open it for editing:
sudo chmod +w /etc/profile
sudo vim /etc/profile
Once open append the lines bellow to set the proper variables. These variables will enable color for terminal and the files shown just like most default settings on Linux, set compiler flags and the ever so annoying SVN Keywords command:
| # Terminal Colors | |
| export CLICOLOR=1 | |
| export LSCOLORS=GxFxCxDxBxegedabagaced | |
| # Compilier options | |
| export ARCHFLAGS="-arch x86_64" | |
| export CC=/usr/bin/gcc-4.2 | |
| # SVN Keyowkrds | |
| alias svnkeywords='svn propset svn:keywords '\''Rev Revision Id Header'\''' |
Install Base Packages
On my system I started by adding some base tools for formulas that did not needed any modifications on my part, I installed Nmap, THC Hydra, MacVim, Tmux, Hping and PostgreSQL:
brew install nmap hydra macvim tmux wget hping postgresql readline
The PostgreSQL package after install will need a bit of more work since a system database must be created, set up the engine to start at user logon and create a base user and database for Metasploit that is why I use PostgreSQL for. Lets start by initializing the database:
initdb /usr/local/var/postgres
Configure Database for Startup at Logon
mkdir -p ~/Library/LaunchAgentscp /usr/local/Cellar/postgresql/9.0.4/org.postgresql.postgres.plist ~/Library/LaunchAgents/
Start PostgreSQL Server and Create User and Database for Metasploit
1: # Start databse sever2: pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start3: # Create user named msf, provide a good password for it and answer no to all other questions4: createuser msf -P -h localhost5: # Create database for use with metasploit called msf and make the user msf the owner6: createdb -O msf msf -h localhost
Once PostgreSQL is up and running I can work on installing other packages that need a bit more work.
The first thing would be to install Apples Package of Java, to do this just issue the command Java in the console and follow the instructions that will appear on the screen.
I start by editing John the Ripper formula to download and Install the latest community patched version
brew edit john
Make sure it looks like this:
| require 'formula' | |
| class John < Formula | |
| url 'http://www.openwall.com/john/g/john-1.7.8-jumbo-4.tar.bz2' | |
| homepage 'http://www.openwall.com/john/' | |
| md5 '7b50641248e9570341d5474b7c83f087' | |
| fails_with_llvm | |
| def install | |
| ENV.deparallelize | |
| arch = Hardware.is_64_bit? ? '64' : 'sse2' | |
| Dir.chdir 'src' do | |
| system "make clean macosx-x86-#{arch}" | |
| end | |
| rm 'README' | |
| # using mv over bin.install due to problem moving sym links | |
| mv 'run', bin | |
| chmod_R 0755, bin | |
| end | |
| end |
Make sure you modified the url variable and the md5 variable, and remove the patch section like shown above. Once modified save and install using the command below:
1: brew install john
Next we install Libdnet since we will need this to be able to install scapy later on, we need to edit the formula and add the additional call to install the python libraries for us:
| require 'formula' | |
| class Libdnet < Formula | |
| url 'http://libdnet.googlecode.com/files/libdnet-1.12.tgz' | |
| homepage 'http://code.google.com/p/libdnet/' | |
| md5 '9253ef6de1b5e28e9c9a62b882e44cc9' | |
| def install | |
| system "./configure", "--disable-debug", "--disable-dependency-tracking", | |
| "--prefix=#{prefix}", | |
| "--mandir=#{man}" | |
| system "make install" | |
| system "cd python && /usr/bin/python setup.py install" | |
| end | |
| end |
The line that needs to be added is line afte the make install command where you move in to the python directory and install the python libraries. Installation is the same as the other formulas above using the brew install command.
Configuring RVM and Installing Ruby Versions
As many of you may already know I contribute a lot of time, code and resources for free to the Metasploit project and consider my self a Metasploit Junkie when it comes to coding for it, so for me having Ruby install just right is important as well as to have several versions of ruby to test against, for this I use RVM the Ruby Version Manager.
To install RVM system wide you issue the following command:
bash < <(curl -s https://rvm.beginrescueend.com/install/rvm)
This will install RVM on your system, use sudo and specify ruby befole launchin program like msfconsole to ensure you are using the right ruby when root privilages are needed. Sadly Ruby on OSX is compiled to use LibEdit and not GNU Readline, Metasploit takes advantage of many of the features of GNU Readline for that reason we must do some configuration ahead of time to make sure the versions of ruby we configure are compatible and will not cause problems down the road.
Once it is installed we must override the compilation parameters of RVM to make sure it always compiles against the GNU version of Readline we just installed we open the RVM db file with the following command:
vim ~/.rvm/user/db
and we append to the end the following line:
ruby_configure_flags=--with-readline-dir=/usr/local/Cellar/readline/6.2.1/
Now that this is done, we can start installing the most used versions of Ruby, you will notice that I installed a specific patch set for Ruby 1.9.1 since versions above that one have a know problem with Gem and you will see I set the system default to 1.9.2.
rvm install 1.8.7rvm install 1.9.1-p378rvm install 1.9.2rvm --default 1.9.2
Once this is done we can start installing the necessary gems, but first I like to set up a gem resource file so documentation and ri are not generated so as to speed installation and update of the gems, if you use from terminal the docs and ri you can skip this step:
echo "gem: --no-ri --no-rdoc" >> ~/.gemrc
Now we can install the gems we need on the RVM versions of Ruby and on the local version that comes with Lion:
rvm gem install wirble sqlite3 pg activerecord wirb hirb awesome_print interactive_editorgem install wirble sqlite3 pg activerecord wirb hirb awesome_print interactive_editor
Now that we have Ruby we can download and install Metasploit
Installing Metasploit
In my case since I do mostly development with Metasploit and do not use it on a daily basis now that I changed jobs, my set up will not be a system wide one, since many times I will branch and make copies of the framework to experiment and modify. For this I create in my home directory a folder called Development and place Metasploit there:
mkdir ~/Development
I change to that directory and download my latest copy of the framework.
svn co https://www.metasploit.com/svn/framework3/trunk/ msf
Once this finishes you can move inside the folder a launch msfconsole by running the command
cd ~/Development/msf./msfconsole
An you should be greeted by the banner and the prompt. Once this is done we must install the pcaprub gem that comes with the framework on all the version of Ruby that we have installed with RVM.
cd external/pcaprybrvm 1.8.7ruby extconf.rb && make && sudo make installrvm 1.9.1-p378ruby extconf.rb && make && sudo make installrvm 1.9.2ruby extconf.rb && make && sudo make installrvm systemruby extconf.rb && make && sudo make install
Once this is done we have to configure MSF to always logon to the PostgreSQL Database we configured. For this we create a yaml configuration file in our msf pofile.
vim ~/.msf4/database.yml
We then enter the following yaml parameters:
| production: | |
| adapter: postgresql | |
| database: <database name> | |
| username: <use we created> | |
| password: <Password we gave the user> | |
| host: 127.0.0.1 | |
| port: 5432 | |
| pool: 75 | |
| timeout: 5 |
Next time we launch msfconsole we should see the creation of the tables and when a db_status is issued we should be connected to the database we specified.
Installing Scapy and DNSRecon
Another of the tools I tend to use and code against is Scapy, I also wrote a small enumeration tool in python call DNSRecon that I maintain and plan to expand shortly . To install lit first we must prep our Python environment and make sure we have all the necessary libraries, lets start with those we can automate, I use pip for python since it will not install incomplete downloads and will allow me to uninstall and install a later version of a library.
sudo easy_install pipsudo pip install Mercurialsudo pip install pycryptosudo pip install pybonjoursudo pip install dnspythonsudo pip install netaddr
One library that we can not download from the command like and needs to be download by hand is pylibpcap, once we have downloaded the tar.gz file we can decompress it and install it:
tar xvzf pylibpcap-0.6.2.tar.gzcd pylibpcap-0.6.2sudo python setup.py install
After we have the libraries install I can download the latest repos of the projects:
cd ~/Development# Scapy Community Repohg clone http://hg.secdev.org/scapy-com# Scapy Main Branchhg clone http://hg.secdev.org/scapy# DNSRecongit clone https://github.com/darkoperator/dnsrecon.git
To install the latest version of Scapy we just go in to the Scapy folder and install it
cd scapysudo python setup.py install
The reason I do not install the community edition since it has the latest contributions is that anybody can contribute to it and I tend to check each update before even thinking of running scapy out of it.
Configuring System Resource Files
Now that I have this packages set, I setup my tmux.conf and vimrc files since I use tmux to manage my terminal screens and positions and vim for editing files quickly in the termenial. I start by copying my tmux conf file to my home folder and to the root user home folder, since some times I do stuff as root. the filename is .tmux.conf and the contents is as follows:
| # ` is an interesting key for a prefix | |
| set-option -g prefix ` | |
| # set-option -g prefix C-a | |
| unbind-key C-b | |
| bind-key C-a last-window | |
| bind-key ` last-window | |
| bind-key a send-prefix | |
| # Set ways to split screen | |
| unbind % | |
| bind "|" split-window -h | |
| bind "-" split-window -v | |
| # we might need ` at some point, allow switching | |
| # we can also send the prefix char with `-a | |
| bind-key F11 set-option -g prefix C-a | |
| bind-key F12 set-option -g prefix ` | |
| # 0 is too far from ` ;) | |
| set -g base-index 1 | |
| # set-option -g default-terminal "screen-256color" | |
| set-option -g mouse-select-pane on | |
| set-option -g status-keys vi | |
| set-option -g bell-action any | |
| set-option -g set-titles on | |
| set-option -g set-titles-string '#H:#S.#I.#P #W #T' # window number,program name,active (or not) | |
| set-option -g visual-bell on | |
| setw -g mode-keys vi | |
| setw -g mode-mouse on | |
| setw -g monitor-activity on | |
| bind e previous-window | |
| bind f next-window | |
| #bind j up-pane | |
| #bind k down-pane | |
| set-option -g status-utf8 on | |
| # set-option -g status-justify center | |
| set-option -g status-justify left | |
| set-option -g status-bg black | |
| set-option -g status-fg white | |
| set-option -g status-left-length 40 | |
| set-option -g pane-active-border-fg green | |
| set-option -g pane-active-border-bg black | |
| set-option -g pane-border-fg white | |
| set-option -g pane-border-bg black | |
| set-option -g message-fg black | |
| set-option -g message-bg green | |
| #setw -g mode-bg black | |
| setw -g window-status-bg black | |
| setw -g window-status-current-fg green | |
| setw -g window-status-alert-attr default | |
| setw -g window-status-alert-fg yellow | |
| set -g status-left '#[fg=red]#H#[fg=green]:#[fg=white]#S #[fg=green]][#[default]' | |
| # set -g status-right '#[fg=green]][#[fg=white] #T #[fg=green]][ #[fg=blue]%Y-%m-%d #[fg=white]%H:%M#[default]' | |
| set -g status-right '#[fg=green]][ #[fg=blue]%Y-%m-%d #[fg=white]%H:%M#[default]' | |
| set -g history-limit 4096 | |
| bind r source-file ~/.tmux.conf |
I invite you to read it and modify as to your own preferences.
Next I modify and set my .vimrc file to my liking for highlighting text instances, syntax highlighting, line numbers and some specifics for NASL and Ruby. Firs I download and install my backup of my .vim folder. You will see that it is pretty simple sine I do not do any major coding on vim and tend to use an IDE for it. Vim Resource File:
| autocmd FileType ruby,eruby set omnifunc=rubycomplete#Complete | |
| autocmd FileType ruby,eruby let g:rubycomplete_buffer_loading = 1 | |
| autocmd FileType ruby,eruby let g:rubycomplete_rails = 1 | |
| autocmd FileType ruby,eruby let g:rubycomplete_classes_in_global = 1 | |
| autocmd FileType ruby,eruby let g:rubycomplete_include_object = 1 | |
| autocmd FileType ruby,eruby let g:rubycomplete_include_objectspace = 1 | |
| autocmd CursorMoved * silent! exe printf('match IncSearch /\<%s\>/', expand('<cword>')) | |
| set nu " Set line numbers | |
| set nocompatible " We're running Vim, not Vi! | |
| syntax on " Enable syntax highlighting | |
| filetype on " Enable filetype detection | |
| filetype indent on " Enable filetype-specific indenting | |
| filetype plugin on " Enable filetype-specific plugins | |
| au BufEnter *.nasl setfiletype nasl | |
| au BufEnter *.inc setfiletype nasl |
I do tend to have a custom IRB Resource file so as to make life easier and text easier to read when working on irb, here is the simple version of it:
| require 'rubygems' | |
| require 'wirble' | |
| require 'irb/completion' | |
| require 'pp' | |
| IRB.conf[:AUTO_INDENT] = true | |
| Wirble.init | |
| Wirble.colorize | |
| class Object | |
| # get all the methods for an object that aren't basic methods from Object | |
| def local_methods | |
| (methods - Object.instance_methods).sort | |
| end | |
| end | |
| ANSI_BOLD = "\033[1m" | |
| ANSI_RESET = "\033[0m" | |
| ANSI_LGRAY = "\033[0;37m" | |
| ANSI_GRAY = "\033[1;30m" | |
| def pm(obj, *options) # Print methods | |
| methods = obj.methods | |
| methods -= Object.methods unless options.include? :more | |
| filter = options.select {|opt| opt.kind_of? Regexp}.first | |
| methods = methods.select {|name| name =~ filter} if filter | |
| data = methods.sort.collect do |name| | |
| method = obj.method(name) | |
| if method.arity == 0 | |
| args = "()" | |
| elsif method.arity > 0 | |
| n = method.arity | |
| args = "(#{(1..n).collect {|i| "arg#{i}"}.join(", ")})" | |
| elsif method.arity < 0 | |
| n = -method.arity | |
| args = "(#{(1..n).collect {|i| "arg#{i}"}.join(", ")}, ...)" | |
| end | |
| klass = $1 if method.inspect =~ /Method: (.*?)#/ | |
| [name, args, klass] | |
| end | |
| max_name = data.collect {|item| item[0].size}.max | |
| max_args = data.collect {|item| item[1].size}.max | |
| data.each do |item| | |
| print " #{ANSI_BOLD}#{item[0].to_s.rjust(max_name)}#{ANSI_RESET}" | |
| print "#{ANSI_GRAY}#{item[1].ljust(max_args)}#{ANSI_RESET}" | |
| print " #{ANSI_LGRAY}#{item[2]}#{ANSI_RESET}\n" | |
| end | |
| data.size | |
| end |
It does looks like much but in fact this is my basic setup for Lion. Now to make a backup
Took me 2 days to get all of this sorted and tested. Hope some of you find it useful.
Carlos Perez
After testing with otool found that readline was not linking properly, also had several problems with system wide RVM so I changed the guide to single user.
Carlos Perez
Made some minor fixes and removed the Wireshark Brew install, I do recomend the package as a better option, one less application to link to the /Applications folder and less dependancies to worry abaout, since a lot of custom modding of formulas can be counter productive in the long run, at least for my setup.
