My Basic Setup on OSX Lion

On this blog post I will covered what I learned to setup my basic environment that use from terminal to do my development and research on my Macbook, in addition to the steps you will see here I also. In addition to this I download and install VMware Fusion, Nessus, Netbeans, Colloquy, Chrome, Firefox, Adium, Skype, WebSecurify just to mention a few. But the major pain point has always been not the pre-package apps but the terminal environment so here I will share my basic setup of that environment.

The first thing is to do a software update and make sure you have any new patches that there may be from Apple for the OS, then go in to the App Store and download and install the latest version of Xcode for Lion once this has been done we can install the latest version of Homebrew, an alternative package manager for OSX similar to MacPorts.

---

Install Homebrew

ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"

Before installing any packages I recommend you modify that shell profile to add environment variables for the new compiler in version 4 of Xcode so we do not get any errors compiling Ruby Gems, Python Eggs and software found in the Homebrew Formulas.  To be able to modify the system wide profile we must first make it writable and then open it for editing:

  sudo chmod +w /etc/profile
  sudo vim /etc/profile

Once open append the lines bellow to set the proper variables. These variables will enable color for terminal and the files shown just like most default settings on Linux, set compiler flags and the ever so annoying SVN Keywords command:

---

Install Base Packages

On my system I started by adding some base tools for formulas that did not needed any modifications on my part, I installed Nmap, THC Hydra, MacVim, Tmux, Hping and PostgreSQL:

brew install nmap hydra macvim tmux wget hping postgresql readline

The PostgreSQL package after install will need a bit of more work since a system database must be created, set up the engine to start at user logon and create a base user and database for Metasploit that is why I use PostgreSQL for. Lets start by initializing the database:

  initdb /usr/local/var/postgres

Configure Database for Startup at Logon

  mkdir -p ~/Library/LaunchAgents
  cp /usr/local/Cellar/postgresql/9.0.4/org.postgresql.postgres.plist ~/Library/LaunchAgents/

Start PostgreSQL Server and Create User and Database  for Metasploit

  1: # Start databse sever
  2: pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start
  3: # Create user named msf, provide a good password for it and answer no to all other questions
  4: createuser msf -P -h localhost
  5: # Create database for use with metasploit called msf and make the user msf the owner
  6: createdb -O msf msf -h localhost

Once PostgreSQL is up and running I can work on installing other packages that need a bit more work.

The first thing would be to install Apples Package of Java, to do this just issue the command Java in the console and follow the instructions that will appear on the screen.

I start by editing John the Ripper formula to download and Install the latest community patched version

  brew edit john

Make sure it looks like this:

Make sure you modified the url variable and the md5 variable, and remove the patch section like shown above. Once modified save and install using the command below:

  1: brew install john

Next we install Libdnet since we will need this to be able to install scapy later on, we need to edit the formula and add the additional call to install the python libraries for us:

The line that needs to be added is line afte the make install command where you move in to the python directory and install the python libraries. Installation is the same as the other formulas above using the brew install command.

---

Configuring RVM and Installing Ruby Versions

As many of you may already know I contribute a lot of time, code and resources for free to the Metasploit project and consider my self a Metasploit Junkie when it comes to coding for it, so for me having Ruby install just right is important as well as to have several versions of ruby to test against, for this I use RVM the Ruby Version Manager.

To install RVM system wide you issue the following command:

bash < <(curl -s https://rvm.beginrescueend.com/install/rvm)

This will install RVM on your system, use sudo and specify ruby befole launchin program like msfconsole to ensure you are using the right ruby when root privilages are needed. Sadly Ruby on OSX is compiled to use LibEdit and not GNU Readline, Metasploit takes advantage of many of the features of GNU Readline for that reason we must do some configuration ahead of time to make sure the versions of ruby we configure are compatible and will not cause problems down the road.

Once it is installed we must override the compilation parameters of RVM to make sure it always compiles against the GNU version of Readline we just installed we open the RVM db file with the following command:

vim ~/.rvm/user/db

and we append to the end the following line:

ruby_configure_flags=--with-readline-dir=/usr/local/Cellar/readline/6.2.1/

Now that this is done, we can start installing the most used versions of Ruby, you will notice that I installed a specific patch set for Ruby 1.9.1 since versions above that one have a know problem with Gem and you will see I set the system default to 1.9.2.

rvm install 1.8.7

rvm install 1.9.1-p378

rvm install 1.9.2

rvm --default 1.9.2

Once this is done we can start installing the necessary gems, but first I like to set up a gem resource file so documentation and ri are not generated so as to speed installation and update of the gems, if you use from terminal the docs and ri you can skip this step:

echo "gem: --no-ri --no-rdoc" >> ~/.gemrc

Now we can install the gems we need on the RVM versions of Ruby and on the local version that comes with Lion:

rvm gem install wirble sqlite3 pg activerecord wirb hirb awesome_print interactive_editor
gem install wirble sqlite3 pg activerecord wirb hirb awesome_print interactive_editor

Now that we have Ruby we can download and install Metasploit

---

Installing Metasploit

In my case since I do mostly development with Metasploit and do not use it on a daily basis now that I changed jobs, my set up will not be a system wide one, since many times I will branch and make copies of the framework to experiment and modify. For this I create in my home directory a folder called Development and place Metasploit there:

mkdir ~/Development

I change to that directory and download my latest copy of the framework.

svn co https://www.metasploit.com/svn/framework3/trunk/ msf

Once this finishes you can move inside the folder a launch msfconsole by running the command

cd ~/Development/msf

./msfconsole

An you should be greeted by the banner and the prompt. Once this is done we must install the pcaprub gem that comes with the framework on all the version of Ruby that we have installed with RVM.

cd external/pcapryb

rvm 1.8.7

ruby extconf.rb && make && sudo make install

rvm 1.9.1-p378

ruby extconf.rb && make && sudo make install

rvm 1.9.2

ruby extconf.rb && make && sudo make install

rvm system

ruby extconf.rb && make && sudo make install

Once this is done we have to configure MSF to always logon to the PostgreSQL Database we configured. For this we create a yaml configuration file in our msf pofile.

vim ~/.msf4/database.yml

We then enter the following yaml parameters:

Next time we launch msfconsole we should see the creation of the tables and when a db_status is issued we should be connected to the database we specified.

---

Installing Scapy and DNSRecon

Another of the tools I tend to use and code against is Scapy, I also wrote a small enumeration tool in python call DNSRecon that I maintain and plan to expand shortly . To install lit first we must prep our Python environment and make sure we have all the necessary libraries, lets start with those we can automate, I use pip for python since it will not install incomplete downloads and will allow me to uninstall and install a later version of a library.

sudo easy_install pip 

sudo pip install Mercurial

sudo pip install pycrypto

sudo pip install pybonjour

sudo pip install dnspython

sudo pip install netaddr

One library that we can not download from the command like and needs to be download by hand is pylibpcap, once we have downloaded the tar.gz file we can decompress it and install it:

tar xvzf pylibpcap-0.6.2.tar.gz

cd pylibpcap-0.6.2

sudo python setup.py install

After we have the libraries install I can download the latest repos of the projects:

cd ~/Development

# Scapy Community Repo

hg clone http://hg.secdev.org/scapy-com

# Scapy Main Branch

hg clone http://hg.secdev.org/scapy

# DNSRecon

git clone https://github.com/darkoperator/dnsrecon.git

To install the latest version of Scapy we just go in to the Scapy folder and install it

cd scapy

sudo python setup.py install

The reason I do not install the community edition since it has the latest contributions is that anybody can contribute to it and I tend to check each update before even thinking of running scapy out of it.

---

Configuring System Resource Files

Now that I have this packages set, I setup my tmux.conf and vimrc files since I use tmux to manage my terminal screens and positions and vim for editing files quickly in the termenial. I start by copying my tmux conf file to my home folder and to the root user home folder, since some times I do stuff as root. the filename is .tmux.conf and the contents is as follows:

I invite you to read it and modify as to your own preferences.

Next I modify and set my .vimrc file to my liking for highlighting text instances, syntax highlighting, line numbers and some specifics for NASL and Ruby.  Firs I download and install my backup of my .vim folder. You will see that it is pretty simple sine I do not do any major coding on vim and tend to use an IDE for it. Vim Resource File:

I do tend to have a custom IRB Resource file so as to make life easier and text easier to read when working on irb, here is the simple version of it:

It does looks like much but in fact this is my basic setup for Lion. Now to make a backup  

Took me 2 days to get all of this sorted and tested. Hope some of you find it useful.

Update on Monday, July 25, 2011 at 1:47PM by Carlos Perez

After testing with otool found that readline was not linking properly, also had several problems with system wide RVM so I changed the guide to single user.

Update on Saturday, July 30, 2011 at 12:37PM by Carlos Perez

Made some minor fixes and removed the Wireshark Brew install, I do recomend the package as a better option, one less application to link to the /Applications folder and less dependancies to worry abaout, since a lot of custom modding of formulas can be counter productive in the long run, at least for my setup. 

Comments

[Kx499]

I usually read up on the different stuff you're contributing to the framework modules, scripts, etc. The post exploitation stuff is great...actually have taken an interest in that side of things as well. This post on MacBook setup is good stuff, I'm looking to educate myself on the Macs in the near future and this will come in handy when I actually go get one and try it out. Just never had any real exposure to them and with them gaining market share like they have been, i really need to.

On a side note I got a chuckle when I read the part about your script called dnsrecon.py, only because earlier this year I wrote a script with similar functionality (although nowhere near as robust) and called it dnsrecon.py when I was taking the offsec PWB class. I was learning python and wanted to recreate the perl scrip that came with BT4 in python so I cam up with my quick and dirty python rendition of dnsenum.pl. Anyways, I uploaded it to my site, have a look at it when you get a chance http://www.losingfocus.net/scripts/dnsrecon.py

[Mir_Allen]

Hello, how to set up development environment in ubuntu ?

[Shawn Workman]

I had some problems once I got to Wireshark.. Errors installing libgpg-error.. It goes on with others. I just fix them as they come up.

I ended up commenting out a line in the formula to get it to work..

ENV.universal_binary

After I commented it out, it worked fine.. Probably due to the fact that XCode 4.2 won't build Universal Binaries.. Or at least that is what I think ;) Please let me know if I am wrong.

[yasha]

Hi Carlos
thanks for the tutorial very useful ,I'm trying to setup my box with mysql instead of pg can you give us some help
with that.

[Carlos Perez]

Sadly MySQL is no longer supported only Postgres as of version 4.0

[Nicolas Sosa]

Hi Carlos, thank you for the tutorial and congratulations for the podcast in spanish, i'm from Argentina! I'd like to ask you, what macbook do you have, do you use more than one for development?

[Carlos Perez]

Thanks glad you like it :) I currently use a MacBook Pro 13" from late 2009, my wife has an iMac and a Air, buts most OSX testing is done on VMs

[ravjr]

I've been meaning to document a base MAC OSX install, but you beat me to it. Thank you very much (i.e., for the excellent write-up!) Besides the minor issues of permissions, paths, etc. it was pretty straight. The only thing I thought would be helpful would be for installing the rvm:

curl -L get.rvm.io | bash -s stable

[kernelsmith]

Carlos,
Seems like there is some content missing. In the /etc/profile section and after "Once open append the lines bellow to set the proper variables" there are no lines given. It goes straight into "Install Base Packages"

-Josh

[zen]

I'm very sorry but msf guide have not ask question about:

[bash-output]
$ brew install homebrew/versions/ruby193
==> Downloading http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p448.tar.bz2
######################################################################## 100.0%
==> ./configure --prefix=/usr/local/Cellar/ruby193/1.9.3-p448 --enable-shared --disable-tcltk-framework --with-out-ext=tcl --with-out-ext=tk --disable-i
==> make
brew: superenv removed: -O3 -ggdb -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wpointer-arith -Wwrite-strings -Wdeclaration-after-statement -Wshorten-64-to-32 -Wimplicit-function-declaration -L/usr/local/lib
generating encdb.h
make: *** [.rbconfig.time] Illegal instruction: 4
make: *** Waiting for unfinished jobs....
make: *** [encdb.h] Illegal instruction: 4

READ THIS: https://github.com/mxcl/homebrew/wiki/troubleshooting
[/bash-output]

It's a OS X 10.7.5 in 64 mode running … Can you help to fix it?

With RVM sometimes it possible to install 1.9.3 but have an issues for other environment.

TNX!

[Carlos Perez]

I would recommend opening a issue at the homebrew GitHub. Only thing the comes to mind is to specify clang for linking with the --with-gcc=clang option when installing ruby. If you can not use RVM try chruby as another option