-
Notifications
You must be signed in to change notification settings - Fork 15
Software installation
This is a guide for installing the software you need for development, from scratch, whether on Mac OSX, Linux or Windows. It's likely you'll already have some of the things below installed (e.g. Apache) if you're already doing some kind of software development, so some sections you may be able to skip.
This guide is written by Martin who is a newcomer to Ruby on Rails (and who develops generally on Windows!), so should hopefully be understandable by a range of web developers.
We're using some clever geo software that doesn't yet run on Windows. Therefore there is no point trying to install Ruby on Rails on Windows, as you'll get to a stage where one key component simply won't work.
Therefore, the simplest thing is just to install a 'Virtual Machine' on Windows. It's actually reasonably straightforward and basically just gives you a Linux installation in a window in Windows.
Step-by-step instructions are at: Software-installation:-Adding-a-Ubuntu-Virtual-Machine-in-Windows
These are useful bits of general Linux software.
Note that, throughout this guide, we're using apt-get, which is the really excellent command-line package management tool that Ubuntu (and its parent, Debian) uses.
sudo apt-get install curl
sudo apt-get install libxslt-dev libxml2-dev libsqlite3-dev
sudo apt-get install build-essential
sudo apt-get install locate
We'll need Git so that we can get the toolkit software from our Github repository, and commit back to it. (If you've only used SVN before, you'll find Git a real breath of fresh air!)
If you're new to Git, here's a quick Git crash course for SVN users: https://git.wiki.kernel.org/articles/g/i/t/GitSvnCrashCourse_512d.html
sudo apt-get install git-core git-gui git-doc
This is as described at: http://help.github.com/linux-set-up-git/ (which is for Linux) or http://help.github.com/mac-set-up-git/ (on Mac OSX).
Github doesn't use a password system for getting/adding code. Instead, it requires that you create a key (basically like a token). The key is stored in your home directory and is automatically presented to Github when you connect to it from the command-line. This avoids having to quote a password each time.
Here we create the key then log in to Github to add it in your user control panel. Should your computer be hacked, you can just go to Github to rescind the key.
- Generate a key on your computer:
ssh-keygen -t rsa -C "[email protected]"
- View the key:
less ~/.ssh/id_rsa.pub
- Copy that text to the clipboard.
- Open a web browser and go to https://github.com/account/ssh .
- Paste in the key.
- Test using:
ssh -T [email protected]
Add your name and e-mail to the Git software configuration on your computer.
git config --global user.name "Firstname Lastname"
git config --global user.email "[email protected]"
We'll need the Apache webserver so we can serve the files efficiently and easily.
sudo apt-get install apache2 apache2-mpm-prefork apache2-prefork-dev
PostgreSQL (usually known as 'Postgres') is a very robust relational database management system (RDBMS) - MySQL is a commonly-used RDBMS too but we won't b using MySQL. Unlike MySQL, Postgres has excellent support for geospatial queries. For the geo aspects it uses an extension called Postgis, which we'll install later below.
If you're used to MySQL, don't worry! A lot of the concepts are the same, and we'll point out below any key differences that you'll encounter (there are only a few).
Postgres 9.0 unfortunately isn't included with Ubuntu 11.10. So we have to use a software repository containing 'backports' (older versions). Ubuntu has a list of sources of such software repositories, so we have to add a new source to it so that the backport software will be found during installation.
- Open the sources list config file /etc/apt/sources.list for editing. Here, we're using Pico which is a simple text editor.
sudo pico -w /etc/apt/sources.list
- Then add:
# Add this new line to the end of the file
deb http://backports.debian.org/debian-backports squeeze-backports main
- Save the file. In Pico, use Control-x to save and then y to confirm.
- Now register this new repository, in the process installing a security key that Ubuntu will check when using the repository.
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys AED4B06F473041FA
sudo apt-get update
Install Postgres 9.0 and related components from the backports repository (that we added above).
sudo apt-get -t squeeze-backports install postgresql-9.0 postgresql-contrib-9.0 postgresql-server-dev-9.0
PostGIS is the GIS component of Postgres. It effectively adds lots of geometry types and stored procedures that we can use in SQL, making things like "Find me objects near this point" much easier to do, and very fast.
Unfortunately there doesn't any more seem to be a PostGIS package that works for Postgres 9.0, so we have to install this manually if the automatic installation doesn't work.
If PostGIS installed correctly, it will add a load of scripts in the main Postgres installation directory. These scripts can be used by various installer programs to add the PostGIS geometry types to any database that needs them.
It's worth trying this first, just in case. At the time of writing (February 2012), however, the package seems to have disappeared.
# These notes are based on http://www.sourcepole.ch/2011/3/25/ubuntu-postgis-package-for-postgresql-9-0
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:ubuntugis/ubuntugis-unstable
sudo add-apt-repository ppa:pitti/postgresql
sudo add-apt-repository ppa:pi-deb/gis
sudo apt-get update
sudo apt-get install postgresql-9.0-postgis
Make sure there are now files at /usr/share/postgresql/9.0/contrib/postgis-1.5. If not, something went wrong.
ls -lAF /usr/share/postgresql/9.0/contrib/postgis-1.5
# These notes are based on http://wiki.openstreetmap.org/wiki/Osmosis/PostGIS_Setup#Postgres_9.0.1
sudo apt-get install libgeos-c*
sudo apt-get install proj
cd /tmp/
wget http://www.postgis.org/download/postgis-1.5.3.tar.gz
tar -xvzf postgis-1.5.3.tar.gz
cd postgis-1.5.3
./configure
make
#make comments
sudo make install
#make comments-install
Make sure there are now files at /usr/share/postgresql/9.0/contrib/postgis-1.5. If not, something went wrong.
ls -lAF /usr/share/postgresql/9.0/contrib/postgis-1.5
Postgres uses roles (effectively users) which we have to connect to it with in order to do anything. Unlike MySQL, Postgres will, by default, use the username of the Unix (Ubuntu) user you're connecting to it with. In other words, by default it doesn't use usernames/passwords itself, but just trusts what the OS gives it.
We're going to add a new Postgres user so that our software connects explicitly as a user. We're then going to enable that 'connect with username/password' functionality in the Postgres configuration.
Here, sudo -u postgres switches us to the Postgres user account (which means that Postgres itself don't ask us for a password). We then run 'createuser' which is a Unix command installed as part of the Postgres installation that creates a user.
# Create a new postgres user that our application runtime will us. Make a note of the password that you create!
# Note that `createuser -P` will issue a prompt for the password of the new user.
sudo -u postgres createuser -P cyclekit
It will ask you three questions. Answer n to each, as this user will only need to read/write to an SQL database, so it doesn't need greater privileges.
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) n
Shall the new role be allowed to create more new roles? (y/n) n
For convenience, let's create a postgres admin user in case we need it.
sudo -u postgres createuser -P martin
# This time answer 'y' to get the superuser privileges!
7b. Enable postgres users to connect using a password rather than just implicitly through a user account
By default, the Postgres configuration doesn't allow us to connect to Postgres using an explicitly-supplied username & password combination. Instead, it assumes that all connections are through a user account.
These notes are based on: https://help.ubuntu.com/10.04/serverguide/C/postgresql.html
- Open the Postgres configuration file in a text editor
sudo pico -w /etc/postgresql/9.0/main/pg_hba.conf
- Scroll down to find the line containing
local all all identwhich basically means that Postgres just trusts whatever Unix account you're connecting as. - Before that line, add this new entry:
local all all md5
- Save the file and exit
- In the shell, run this to reload the configuration:
sudo -u postgres service postgresql restart
- You can now check this by connecting to psql (which is the equivalent of the MySQL shell 'mysql') with the user and password you created above.
psql -W -U martin
# This will probably respond with 'psql: FATAL: database "martin" does not exist', which is fine
# - it means you've got in OK, but there isn't a 'martin' database yet so it can't continue.
Here, for quick reference, are some common postgres administrative commands for managing users and databases.
Unlike MySQL, creating/removing users and databases can be done easily on the command-line rather than within Postgres itself. Also, Postgres refers to users as 'roles' - but effectively it means the same thing.
# Create a user (role) called someuser, prompting for a password
sudo -u postgres createuser -P jason
# Drop the user (role) jason
sudo -u postgres dropuser jason
# Create a database called foobar
sudo -u postgres createdb foobar
# Drop the database called foobar
sudo -u postgres dropdb foobar
# Restart the postgres service
sudo -u postgres service postgresql restart
# Import an SQL dump into the postgres database called cyclekit
sudo -u postgres psql -d cyclekit < /path/to/cyclescapeDB.sql
# Connect explicitly as the user (role) called jason, ensuring that a password is requested
# (Note that this will only work if you've added support for such direct connection in pg_hba.conf - see above.)
psql -W -U jason
Ruby is the core language that the toolkit software is written in. (Ruby is a programming language in the same way that PHP is.)
Ruby on Rails is a framework that runs on top of Ruby. (In the PHP world, the equivalent would be things like Zend Framework - basically a framework that makes development much quicker and more formalised.)
The Ruby and Ruby on Rails worlds are fast-moving, and therefore the packages that are supplied with Ubuntu or Mac OSX are generally out-of-date. Therefore, do not install Ruby and Ruby on Rails using the standard Linux/Mac package management systems, as this will cause you to run into problems.
Some of these notes are based on http://johan.driessen.se/posts/Getting-Ruby-on-Rails-1.9.3-to-work-on-Ubuntu-11.10
Ruby has its own package manager for managing different versions of Ruby, called RVM - Ruby Version Manager. This makes things much easier. See: https://rvm.beginrescueend.com/rvm/install/
- Get the RVM installer (we do this using Curl to grab the installer script directly, and then run that in bash)
bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)
- Open your ~/.bashrc file (which sets up your user account when you log in)
pico -w ~/.bashrc
- Add to end of ~/.bashrc :
[[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm"
- Change the line that reads
[ -z "$PS1" ] && return
to
if [[ -n "$PS1" ]] ; then
Add before the row you added to the end (on its own row):
fi
# which is then followed by the line you added above, i.e.
# [[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm"
- Save the file. (In Pico, this is Control-x then y.)
- Back in shell:
source "$HOME/.rvm/scripts/rvm"
Now that we have RVM (Ruby Version Manager) installed, we can add Ruby.
# Install Ruby 1.9.2 using RVM
rvm pkg install zlib
rvm pkg install openssl
rvm install 1.9.2 --with-openssl=$HOME/.rvm/usr
rvm --default use 1.9.2
So we should now have Ruby!
Gems (in Ruby) are packaged code libraries that we can use to make development work quicker. These provide functionality that is commonly-needed in web applications, and are well-tested. These libraries saving us writing such things from scratch.
To use these Gems, we have to install the Rubygems package. Once this is installed, we will then be able to do commands like gem install somegem.
cd ~
wget http://production.cf.rubygems.org/rubygems/rubygems-1.8.15.zip
unzip rubygems*
cd rubygems*
ruby setup.rb
cd ~
We just need to do a few additional tasks.
Ruby on Rails needs a JS runtime. We will use nodejs.
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get install nodejs
gem install annotate
Now we should have all the core server software we need. Well done!
Now we get the toolkit software from Github. In this example, we will add this to a folder rails/ in our home folder. If you're using the Windows Virtual Machine solution (see above), you might want to use /h/ (or whatever) instead.
# Make a space where we will work on the toolkit software
# ... or if using a Windows Virtual Machine and you want the files editable on the Windows side too, use cd /h/
cd ~
mkdir rails/
cd rails/
# Now get (clone) the existing toolkit repository, so we get the latest version of the software, and go into its folder
git clone [email protected]:cyclestreets/toolkit.git
cd toolkit
The toolkit application is written in Ruby on Rails, often just called 'Rails'.
The application will obviously need to connect to a database to read/write data. In our case, we're using Postgres (as installed above). In Rails, there is a config file at config/database.yml where we define what the database connection details are.
Firstly, we must have created a user using the Postgres 'createuser' command.
# Above we did this, which will have prompted us for a password:
# sudo -u postgres createuser -P cyclekit
We'll now add that user to the database configuration so that the application can connect using it.
This configuration file has three sections, development, test, and production. These are the three contexts that Rails runs in. We need to add the database configuration to all three of them.
- Copy the example database configuration template
cp -pr config/database.example.yml config/database.yml
- Open the config file in a text editor. As above, we're using Pico.
pico -w config/database.yml
- We're now going add some details to the block headed 'development:'. You need to use the username and password you created.
- Add these details. You'll notice in the file that each line has two spaces at the start. Make sure you retain the (two) spaces that indent them as this is part of the YAML (Yet Another Markup Language) format that this file is written in.
username: cyclekit
password: <password that you used above, without these brackets around>
- Make sure there is this script_dir (which was created in the PostGIS installation stage above)
script_dir: /usr/share/postgresql/9.0/contrib/postgis-1.5
- We'll also add these details which enable the database to be created, and the PostGIS scripts to be run, when the application is being setup. We could actually remove these afterwards if you want.
su_username: postgres
su_password: <password as above>
- Now edit/add the same details to the 'test:' and 'production:' blocks, leaving other lines that were already there:
username: cyclekit
password: <password as above>
script_dir: /usr/share/postgresql/9.0/contrib/postgis-1.5
su_username: postgres
su_password: <password as above>
When you write an application in Ruby on Rails, you define as part of this what set of Gems you need. This is listed in a configuration file. Then, when we set up the application, we use the 'bundle' command which will grab all the required Gems automatically.
# NB Do this from when in the top-level of the 'toolkit' directory
bundle install
Now we can get the database created with the schema and basic data.
rake db:create
rake db:schema:load
rake db:seed
The application should now be installed and ready to run!
Rails has a basic built-in webserver. It's basically just for simple testing and isn't suitable for production use.
It might take around 30 seconds to start running, while it loads all the gems, the Rails framework, and everything else.
# Start the webserver, still from within the toolkit/ directory
rails server
# NB You can use Control-c to stop it
The toolkit application should now be running at:
Go and make some nice tea and have a break!
In theory you can now start developing! However, it's best if we just go a little further:
However, it's best if we just go a little further: and run it through Apache, as that will be quicker and is a more realistic kind of installation.
Firstly make sure that you've stopped the basic port-3000 server as above, by using Control-c.
To run Rails applications in an Apache context, we use 'passenger', sometimes called 'mod-rails'. This works in a very similar manner to mod_php - effectively the Rails stack is loaded in memory as part of Apache, so it is much more nicely integrated and faster. For more details, see: http://www.modrails.com/install.html
sudo apt-get install libcurl4-openssl-dev
gem install passenger
passenger-install-apache2-module
- Edit the file for this site, which defines the configuration of a VirtualHost that will run this site. (Using a VirtualHost means you can add other sites on the same server installation in future.)
sudo pico -w /etc/apache2/sites-available/cyclescape
- Add to that file, making sure that the DocumentRoot matches the public directory in where your toolkit installation is.
# Enable passenger
LoadModule passenger_module /home/martin/.rvm/gems/ruby-1.9.2-p290/gems/passenger-3.0.11/ext/apache2/mod_passenger.so
PassengerRoot /home/martin/.rvm/gems/ruby-1.9.2-p290/gems/passenger-3.0.11
PassengerRuby /home/martin/.rvm/wrappers/ruby-1.9.2-p290/ruby
# Add a VirtualHost to run the application whose files are at /h/rails/toolkit, which will run at http://www.cyclescape.local/
<VirtualHost *:80>
ServerName www.cyclescape.local
ServerAlias *.cyclescape.local
DocumentRoot /h/rails/toolkit/public
RailsEnv development
<Directory /h/rails/toolkit/public>
AllowOverride all
Options -MultiViews
</Directory>
</VirtualHost>
- Save the file (with Pico this is Control-x as usual).
- Enable the VirtualHost we've just created and restart Apache to catch this updated configuration:
sudo ln -s /etc/apache2/sites-available/cyclescape /etc/apache2/sites-enabled/cyclescape
sudo /etc/init.d/apache2 restart
In the configuration above, we said that the site will run from http://www.cyclescape.local/ . This is basically a local address that we have to create.
It won't work in your browser until we add this address to the 'hosts' file on your computer. The hosts file defines what addresses match 'localhost' (and potentially other addresses).
- Open the hosts file for editing
pico -w /etc/hosts
- Add these lines, which should be enough to get you going:
127.0.0.1 www.cyclescape.local
127.0.0.1 camcycle.cyclescape.local
127.0.0.1 placeford.cyclescape.local
- Save the file
You should now find that http://www.cyclescape.local/ and the above variants should work.
If you're used to using PhpMyAdmin if developing with PHP/MySQL, you'll be reassured to find that there is a similar tool for Postgres, called PhpPgAdmin. We install this below. This is based on the notes at http://www.molecularsciences.org/postgresql/installing_phppgadmin_on_ubuntu .
sudo apt-get install phppgadmin
sudo /etc/init.d/apache2 restart
This will now appear at http://localhost/phppgadmin .
As long as you edited the pg_hba.conf file (see above) to allow connections to Postgres using a username and password, this should let you log in.
To test what outgoing emails look like:
- Use the outbound_email command in the Procfile
- Follow the development log to get the emails that are sent:
tail -f log/development.log
## install homebrew if not already installed.
brew install postgres
# and follow the instructions in the caveats as not auto started, next steps may be different if upgrade
initdb /usr/local/var/postgres -E utf8
cp /usr/local/Cellar/postgresql/9.2.1/homebrew.mxcl.postgresql.plist ~/Library/LaunchAgents/
launchctl load -w ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
brew install postgis # Must be after otherwise it might link to a previous install of postgres.
brew install imagemagick
# Add /usr/local/bin to the ~/.profile
brew install redis
# and follow the instructions in the caveats as not auto started and may be different if it's an upgrade
cp /usr/local/Cellar/redis/2.4.17/homebrew.mxcl.redis.plist ~/Library/LaunchAgents/
launchctl load -w ~/Library/LaunchAgents/homebrew.mxcl.redis.plist
brew install ruby # to get ruby 1.9.3 as ML only supplies 1.8.7.
git clone [email protected]:cyclestreets/toolkit.git
cd toolkit
git checkout staging
gem install bundler
bundle install
cp config/database.example.yml config/database.yml
vim config/database.yml # update to the required settings
rake db:create
rake db:schema:load
rake db:seed
rails server
# and now you can access the site at http://localhost:3000/
I've added ipswich.localhost, ipswich.ipswich.localhost, www.ipswich.localhost to my /etc/hosts file so that I could navigate to the group that I had created locally.