A little Whois reader I wrote while messing around with sockets in Python.  It doesn’t do any processing on the reply, it just returns the whole thing as a string.  It’s also missing any mechanism to determine which Whois servers to use for which TLDs.  But anyway, it’s a good start!

#!/usr/bin/env python
import socket
import select
 
PORT = 43
BUFSIZE = 1024
LINEEND = '\r\n'
 
def whois(domain, server, port=PORT):
    '''
    Perform a WHOIS search for domain on server/port.
    '''
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((server, port))
 
    msg = ''
    lookup = domain + LINEEND
    readable, writable, error = select.select([],[s],[],60)
 
    if s in writable: s.send(lookup)
 
    readable, writable, error = select.select([s],[],[],60)
    while s in readable:
 
        data = s.recv(1024)
        msg = msg + data
        if not data:
            s.shutdown(socket.SHUT_RDWR)
            s.close()
            break
        readable, writable, error = select.select([s],[],[],60)
 
    else:
        s.shutdown(socket.SHUT_RDRW)
        s.close()
 
    return msg.strip()
 
if __name__ == '__main__':
 
    print whois('google.com', 'whois.markmonitor.com')

I wrote a little cross-platform Python script to run a monitor through a sequence of colors to assist in detecting stuck pixels. The project is hosted here.

I’ve written a very simple python module/script to help find postal code, city, province, latitude/longitude matching the supplied postal code, city, province, or coordinate.

Check out the Google Code Project page for more information.

You can checkout the source code by issuing the following command:

svn checkout http://python-geolocate.googlecode.com/svn/trunk/ python-geolocate-read-only

I’m posting this for anyone who’s had a hard time getting Mapnik installed
along with some handy utilities on Ubuntu 9.04. Here we go.

Mapnik from Source

Update the package listing and then upgrade all packages:

apt-get update
apt-get upgrade

After that install the dependencies:

sudo apt-get install g++ cpp \
libboost1.35-dev libboost-filesystem1.35-dev \
libboost-iostreams1.35-dev libboost-program-options1.35-dev \
libboost-python1.35-dev libboost-regex1.35-dev \
libboost-thread1.35-dev \
libxml2 libxml2-dev \
libfreetype6 libfreetype6-dev \
libjpeg62 libjpeg62-dev \
libltdl7 libltdl7-dev \
libpng12-0 libpng12-dev \
libgeotiff-dev libtiff4 libtiff4-dev \
libcairo2 libcairo2-dev python-cairo python-cairo-dev \
libcairomm-1.0-1 libcairomm-1.0-dev \
ttf-dejavu ttf-dejavu-core ttf-dejavu-extra \
libgdal1-dev python-gdal \
postgresql-8.3-postgis postgresql-8.3 \
postgresql-server-dev-8.3 postgresql-contrib-8.3 \
libsqlite3-dev  \
subversion build-essential

Now we need to get the Mapnik source from the repository:

mkdir -v ~/src
cd ~/src
svn co http://svn.mapnik.org/trunk mapnik
cd mapnik

After that we can configure and build Mapnik:

python scons/scons.py configure INPUT_PLUGINS=all \
OPTIMIZATION=3 \
SYSTEM_FONTS=/usr/share/fonts/truetype/ttf-dejavu/
python scons/scons.py
sudo python scons/scons.py install
sudo ldconfig

Now we’ll do a quick test to see if Mapnik installed correctly:

python # will start the python interpreter
>>> import mapnik
>>> exit()

If the import mapnik line didn’t spit out any errors, everything is probably ok.

Setting Up the Database

First we need to tweak some configuration files. Open
/etc/postgresql/8.3/main/postgresql.conf with your favourite
text edit and modify the following lines:

shared_buffers = 128MB
checkpoint_segments = 20
maintenance_work_mem = 256MB
autovacuum = off

Next edit /etc/sysctl.conf and add the following at the end:

kernel.shmmax=268435456

Now to avoid a reboot, we’ll update the kernel like this:

sudo sh -c 'echo 268435456 >/proc/sys/kernel/shmmax'

Then restart the PostgreSQL server:

sudo /etc/init.d/postgresql-8.3 restart

The next thing we need to do is create the database. Substitute
“your_user” below with your user name. Note: If your user name is ‘user’ you might get an error from the commands below, I did when I tried.

sudo -u postgres -i
createuser -s your_user
createdb -E UTF8 -O your_user gis
createlang plpgsql gis
exit

Now setup PostGIS on our database:

psql -d gis -f /usr/share/postgresql-8.3-postgis/lwpostgis.sql

And then change some permissions on the database:

echo "ALTER TABLE geometry_columns OWNER TO your_user; ALTER TABLE spatial_ref_sys OWNER TO your_user;" | psql -d gis

Now we’ll enable the intarray module:

psql gis < /usr/share/postgresql/8.3/contrib/_int.sql

Installing Some Extra Tools

The tools we’ll be installing here are useful for working with OpenStreetMap
data and also to facilitate generating maps with Mapnik.

OSM2PGSQL

This utility will import OpenStreetMap data files into the PostGIS database. First we’ll install some dependencies:

sudo apt-get install build-essential libxml2-dev libgeos-dev libpq-dev libbz2-dev proj

And then checkout the latest version from the repository:

cd ~/src
svn co http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/
cd osm2pgsql
make

Mapnik-Tools

We’re going to put this right in our home directory:

cd ~
svn co http://svn.openstreetmap.org/applications/rendering/mapnik/

Setting up the World

Now we’re going to grab some boundaries data for the world.

cd ~/mapnik
wget http://tile.openstreetmap.org/world_boundaries-spherical.tgz
wget http://tile.openstreetmap.org/processed_p.tar.bz2
wget http://tile.openstreetmap.org/shoreline_300.tar.bz2

We’ll extract them each into ~/mapnik/world_boundaries:

tar xfv world_boundaries-spherical.tgz
rm -v world_boundaries-spherical.tgz
mv -v processed_p.tar.bz2 shoreline_300.tar.bz2 world_boundaries/
cd world_boundaries
tar xfv processed_p.tar.bz2
tar xfv shoreline_300.tar.bz2
rm -v processed_p.tar.bz2 shoreline_300.tar.bz2

Importing Some Data

The last thing we’re going to do is put some OpenStreetMap data into our database. You can either import the whole planet:

cd ~
wget http://ftp.heanet.ie/mirrors/openstreetmap.org/planet-latest.osm.bz2

But it’s a HUGE file, so if you just want to have a smaller part of the world,
you can use extracts. To create your own extracts, read this tutorial. Otherwise
grab an extract for your area of interest from somewhere (ex.CloudMade). You’ll want to download the file ending in .osm.bz2. In my example I’ll be using british_columbia.osm.bz2.

cd ~
wget http://downloads.cloudmade.com/north_america/canada/british_columbia/british_columbia.osm.bz2

We just need to import the data into the database now, for this we’ll use osm2pgsql which we installed earlier.

cd ~/src/osm2pgsql
./osm2pgsql --slim -d gis ~/british_columbia.osm.bz2

And there you have it. If you want to test this out from the command line, you
can use the last portion of this tutorial, or to use python, try the examples here.

Resources

Most of this tutorial was ripped directly off from the following sites:

• http://weait.com/content/build-your-own-openstreetmap-server
• http://www.weait.com/content/make-your-first-map
• http://trac.mapnik.org/wiki/UbuntuInstallation
• http://wiki.openstreetmap.org/wiki/Mapnik#World_Boundaries

As part of one of my current projects, maptop, I needed a simple module to detect whether or not a GPS receiver is connected to the computer, and if so, which port is it connected to.

For the task, I used PySerial, along with some of their example code, to test if each serial port can be opened, and if so, if there are NMEA GPS sentences coming in on it.  It uses a fixed baud rate of 4800, which is what my receiver is using and is also specified here.  The timeout of 2 seconds seems to work (at least with my device).  Both the baud rate and the timeout can be changed by modifying the BAUD_RATE and TIMEOUT “constants” at the top of the script.

I’m not sure if this works on Windows or not, I haven’t tested it yet, but it *should work*.

Below is the code for the module:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88

#!/usr/bin/env python
# 
# Copyright 2009 Matthew Brush < mbrush AT leftclick DOT ca >
# 
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
# 
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
# 
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
# MA 02110-1301, USA.
 
__version__ = "0.1"
__all__ = [ 'get_gps_port', 'test_port' ]
 
import serial
import glob
import re
import platform
 
BAUD_RATE = 4800    # the standard nmea baud rate
TIMEOUT = 2         # time to wait for nmea sentences
 
class NoGpsReceiverError(Exception):
    def __init__(self, value):
        self.value = value
    def __str__(self):
        return repr(self.value)
 
def scan():
    """ Scan for available ports. return a list of device names. """
    if platform.system == 'Windows':
        import scanwin32
        ports = []
        for order, port, desc, hwid in sorted(scanwin32.comports()):
            ports.append(port)
        return ports
    else:
        import scanlinux
        return scanlinux.scan()
 
 
def test_port(port):
    """ Detects whether NMEA GPS sentences can be read from the port. """
    nmea_gps_pattern = '^\$GP[A-Z]{3},'
    gps_detected = False
    try:
        s = serial.Serial(port, BAUD_RATE, timeout=TIMEOUT)
        for i in range(1,10):   # try 10 lines in a row
            line = s.readline().strip()
            m = re.match(nmea_gps_pattern, line)
            if m:
                gps_detected = True
                break
        s.close()
        return gps_detected
    except serial.SerialException:
        return False
 
def get_gps_port():
    """ Looks for GPS data coming in on any serial port and returns the port name. """
    for port in scan():
        if test_port(port):
            return port
    raise NoGpsReceiverError('Unable to locate a suitable GPS receiver.')
 
def main():
    print "Testing ports to find GPS receiver..."
    try:
        if platform.system == 'Windows':
            print "  Warning: gpsdetect not tested in Windows yet."
        for port in scan():
            if test_port(port):
                print "Detected GPS receiver on '" + port + "'."
                found_gps = True
                break
    except NoGpsReceiverError:
        print "No GPS receivers were detected on the system."
 
 
if __name__ == '__main__': main()

The complete source can be downloaded here.