You are here: Home Blog Authors michaelc

Mike Cullerton

Aug 14, 2013

Public Data Hacking with Python in Larimer County

by Mike Cullerton — last modified Aug 14, 2013 12:25 AM

Inspired by Dave Beazley's Public Data Hacking tutorial, I decided to investigate my local public data. It turns out, Larimer County provides API access to property and property tax records. The access is limited, but does provide some beginner experience with public data.

At our recent Python Web Dev meetup [1], we worked on Dave Beazley's Learn Python Through Public Data Hacking tutorial [2]. In the tutorial, Dave introduces python with a series of coding challenges involving public data resources like transit information and health inspections.

Many governments and municipalities now provide access to public records over the Internet, both as part of open information acts and to lower their costs of providing that information. Some even provide direct access to their data through APIs.

Working on the problems in the tutorial got me wondering about the public data available here in Larimer County, CO.

Larimer County provides access to a wide variety of data through the web [3], including property records, voter registration, and restaurant health inspections.

The county also allows direct access to property and property tax information through APIs. The APIs are implemented as SOAP web services.

The county has sample code for languages they call ColdFusion, C#, and PHP.

Hello, the 1990's called. They want their manuals back. [4].

I offer these Python examples.

Code

The API's are somewhat limited. To use them you need to know the schedule number or parcel number for a property. They don't allow you to easily grab a chunk of data to play around with while learning.

# import suds client for SOAP layer
from suds.client import Client

# get a tax result
property_tax_url = 'http://www.larimer.org/webservices/PropertyTaxes.cfc?wsdl'
property_tax_client = Client(property_tax_url)
property_tax_result = property_tax_client.service.GetPropertyTax(scheduleNum='', parcelNum=1234567890, taxYear=2012)
print(property_tax_result)

# get an info result
property_info_url = 'http://www.larimer.org/webservices/PropertyInformation.cfc?wsdl'
property_info_client = Client(property_info_url)
property_info_result = property_info_client.service.GetPropertyInfo(scheduleNum='', parcelNum='1234567890')
print(property_info_result)

The property information client has additional web services available. The county has documentation, code examples, etc. [5]

Comments

It's cool the county provides access to this information online. But, it is very limited in how you can access it.

Hopefully, in the future we'll see more APIs that allow us greater flexibility in our searches.

It was a good learning experience.

Links

[1] Python Web Dev Meetup

[2] Learn Python Through Public Data Hacking

[3] Larimer County Public Records Databases

[4] I'm joking.

[5] Larimer County Public Records Databases API


Jul 25, 2013

First impressions with Kotti

by Mike Cullerton — last modified Jul 25, 2013 03:15 PM
Filed Under:

The boss is away this week, so I'm learning about Kotti. Here's how I solved the problem of limiting a content type to the root of the site.

The Basics

My application is very simple. There are two content types; Forum and Idea. A forum is added to the root of the site, and then ideas are added to the forum.

I was already familiar with kotti's documentation and the tutorial there, so getting the basic application running was very easy.

First, I set up a virtualenv, and installed kotti along with a skeleton for my package.

$ virtualenv --no-site-packages kotti_agora-27
$ cd kotti_agora-27/
$ ./bin/pip install -r https://raw.github.com/Kotti/Kotti/master/requirements.txt
$ wget https://raw.github.com/Kotti/Kotti/master/app.ini
$ ./bin/pip install kotti_paster
$ ./bin/paster create -t kotti_addon kotti_agora
$ cd kotti_agora/
$ ../bin/python setup.py develop

The pip install step takes a little while.

From there, I was able to add code to kotti_agora, including my two content types, their views, and some kotti plumbing to wire it all up. For the most part, I stole everything from the kotti tutorial mentioned above and from the source code for kotti_blog.

Limiting Ideas to a Forum

The first problem I had to solve was limiting ideas to a forum. Kotti has an addable_to attribute when creating content types. I set the addable_to attribute of Idea to Forum.

class Idea(Content):
    id = Column(Integer(), ForeignKey('contents.id'), primary_key=True)
    body = Column(Text())
    date = Column('date', UTCDateTime())

    type_info = Content.type_info.copy(
        name=u'Idea',
        title=u'Idea',
        add_view=u'add_idea',
        addable_to=[u'Forum'],
        )

Limiting Forum to the Site Root

At this point, I had an application that worked. I could add a forum to the root of my site, and then add ideas to it.

But, you could add a Forum to any document in the site, and you could add more than one Forum.

I asked a couple questions about this on the #kotti irc channel and was pointed to an example in kotti_media. The solution to both problems is a custom TypeInfo. This one checks whether we have the site root for our context, and if a Forum already exists.

class ForumTypeInfo(TypeInfo):

    def addable(self, context, request):
        """only add once, and only at the root"""
        addable = context == get_root()
        child_type_already_added = self in [
                c.type_info for c in context.children]
        return addable and not child_type_already_added

I then use ForumTypeInfo for Forum's type_info. Note how this is different from the Idea class above.

class Forum(Content):
    id = Column(Integer(), ForeignKey('contents.id'), primary_key=True)

    type_info = ForumTypeInfo(
        name=u'Forum',
        title=u'Forum',
        add_view=u'add_forum',
        addable_to=[u'Document'],
        )

Add Forum Automatically

For a little icing on the cake, I decided to add the Forum to the site automatically, using a populator. I put mine in __init__.py. If forum does not exist, the populator adds it.

def populate():
    root = get_root()
    if 'forum' not in root.keys():
        root['forum'] = Forum(title=u'Forum')

Then add configuration to your .ini file.

kotti.populators = kotti_agora.populate

Pretty slick.

Now, there is one Forum and it only exists at the root of the site. And, if it doesn't exist, it is created when the site starts.

Impressions

There are so many options for python web programmers these days. My expertise is with Plone, but after using Pyramid for a few small projects I wanted to try out Kotti CMS. Something about Pyramid seems right to me, and Kotti fits the same mold. Kind of like working on a bicycle or a VW bus. I can see it all. It fits in my head nicely, and the problems are approachable. 

Kotti has great documentation, which helps you get started. The project is relatively young and there are some holes in the narrative documentation, but it has evolved in the short time I've used it.

And, a big shout out to the #kotti irc channel. Very friendly. Very helpful.

 

Jul 09, 2013

Setuptools and a failing bootstrap

by Mike Cullerton — last modified Jul 09, 2013 12:15 PM
Filed Under:

After distribute and setuptools merged back together, I had a problem running bootstrap. Upgrading setuptools fixed the problem for me.

I used zopeskel 2.21.2 to install a development version of Plone, but got an error when I tried to bootstrap it.

michaelc@Cullerton$ ../bin/python bootstrap.py
Traceback (most recent call last):
  File "bootstrap.py", line 117, in <module>
    ws.require('zc.buildout' + VERSION)
  File "/Users/michaelc/plone_dev/decsys-26/lib/python2.6/site-packages/distribute-0.6.27-py2.6.egg/pkg_resources.py", line 690, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/Users/michaelc/plone_dev/decsys-26/lib/python2.6/site-packages/distribute-0.6.27-py2.6.egg/pkg_resources.py", line 588, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: setuptools>=0.7

 

Upgrading setuptools fixed the problem for me.

easy_install --upgrade setuptools

 

Hope this helps someone.


Reinout has a related post. http://reinout.vanrees.org/weblog/2013/07/08/new-setuptools-buildout.html

Apr 09, 2013

How to enable User Folders in Plone

by Mike Cullerton — last modified Apr 09, 2013 03:22 PM

User Folders are an option in Plone. You can turn them on In the Security settings area of Site Setup (Plone Control Panel). Here's how I enabled User Folders in Plone from within my product.

I needed to enable the User Folders feature of Plone from within my product. My first thought was to use a Generic Setup profile, but the solution was setuphandlers.py.

I inspected the Enable User Folders button on the Security settings page of Site Setup, and saw the attribute was named enable_user_folders. I tried to find a generic setup profile I could edit.

The closest I got was adding enable_user_folders to properties.xml. This turned the attribute on in the ZMI, but not in Site Setup and User Folders were not enabled.

Then I found a reference to

security.enable_user_folders

in a setuphandlers.py file from the Rhaptos git repo.

So, I added these lines to setupVarious in my setuphandlers.py.
from plone.app.controlpanel.security import ISecuritySchema
site = context.getSite()
security = ISecuritySchema(site)
security.enable_user_folders = True

Now, when I install my product, User Folders are enabled automatically.

Note that you need to make sure setupVarious is turned on in configure.zcml.

<genericsetup:importStep
    name="your.package"
    title="your.package special import handlers"
    description=""
    handler="your.package.setuphandlers.setupVarious"
    />

 Hope this helps someone.

 

Apr 08, 2013

How to open SSH links in iTerm 2

by Mike Cullerton — last modified Apr 08, 2013 04:45 PM
Filed Under:

When I switched to iTerm 2, it wouldn't open ssh links for me. Instead, it opened new tabs. I finally solved the problem with a simple configuration change.

I know some of you have this problem. I've seen a few references to it on the net, but no solutions. iTerm 2 won't open ssh links.

I have a pyramid application that uses boto to query ec2 instances and then posts information about the instances--including an ssh link, to a web page on my laptop.

The ssh links open fine in iTerm. I couldn't get them to open in iTerm 2 though. Instead, they just opened new tabs.

I really like using iTerm 2, so to 'solve' the problem, I added the actual ssh shell command for each of the instances as text on the page.

To open an ssh connection to an instance, I simply copy the ssh command, click the link, paste the ssh command into the new terminal tab, and hit return.

Hey, it works.

Every once in a while, when it really bothers me, I try to figure out a real solution.

Today was one of those days.

The solution turned out to be pretty simple.

In the iTerm 2 Preferences, under the Profiles tab, General section, change the Command to Login shell.

Now, when I click on one of the links, it opens correctly in iTerm 2.

I hope it works for you too.

*Note that you must first have iTerm 2 set up to accept ssh connections. Preferences->Profiles->General->URL Schemes

 

 
 
Copyright © 2003-2017 Core Software Group | 303/809-1001 | Fort Collins, Colorado | All rights reserved.