PostgreSQL Query Slow as a Function

Posted 3 months ago on February 18, 2012

There's nothing more frustrating than a slow running query, and that is exactly what I had while working with a PostgreSQL database recently. Actually it wasn't the query that was slow, but my implementation of PostgreSQL's functions. I'm documenting a solution here in hope that someone else will find this when having similar problems.

The query in question was extremely simplistic, similar to this:

SELECT t.*
FROM some_table t
WHERE name = 'xyz';

When I ran that guy from the command line I was getting response times right around 10 milliseconds on a table with 500k records, and an index on the name column. Pretty much expected.

Then I tried wrapping up that query into a function:

CREATE OR REPLACE FUNCTION test_function(name_param character(9))
  RETURNS SETOF RECORD AS
$BODY$

BEGIN
	RETURN QUERY
		SELECT t.*
		FROM some_table t
		WHERE name = 'xyz';
END;

$BODY$
  LANGUAGE plpgsql STABLE;

When I ran that function, the response time was around 500 milliseconds. Turned out that the problem was that the name column in some_table was defined as character varying(9), and the name_param input parameter was defined as character(9). This type mismatch completely threw off the query planner and caused my index to be ignored.

Using StructureMap Inject To Avoid External Dependencies In Integration Tests

Posted 10 months ago on July 16, 2011

At the end of last week I wrote a class to manage sending e-mail from our application. This class (called it EmailProcessingManager) had all the usual suspects:

  • Get a list of to, cc, bcc, from and reply to email addresses
  • Find a template
  • Replace tokens in said template with values from the database
  • Send the email(s)

Wound up making a class for each one of these separate duties and wrote tests for each. But to seal it all together, I wanted to write an integration test that would test the whole operation of sending an email. This test would hit the database, pull in the actual HTML template, replace token values, etc. But what about the actual sending of email? I didn't want to go to the trouble of trying to figure out how to hit a pop email inbox and reading in the email...there had to be an easier way!

ObjectFactory.Inject() to the rescue!

The problem I had was that I wanted to use the entire EmailProcessingManager class as it would be used in production, except I didn't want the mail to actually go out. I wanted all the classes' dependencies to be real implementations except for the class that actually spun up the SMTP client class to send the mail. That's when I found the Inject method.

(Using Moq in this example)

<TestFixtureSetUp()>
Public Sub Setup()
    _mailSenderMock = New Mock(Of IEmailSender)
    _mailSenderMock.Setup(Function(mock) mock.SendEmail(It.IsAny(Of List(Of MailMessage)))) _
        .Callback(Sub(messages) _emailsThatWouldHaveBeenSent = messages)
        
    'Inject the mocked object instead of using the production code
    StructureMap.ObjectFactory.Inject(Of IEmailSender)(_mailSenderMock.Object)

    'Get an instance of the class under test
    _emailProcessingManager = StructureMap.ObjectFactory.GetInstance(Of IEmailProcessingManager)()        
    
    Dim recipientList = New List(Of String) From {"paco@joescrabshack.com"}
    _emailProcessingManager.ProcessEmailRequest(EmailType.Registration, recipientList)   
End Sub

<Test()>
Public Sub Can_create_a_request_for_an_authorization_email_and_have_it_sent_to_the_recipient()
    _emailsThatWouldHaveBeenSent.First(Function(email) email.To.First().Address = "paco@joescrabshack.com").ShouldNotBeNull()
End Sub  

All of the value of an integration test, but none of the pain of dealing with an external dependency. I like it.

Now that I've discovered this, I have other ideas for it's use. One situation that comes to mind is an integration test that saves a complex object graph to the database. Those tests are always hard to clean up after, so instead of saving the object I'll just use a mock to retrieve the passed object and do my asserts. No messy cleanup!

Experience with Heroku and AppHarbor

Posted 11 months ago on June 12, 2011

In the past 2 months I've had the opportunity to use two popular web hosting platforms: Heroku and AppHarbor. I used Heroku to host the blog I wrote using Ruby, Sinatra, and Padrino. AppHarbor was used to host a ASP.Net MVC3 application with SQL Server. I'm writing my experience here.

Heroku

The experience in deploying to both Heroku and Appharbor was great overall. Without Heroku I could have potentially faced hours of reading through help docs and googling error messages, and it's likely I would've given up. Heroku gave me the ability to get my app deployed, and for a novice *nix user that was a huge value. However, it didn't take me long to remember a familiar quote "Automatic just means you can't fix it when it breaks". So far it hasn't broke, so it hasn't been a problem.

Database development and management is a little awkward. I used Sqlite for development, and used the tools in ActiveRecord to connect to a Postgresql instance on Heroku's servers. In my trivial blog application this worked without incident. However, I have never used Postgresql and don't have any motivation to. If I would have needed to leverage some more advanced RDBMS capabilities, I would have preferred to use something I'm more familiar with: SQL Server or MySql.

As a Mercurial user, the deployment experience was a little more arduous than if I had been using Git. The workaround is to use an extension called Hg-Git which acts as a bridge. Hg-Git took longer to configure and setup than I was willing to spend on it(2 minutes?) so I abandoned it. Instead I just created a git repository in my application and committed to it only when I needed to deploy. My .gitignore filtered all the Hg related files, and my .hgignore filtered Git related files. Messy.

The interface for deploying to Heroku is done through a ruby gem. Setting up an account and new application has very few interaction points. I would have preferred to interact with a GUI than reading through help docs to find the right command.

This blog is hosted for free on Heroku.

AppHarbor

Most of my career has been spent building and deploying Microsoft ASP.Net based solutions to IIS. I've fought those dragons enough times that I know what to expect and how to get around the common error messages. Even so, I found AppHarbor to be a breeze compared to a manual deployment.

I use Mercurial almost exclusively for source control along with Bitbucket for free private repository hosting. I was extremely impressed with the integration between Bitbucket and AppHarbor. All it takes is "hg push" for my app to be copied to my shared Bitbucket repository which triggers a copy to AppHarbor which triggers a build and deploy. If you're a Git user, it is also natively supported.

AppHarbor supports MySql and SQL Server. This made database development simple since I can use SQL Server Management Studio to connect and manage my database hosted at AppHarbor. This also means I can use all the tools I know and love for copying schema and/or data.

Setting up a new app is all very simple and straightforward and done through the website.

My small app is hosted for free on AppHarbor.

Summary

I think it's awesome that we have so many great options for development, deployment, and hosting. I've never wanted to spend time figuring out deployment, hosting, or source control issues. These tools get me very close to not having to think about them at all.

Heroku caused a lot of fuss when it was made available for the ruby web world, but AppHarbor is a solid answer that stands up to and even exceeds Heroku's usability. I'm sure each will continue to grow and get even better.

Close Named Branch In Mercurial

Posted 12 months ago on June 01, 2011

I've found that closing named branches in Mercurial has been a good practice. Not only is an explicit statement that the feature is wrapped up, it also removes the head (a changeset with no children) from the repository.

You also get some nice visual cues from the TortoiseHG Workbench UI:

Steps to close the branch and merge it back to default:

$ hg up feature-branch-name
$ hg ci -m 'Closed branch feature-branch-name' --close-branch
$ hg up default
$ hg merge feature-branch-name
$ hg ci -m merge

Creating a Stub That Redirects Input Parameter to Method Return Value

Posted 12 months ago on June 01, 2011

In the case where you want to create a stub that simply passes the input parameter through to it's return value, you can use this syntax in Moq:

mock.Setup(m => m.Method(DateTime.Now)).Returns((DateTime param) => param);

Documenting this here because I use it just often enough to forget.