Agile Ajax » More Named Scope Awesomeness

More Named Scope Awesomeness

Noel Rappin, Friday, June 27, 2008 @ 2:19 pm

I'm back writing about named scopes and still including silly scope visual pun pictures.

Named Scopes and Find Methods

First off, I wanted to mention something that I haven't seen talked about much in other discussions of named scopes -- integrating your names scopes with traditional ActiveRecord finder methods.

This is possible, but only in a limited kind of way. If you have a chain of named scopes, you can add a regular finder method at the end of the chain, and only at the end of the chain.

In other words, you can augment the contains scope introduced in last week's post with a regular find method like this:

>> Person.contains(:first_name, "el").find_all_by_last_name("Smith")
   Person Load (0.000763)   SELECT * FROM `people` WHERE (`people`.`last_name` = 'Smith') AND (lower(first_name) LIKE '%el%')

(SQL statement in the console via a little .irbrc hack that maps the Rails logger to standard out.) Notice that the LIKE condition from the contains clause and the last name condition from find_by_last_name have been merged into the SQL statement.

This works because named scopes are just wrappers around Active Record's with_scope method -- essentially the named scopes create a series of nested scopes that last until the end of the method call chain -- the find statement at the end is effectively executed inside all the nested scopes.

The find method needs to be last in the chain:

>> Person.find_all_by_last_name("Smith").contains(:first_name, "el")
   Person Load (0.072997)   SELECT * FROM `people` WHERE (`people`.`last_name` = 'Smith')
NoMethodError: undefined method `contains' for #<Array:0xe7d1e74>     from (irb):10     from :0

In this case, the find_all_by_last_name is called normally, but the resulting array does not, of course, have a contains method.

On the plus side, the last slot isn't limited to find methods created by Rails -- any method you put last in the chain is evaluated as though inside all of the named scopes. For all you will_paginate fans, the paginate method will work here too -- but you do need to be careful that limit or offset options are not set on any of the earlier scopes.

Named Scopes and Advanced Search

Last time I had mentioned using advanced scopes to support search. Ryan Bates does a version of this in Railscast 112. Here's an even more general solution.

Let's say you have a dynamic search form that lets the users specify options in something sort of like an iTunes smart playlist structure:

The user can specify an arbitrary number of options, to keep this a little simple, we'll limit this to string fields.

The first step in using named scopes to support this kind of search is to create individual scopes for each possible operator, as I did for contains last time. A little bit of metaprogramming does this neatly. I put this code in config\initializers\global_named_scopes.rb so it is automatically loaded when Rails starts. The .code is monkey patched inside ActiveRecord::Base

class ActiveRecord::Base
  STRING_SCOPES = {:contains => ["LIKE", "%", "%"],
    :does_not_contain => ["NOT LIKE", "%", "%"],
    :starts_with => ["LIKE", "", "%"],
    :does_not_start_with => ["NOT LIKE", "", "%"],
    :is => ["=", "", ""],
    :is_not => ["<>", "", ""]}    

  STRING_SCOPES.each do |key, value|
    operator, prefix, suffix = value
    named_scope key, lambda { |column, text|
      {:conditions => ["lower(#{column}) #{operator} ?",
            "#{prefix}#{text.downcase}#{suffix}"]}     }
    end
  end

The constant hash keys are the scopes being created, and the values are the operator and also whether the operand needs a LIKE wild card at the beginning or end of the text.

The loop gives creates each named scope, creating the correct :conditions list for it, and along the way ensuring that everything is converted to lower case to support case insensitive search. Now any ActiveRecord can manage things like:

User.does_not_start_with(:first_name, "f")

Which actually has a fighting chance of being useful in general, I think.

The next step is to be able to dynamically generate one of these scopes from arguments

  def self.search_by_criteria(column, operator, value)
    scope = operator.gsub(" ", "_").to_sym
    send(scope, column, value)
  end

The first line allows the display to say "starts with" and the resulting scope to be starts_with, then the send just calls the appropriate named scope. This can be called individually.

Person.search_by_criteria(:first_name, "starts with", "fr")

But in order to get the search to work full, you need to combine the scopes. And in order to do that, you need to take advantage of a special scope called scoped. The scoped scope lets you create anonymous scopes on the fly, and is defined for all ActiveRecords as follows:

 named_scope :scoped, lambda { |scope| scope }

That's pretty minimalist -- all it does is take the arguments passed to it.

    def self.search_by(*all_criteria)
     scope = scoped({})
     all_criteria.each do |criterion|
        scope = scope.scoped(search_by_criteria(*criterion).proxy_options)
     end
     scope
   end

And then:

Person.search_by([:first_name, "starts with", "fr"], [:last_name, "contains", "sm"])

The code starts by creating a blank anonymous scope, then inside the loop, the scope is composed together by creating a scope for the individual criterion then taking those options and adding them to the scope being built up -- building an anonymous scope each time. The scope.scoped(inside_scope.proxy_options) is somewhat hacky, in that you're building a scope, then tearing it apart for the options. You need to do something like that because you can't compose search_by_criteria directly -- since it's not, in and of itself, a scope (or at least, I get SQL errors when I try). A less hacky version might have a separate method that just returned an option hash for each criterion, although I think it's nice to have everything as a separate scope.

The nice thing about this scope-based solution is that it's very easy to compose it with global search conditions, such as limiting to active objects, or limiting based on access. They can be composed explicity:

Person.search_by(...).active.for_current_user

Or alternately, conditions can be added to the initial scope in the search_by method. (The production version of this puts some :include options there because some of the search columns are from joined tables...)

Scope And Search Issues

There are a couple of things you should be aware of when using scopes. Merging scopes can be kind of slow, in a url_for kind of way. Especially worth noting is that even if ActiveRecord is caching the result of the SQL query, the scope merge still happens because that's what generates the SQL code in the first place.

This particular search implementation has a couple of limitations at the moment. Most notable is that since scope condition merge is always via AND, doing a search with OR logic is not possible yet. I'd like to have a clean option for this, but I'm not sure what the best API is. Right now, you can work around it a bit by grabbing the conditions for each clause and doing the OR connecting yourself, as in the following code that does a LIKE search over multiple columns:

 named_scope :contains_in, lambda { |text, *columns|
     clauses = columns.map { |col| "lower(#{col}) LIKE ?"}
     conditions = ["((#{clauses.join(') OR (')}))"]
     columns.each { conditions << "%#{text.downcase}%" }
     {:conditions => conditions}   }

I'm trying to come up with something cleaner, I'll report back.

Topics: Ruby on Rails

agile ajax

Comments: 12 so far

Cool. My has_browser plugin does a very similar thing to search_by_criteria, except that it allows you to choose which scopes should be exposed (a little bit like attr_accessible), for security reasons.

http://jamesgolick.com/2008/5/19/introducing-has_browser-parameterized-browse-interfaces-for-your-ar-models

Comment by James Golick, Friday, June 27, 2008 @ 3:10 pm
The power of named_scopes and of the scoped method is just incredible, and this technique in particular looks really interesting. I figured I should mention that we recently got squirrel working with scoped, which makes complex finders like this both easy to read and build.

http://giantrobots.thoughtbot.com/2008/6/25/named-scopes-with-squirrel

Thanks for the great post, and you’ve got a new subscriber.

Comment by Tammer Saleh, Friday, June 27, 2008 @ 4:00 pm
Useful stuff. It’s good to see that the chained scopes will get rolled up into a single SQL statement (avoiding the normal AR foo.bar.baz problem, where missing includes cause lots of trips to the DB).

The first example did trip a buzzer for me, tho. IIRC MySQL, LIKEs need to start with a non-wildcard (’foo%’) to take advantage of the indexes, which are usually btrees built from the left side of string. (http://dev.mysql.com/doc/refman/5.0/en/mysql-indexes.html) Not sure if FULLTEXT indexes help this, haven’t gotten to dig in.

So if this makes it easy to do searches with ‘%foo’ and ‘%foo%’, there’s a noticeable DB impact.

Comment by Rafi, Friday, June 27, 2008 @ 4:17 pm
Nice, I’m just learning named_scope and used it in a project this week. I figured there was a way to generate anonymous scopes on the fly to simplify all my crazy conditional logic. This example really helps alot. cheers mate.

Comment by David, Saturday, June 28, 2008 @ 8:26 am
I’m guessing you check that the operators are from a valid subset - so that people can’t ask for the delete_all operator?

Comment by Ben Nolan, Saturday, June 28, 2008 @ 7:48 pm
Love the articles. Inject is great:

all_criteria.inject(scoped({})) do |scope, criterion|
scope.scoped(search_by_criteria(*criterion).proxy_options)
end

Comment by Nick Kallen, Tuesday, July 1, 2008 @ 12:42 am
James and Ben — in the actual version of this, the user is limited to a specific subset (I think delete_all is excluded because it’ll cause an error when composed with other scopes), plus there are some additional scopes tacked on to all the searches to limit users to objects that they have access to see. Although I probably could do a tighter job of limiting the searches.

Tammer — thanks for the kind words, we use a lot of thoughtbot tools on a daily basis here..

Comment by Noel Rappin, Thursday, July 3, 2008 @ 5:26 pm
There seems to be a bug in the code, if I do a :
Model.some_scoped_proxy.search_by(@searches)
search_by will ignore the previous named_scopes.
but if I do
Model.search_by(@searches).some_scoped_proxy
that will generate the proper sql.

Also I had a seperate question, I tried making my own simple example based off your code. I want a order named_proxy on all AR models, but my code doesn’t work because when it calls column_names the lambda thinks its calling it on ActiveRecord::Base instead of the subclassed model. Why does this not work?

class ActiveRecord::Base
named_scope :order, lambda {|column|
if column_names.include? column
{:order => column}
end
}

end

Also one other thing I noticed about scopes is that they dont override previous scopes, so for example Car.order(’a').order(’b')
I would think that following the “ruby way” of least suprise, the order would on b, but its actually on a

Comment by jtoy, Friday, July 4, 2008 @ 6:59 am
Here’s some code that I came up with a week ago.

http://pastie.org/231954

Is basically takes an array of different scopes you wish to apply and injects them into the the model. I haven’t thought of AND/OR joins yet though.

Comment by Fredrik W, Wednesday, July 16, 2008 @ 3:43 am
[...] that triggers an overlay, you can sort the columns. The advanced search is the flexible one I mention here, so it’s got several Ajax calls associated with it as [...]

Pingback by Pathfinder Development » I’m Cranky Because I’m Not Getting Enough REST, Friday, August 22, 2008 @ 4:02 pm
[...] project that included integrated design drafts, named-scope based reporting, and aggressive markup in the helpers had it’s post-mortem this week. It was interesting to see how [...]

Pingback by Pathfinder Development » A Look Back At Past Posts, Tuesday, December 16, 2008 @ 12:26 pm
I’ve wrote something similiar,
http://github.com/tomaszmazur/trixy_scopes/tree

Comment by Tomasz Mazur, Monday, June 8, 2009 @ 11:11 am