Skip to content

Commit

Permalink
Replaced crack parser with the json gem for improved reliability and …
Browse files Browse the repository at this point in the history 
…performance
  • Loading branch information
@sferik
sferik committed Mar 24, 2010
1 parent f62a150 commit 7f2272c
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 4 deletions.
1 change: 1 addition & 0 deletions Rakefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Jeweler::Tasks.new do |gem|
gem.add_dependency("oauth", "~> 0.3.6")
gem.add_dependency("hashie", "~> 0.2.0")
gem.add_dependency("httparty", "~> 0.5.2")
gem.add_dependency("json", "~> 1.2.3")

gem.add_development_dependency("shoulda", "~> 2.10.1")
gem.add_development_dependency("jnunemaker-matchy", "~> 0.4.0")
Expand Down
6 changes: 4 additions & 2 deletions lib/twitter.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
require "oauth"
require "hashie"
require "httparty"
require "json"

module Twitter
include HTTParty
Expand Down Expand Up @@ -94,12 +95,13 @@ def self.raise_errors(response)
end

def self.parse(response)
Crack::JSON.parse(response.body)
return '' if response.body == ''
JSON.parse(response.body)
end

def self.mash(obj)
if obj.is_a?(Array)
obj.map { |item| make_mash_with_consistent_hash(item) }
obj.map{|item| make_mash_with_consistent_hash(item)}
elsif obj.is_a?(Hash)
make_mash_with_consistent_hash(obj)
else
Expand Down
7 changes: 5 additions & 2 deletions twitter.gemspec
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Gem::Specification.new do |s|

s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
s.authors = ["John Nunemaker", "Wynn Netherland"]
s.date = %q{2010-03-23}
s.date = %q{2010-03-24}
s.email = %q{nunemaker@gmail.com}
s.extra_rdoc_files = [
"README.rdoc"
Expand Down Expand Up @@ -99,7 +99,7 @@ Gem::Specification.new do |s|
s.homepage = %q{http://github.com/jnunemaker/twitter}
s.rdoc_options = ["--charset=UTF-8"]
s.require_paths = ["lib"]
s.rubygems_version = %q{1.3.5}
s.rubygems_version = %q{1.3.6}
s.summary = %q{wrapper for the twitter api}
s.test_files = [
"test/test_helper.rb",
Expand Down Expand Up @@ -133,6 +133,7 @@ Gem::Specification.new do |s|
s.add_runtime_dependency(%q<oauth>, ["~> 0.3.6"])
s.add_runtime_dependency(%q<hashie>, ["~> 0.2.0"])
s.add_runtime_dependency(%q<httparty>, ["~> 0.5.2"])
s.add_runtime_dependency(%q<json>, ["~> 1.2.3"])
s.add_development_dependency(%q<shoulda>, ["~> 2.10.1"])
s.add_development_dependency(%q<jnunemaker-matchy>, ["~> 0.4.0"])
s.add_development_dependency(%q<mocha>, ["~> 0.9.4"])
Expand All @@ -141,6 +142,7 @@ Gem::Specification.new do |s|
s.add_dependency(%q<oauth>, ["~> 0.3.6"])
s.add_dependency(%q<hashie>, ["~> 0.2.0"])
s.add_dependency(%q<httparty>, ["~> 0.5.2"])
s.add_dependency(%q<json>, ["~> 1.2.3"])
s.add_dependency(%q<shoulda>, ["~> 2.10.1"])
s.add_dependency(%q<jnunemaker-matchy>, ["~> 0.4.0"])
s.add_dependency(%q<mocha>, ["~> 0.9.4"])
Expand All @@ -150,6 +152,7 @@ Gem::Specification.new do |s|
s.add_dependency(%q<oauth>, ["~> 0.3.6"])
s.add_dependency(%q<hashie>, ["~> 0.2.0"])
s.add_dependency(%q<httparty>, ["~> 0.5.2"])
s.add_dependency(%q<json>, ["~> 1.2.3"])
s.add_dependency(%q<shoulda>, ["~> 2.10.1"])
s.add_dependency(%q<jnunemaker-matchy>, ["~> 0.4.0"])
s.add_dependency(%q<mocha>, ["~> 0.9.4"])
Expand Down

4 comments on commit 7f2272c

@timhaines
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like this. Did you consider Yajl too? Reputedly it's faster..

@sferik
Copy link
Owner Author

@sferik sferik commented on 7f2272c Apr 16, 2010

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made this change to resolve issue #11, not with performance in mind. If you submit a patch swapping in Yajl (including performance benchmarks showing that it's actually faster) I'm sure it would be merged upstream.

@sferik
Copy link
Owner Author

@sferik sferik commented on 7f2272c Apr 21, 2010

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind. I got to it first: http://github.com/sferik/twitter/commit/1f480a85925025aec1ac5c91cfb45b4e74e4c9c3

@brianmario
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, it's nearly 2x faster than Marshal in some cases: http://gist.github.com/296828

It also has much nicer unicode support (thanks to Yajl itself), and even sets the encoding on all strings returned to the caller to UTF-8 (as valid JSON specifies). It's able to parse stand-alone JSON tokens such as true, false, null, numbers, strings, etc without the need to be wrapped in {} or []. Multiple JSON tokens can exist in the stream/string being parsed, and will be yielded to an optional block passed to the parse method.

Meaning if you were to hand this string (including newlines) over to yajl-ruby:

{"test": "testing"}
[1, 2, 3, 4]
true

It would yield 3 objects, the first a ruby hash, the 2nd a ruby array, the 3rd a TrueClass.

This is the basis of many Twitter streaming API gems, as well as libraries using CouchDB's _changes API both which stream multiple JSON tokens over time to the consumer. The encoder can work similarly, the readme should have enough info to get started.

Also, yajl-ruby is now the default/prefered JSON backend in Rails 2.3 and 3.0

btw - a looooong time ago I forked this gem and ported it over to yajl-ruby, sent a pull request to no avail ;)

Please sign in to comment.