Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Convert image links to shortened URLs #546

Merged
merged 12 commits into from
Jul 23, 2018
1 change: 0 additions & 1 deletion app/controllers/users_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -131,5 +131,4 @@ def allow_iframe_request
# cannot-display-my-rails-4-app-in-iframe-even-if-x-frame-options-is-allowall
end


end
6 changes: 6 additions & 0 deletions app/models/company.rb
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,12 @@ def se_mailing_csv_str
AddressExporter.se_mailing_csv_str( main_address )
end

def get_or_create_short_h_brand_url(user)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint/UnusedMethodArgument: Unused method argument - user. If it's necessary, use _ or _user as an argument name to indicate that it won't be used. You can also write as get_or_create_short_h_brand_url(*) if you want the method to accept any arguments but don't care about them.

found = self.short_h_brand_url
return found unless found.nil?
ShortenUrl.short(company_id: self.id)
end


private

Expand Down
16 changes: 16 additions & 0 deletions app/services/shorten_url.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
class ShortenUrl

SHORTEN_URL_LOG = 'log/tinyurl.log'

def self.short(url)
response = HTTParty.get("http://tinyurl.com/api-create.php?url=#{url}")
raise HTTParty::Error.new response if response.match? 'ERROR'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm note sure we'll ever see a response that matches that string. You should check the response.code against "success" codes, and raise the exception if it is not.

For example, I forced a failure by sending a bad URI:

response = HTTParty.get("http://tinyurl.com/api-create.ph?url=''")
=> "<?xml version=\"1.0\" encoding=\"iso-8859-1\"?>\n" +
"<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\"\n" +
"         \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n" +
"<html xmlns=\"http://www.w3.org/1999/xhtml\" xml:lang=\"en\" lang=\"en\">\n" +
" <head>\n" +
"  <title>404 - Not Found</title>\n" +
" </head>\n" +
" <body>\n" +
"  <h1>404 - Not Found</h1>\n" +
" </body>\n" +
"</html>\n"
[4] pry(main)> response.inspect
=> "#<HTTParty::Response:0x7fd4ae91e850 parsed_response=\"<?xml version=\\\"1.0\\\" encoding=\\\"iso-8859-1\\\"?>\\n<!DOCTYPE html PUBLIC \\\"-//W3C//DTD XHTML 1.0 Transitional//EN\\\"\\n         \\\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\\\">\\n<html xmlns=\\\"http://www.w3.org/1999/xhtml\\\" xml:lang=\\\"en\\\" lang=\\\"en\\\">\\n <head>\\n  <title>404 - Not Found</title>\\n </head>\\n <body>\\n  <h1>404 - Not Found</h1>\\n </body>\\n</html>\\n\", @response=#<Net::HTTPNotFound 404 Not Found readbody=true>, @headers={\"date\"=>[\"Sat, 14 Jul 2018 18:03:21 GMT\"], \"content-type\"=>[\"text/html\"], \"transfer-encoding\"=>[\"chunked\"], \"connection\"=>[\"close\"], \"set-cookie\"=>[\"__cfduid=d071b4ebad4a29a20fd2ba0b0f2cd456a1531591401; expires=Sun, 14-Jul-19 18:03:21 GMT; path=/; domain=.tinyurl.com; HttpOnly\"], \"server\"=>[\"cloudflare\"], \"cf-ray\"=>[\"43a5e7d1967f2180-EWR\"]}>"
[5] pry(main)> 
[6] pry(main)> response.code
=> 404
[7] pry(main)> response.message
=> "Not Found"

In hips_service.rb, for example, we do this:

SUCCESS_CODES = [200, 201, 202].freeze
.
.
.
parsed_response = response.parsed_response

return parsed_response if SUCCESS_CODES.include?(response.code)
.
<handle HTTP error>

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, you don't need the .new - the exception will be created for you when you raise.

Copy link
Author

@tomurb tomurb Jul 14, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pry(main)> response = HTTParty.get("http://tinyurl.com/api-create.php?url=''")
=> "http://tinyurl.com/6mfcgml"

You lost 'p' in 'php' before '?'. The two only errors I know of look like this.

[10] pry(main)> response = HTTParty.get("http://tinyurl.com/api-create.php?url=/")
=> "ERROR"
[11] pry(main)> response.code
=> 200
[12] pry(main)> response = HTTParty.get("http://tinyurl.com/api-create.php?url=")
=> "Error"
[13] pry(main)> response.code
=> 400

Second one I found out about a moment ago. I will change match to /error/i I guess.

Copy link
Collaborator

@patmbolger patmbolger Jul 15, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, that will work. What I was trying to get at is the situation where the website is down and/or the request times out - in that case we won't get an "Error" from the site, but we should get a response.code other than 200.

So, maybe we should check for response matching /error/i or response.code != 200.

response
rescue HTTParty::Error => error
ActivityLogger.open(SHORTEN_URL_LOG, 'TINYURL_API', 'shortening url', false) do |log|
log.record('error', error.message)
end
nil
end
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're raising a Runtime error and then rescuing any exception.

  1. I'd rather raise an HTTParty::Error since that is what we have - an error code and/or body message has been returned that indicates an error.

  2. In general, one should not rescue very broad categories of exceptions. Otherwise, we could be handling an exception that has nothing to do with this method (happening down in the stack).

  3. I'd like to see more information logged - that's so important in helping us figure out problems in production.

I altered this code to:

def self.short(url)
    response = HTTParty.get("http://tinyurl.com/api-create.php?url=#{url}")
    raise HTTParty::Error if response.match?(/error/i) || !SUCCESS_CODES.include?(response.code)
    response
rescue HTTParty::Error => error
    ActivityLogger.open(SHORTEN_URL_LOG, 'TINYURL_API', 'shortening url', false) do |log|
      log.record('error', "Exception: #{error.message}")
      log.record('error', "Attempted URL: #{url}")
      log.record('error', "Response body: #{response.body}")
      log.record('error', "HTTP code: #{response.code}")
    end
    nil
end

I then called the method with: ShortenUrl.short('') and ShortenUrl.short('/') and this what I got in the log:

# Logfile created on 2018-07-20 07:42:14 -0400 by logger.rb/61378
[TINYURL_API] [shortening url] [info] Started at 2018-07-20 11:42:14 UTC
[TINYURL_API] [shortening url] [error] Exception: HTTParty::Error
[TINYURL_API] [shortening url] [error] Attempted URL: 
[TINYURL_API] [shortening url] [error] Response body: Error
[TINYURL_API] [shortening url] [error] HTTP code: 400
[TINYURL_API] [shortening url] [info] Finished at 2018-07-20 11:42:14 UTC.
[TINYURL_API] [shortening url] [info] Duration: 0.0 seconds.

[TINYURL_API] [shortening url] [info] Started at 2018-07-20 11:42:44 UTC
[TINYURL_API] [shortening url] [error] Exception: HTTParty::Error
[TINYURL_API] [shortening url] [error] Attempted URL: /
[TINYURL_API] [shortening url] [error] Response body: ERROR
[TINYURL_API] [shortening url] [error] HTTP code: 200
[TINYURL_API] [shortening url] [info] Finished at 2018-07-20 11:42:44 UTC.
[TINYURL_API] [shortening url] [info] Duration: 0.0 seconds.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, this approach allows us to capture more information for HTTParty-specific errors.

For other exceptions, we could follow this with a second rescue clause, picking up other exceptions such as the one you have now.

That way we rescue all exceptions, but we maximize the information we can get from HTTParty exceptions.

end

2 changes: 1 addition & 1 deletion app/views/users/_company_h_brand.html.haml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
%br
%br
= t('users.show.use_this_image_link_html')
= company_h_brand_url(user, company_id: company.id)
= short_company_h_brand_url(user, company.id)

:javascript
window.onload = function(){
Expand Down
1 change: 1 addition & 0 deletions config/initializers/shorten_url.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
TINYURL_LOG = 'log/shorten_url.log'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not used anymore(?)

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
class AddShortBrandHUrlAndShortProofOfMembershipUrlToCompanies < ActiveRecord::Migration[5.1]
def change
add_column :companies, :short_h_brand_url, :string
add_column :companies, :short_proof_of_membership_url, :string
end
end
8 changes: 4 additions & 4 deletions db/schema.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
#
# It's strongly recommended that you check this file into your version control system.

ActiveRecord::Schema.define(version: 20180428103625) do
ActiveRecord::Schema.define(version: 20180713091110) do

# These are extensions that must be enabled in order to support this database
enable_extension "plpgsql"
Expand Down Expand Up @@ -94,6 +94,9 @@
t.datetime "updated_at", null: false
t.text "description"
t.string "dinkurs_company_id"
t.boolean "show_dinkurs_events"
t.string "short_h_brand_url"
t.string "short_proof_of_membership_url"
t.index ["company_number"], name: "index_companies_on_company_number", unique: true
end

Expand Down Expand Up @@ -240,9 +243,6 @@
add_foreign_key "company_applications", "companies"
add_foreign_key "company_applications", "shf_applications"
add_foreign_key "events", "companies"
add_foreign_key "company_applications", "companies"
add_foreign_key "company_applications", "shf_applications"
add_foreign_key "events", "companies"
add_foreign_key "payments", "companies"
add_foreign_key "payments", "users"
add_foreign_key "shf_applications", "member_app_waiting_reasons", column: "member_app_waiting_reasons_id"
Expand Down
7 changes: 5 additions & 2 deletions db/structure.sql
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,9 @@ CREATE TABLE public.companies (
updated_at timestamp without time zone NOT NULL,
description text,
dinkurs_company_id character varying,
show_dinkurs_events boolean
show_dinkurs_events boolean,
short_h_brand_url character varying,
short_proof_of_membership_url character varying
);


Expand Down Expand Up @@ -1318,6 +1320,7 @@ INSERT INTO "schema_migrations" (version) VALUES
('20180326103433'),
('20180328105100'),
('20180428103625'),
('20180624155644');
('20180624155644'),
('20180713091110');


2 changes: 2 additions & 0 deletions spec/factories/companies.rb
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
phone_number '123123123'
email 'thiscompany@example.com'
website 'http://www.example.com'
short_h_brand_url nil
short_proof_of_membership_url nil

transient do
num_addresses 1
Expand Down
20 changes: 20 additions & 0 deletions spec/models/company_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@

RSpec.describe Company, type: :model do

let(:with_short_urls) do
create(:company, short_h_brand_url: 'http://www.tinyurl.com/hbrand', short_proof_of_membership_url: 'http://tinyurl.com/memproof')
end

let(:no_name) do
create(:company, name: '', company_number: '2120000142')
end
Expand Down Expand Up @@ -94,6 +98,8 @@
it { is_expected.to have_db_column :description }
it { is_expected.to have_db_column :dinkurs_company_id }
it { is_expected.to have_db_column :show_dinkurs_events }
it { is_expected.to have_db_column :short_h_brand_url }
it { is_expected.to have_db_column :short_proof_of_membership_url }
end

describe 'Validations' do
Expand Down Expand Up @@ -697,4 +703,18 @@
end
end

describe '#get_or_create_short_h_brand_url', focus: true do
context 'there is already a shortened url in the table' do
it 'returns shortened url' do
expect(with_short_urls.get_or_create_short_h_brand_url(user)).to eq('http://www.tinyurl.com/hbrand')
end
end
context 'there is no shortened url in the table and ShortenUrl.short is called' do
it 'saves the result if the result is not nil and returns shortened url' do
allow(ShortenUrl).to receive(:short).and_return('http://tinyurl.com/hbrand2')
expect(complete_co.get_or_create_short_h_brand_url(user)).to eq(ShortenUrl.short('a'))
end
it 'does not save anything if the result is nil and returns unshortened url'
end
end
end
19 changes: 19 additions & 0 deletions spec/services/shorten_url_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
require 'rails_helper'

describe ShortenUrl do
describe '.short' do
it 'creates shortened link' do
VCR.use_cassette('shorten_url/short') do
shortened_url = ShortenUrl.short 'http://sverigeshundforetagare.se/anvandare/2/proof_of_membership'
expect(shortened_url).to match 'tinyurl.com'
end
end
it 'if the service raises an error, returns nil and writes to the log' do
VCR.use_cassette('shorten_url/short') do
expect(ActivityLogger).to receive(:open)
shortened_url = ShortenUrl.short '/'
expect(shortened_url).to eq nil
end
end
end
end