Skip to content

mohandast52/medium-scrapper

Repository files navigation

Medium Blog Scrapper

Things you may want to cover:

  • Ruby version: 2.5.3

  • Rails version: 5.2.2

  • Database: sqlite3

  • Prerequisites: Headless chrome browser driver

Home

Picture of acutal home page

home_pic

Search

When users types input and searches. Also has Read More button which gets the next 10 blogs using AJAX request

search

Blog

When the blog is clicked, it opens new tab and displays it

blog

History

All the history is stored in database

history

History search

When the user clicks on History table, the search is triggered

history_search

Adding new entry (add onto history database)

Simulation of user typing in input feild and it is stored into History database, as soon as the search button is clicked

adding_new_entry

Related tags

Related tags and their searches triggered

related_tags

On wrong input

On wrong input, error is displayed

on_wrong_input

On error

on_error

Explanation:

In a nutshell, I have used an idea of headless chrome browser which is automated using nokogiri gem (used for web scraping). It extracts data from that headless chrome, parses it and responses with JSON, where angularJS is used to display it. I mainly used this approach because medium makes ajax request on a scroll, so I was not able to use the traditional idea of paginations.

About

just an assignment

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published