The New York Times articles server

The service scrapes all news headlines from nytimes.com and expose them using the GraphQL API.
The articles can be queried using following GraphQL schema:

type News {
  title: String,
  link: String,
}

type Query {
  news: [News!]!
}

Once service is started it will start scraping headlines and will redirect user to GraphQL Playground page.

Configuration

The service can customised by changing following settings

Setting	Description	Default value
SERVER_HTTP_PORT	Server port	8080
DB_NAME	Database name	wardrobe
DB_HOST	Database server host	localhost
DB_PORT	Database server port	5432
DB_USER	Database user	user
DB_PASSWORD	Database user password	1234
DB_WHETHER_CREATE_SCHEMA	Whether to create a database schema on the system run?	true
NY_TIMES_URL	The URL of the New York Times website	https://www.nytimes.com/
SCRAPE_REPEAT_INTERVAL	Scrape repeating interval	every 4 hours

How to build?

• Clone project
• Build the project
• Run tests

sbt compile
sbt test

Technology stack

• scala 2.13.6 as the main application programming language
• http4s typeful, functional, streaming HTTP for Scala
• sangria a GraphQL implementation for Scala
• scala-scraper a Scala library for scraping content from HTML pages
• quill compile-time language integrated queries for Scala
• cats to write more functional and less boilerplate code
• cats-effect The Haskell IO monad for Scala
• pureconfig for loading configuration files
• refined for type constraints avoiding unnecessary testing and boilerplate
• circe a JSON library for Scala
• scalatest and ScalaCheck for unit and property based testing
• testcontainers to run system dependant services for Integration Testing purposes