yuna88/paginator

By yuna88

•Updated about 4 years ago

Service to parse HTML pages including javascript

Image

469

Overview Tags

yuna88/paginator repository overview

⁠Paginator

⁠Service to parse HTML pages including javascript and caching

⁠Configurations

Please request any issues and wishes to GitHub https://github.com/YunaBraska/paginator⁠

ENV VARIABLE	DEFAULT	DESCRIPTION
SERVER_PORT	8089	Server port
N/A	10000	HTML pages cache limit
N/A	10800000ms	HTML pages cache life time

⁠Endpoints

METHOD	URL	REQUEST BODY	RETURN BODY	Description
GET/PUT	/pages	String url		Get html page from url
GET/PUT	/pages/elements	url, Map<queryId, cssQuery>	Map<queryId, List<Elements>>	Get specific html elements
GET/PUT	/pages/statistics		size, maxLifeTime, sizeLimit	Get cache statistics
GET/PUT	/pages	url, content		Manual add html page to cache

⁠Examples

⁠Get elements from HTML page

Request: GET http://localhost:8089/pages/elements
Body:

{
  "url": "parse.example.com",
  "css_queries": {
    "form_text": "form p"
  }
}

Response

{
  "form_text": [
    {
      "tag": "P",
      "text": "Some example text here.",
      "selector": "html > body > div > form > p:nth-child(1)",
      "attributes": {
      },
      "children": [
      ]
    }
  ]
}

⁠Cache custom html pages

Request: POST http://localhost:8089/pages
Body:

{
  "url": "parse.example.com",
  "content": "<!doctype html><html><head><title>Example Domain</title></head><body><div><h1>Example page</h1></div></body></html>"
}

Request: POST http://localhost:8089/pages
Body:

{
  "url": "parse.example.com",
  "content": "<!doctype html><html><head><title>Example Domain</title></head><body><div><h1>Example page</h1></div></body></html>"
}

⁠Docker build image example

Create jar file: mvn clean -Dmaven.test.skip=true package
Build local image docker build -t paginator .
Docker image tag latest for repo: docker tag "$(whoami)/paginator" SOME_REPO_PATH/paginator:latest;
Docker image push to repo: docker push SOME_REPO_PATH/paginator:latest

Tag summary

Recent tags

Content type

Image

Digest

Size

643.8 MB

Last updated

about 4 years ago

docker pull yuna88/paginator:feat-0-multi_docker_image