Bump urlfrontier-API from 1.0 to 1.2

[dependabot]

Bumps urlfrontier-API from 1.0 to 1.2.

Release notes

Sourced from urlfrontier-API's releases.

What's new in URLFrontier 1.2

This is the 2nd step work towards URL Frontier 2, which is being funded through the NGI0 Discovery Fund.

This release fixes a bug introduced in version 1.1 and adds the following functionality.

The service implementation takes a parameter -s, the value of which is used as port number to expose metrics for Prometheus.

A dashboard for Grafana is provided.

The API and client code remains unchanged from the previous version. Only the service implementation is affected.

What's new in URLFrontier 1.1

This is the initial work towards URL Frontier 2, which is being funded through the NGI0 Discovery Fund.

Please note that the service implementation is now available from Maven, making it easier to write standalone service implementations to extend it.

Logging configuration

The logging is done with Logback. A default configuration is loaded and will dump logs on the console at INFO level and above but the configuration of the logging can be overridden by specifying a configuration file when launching a frontier service, e.g.

java -Dlogback.configurationFile=log-conf.xml ...

The API also has a new endpoint SetLogLevel, which allows changing the level of the logs generated by a running frontier service dynamically. The changes are not persisted between runs of the service.

This is typically done using the CLI

Usage: Client SetLogLevel [-l=STRING] -p=STRING
Change the log level of a package in the Frontier service
  -l, --level=STRING     Log level [TRACE, DEBUG, INFO, WARN, ERROR]
  -p, --package=STRING   package name

for instance

java -jar ~/urlfrontier-client-*.jar SetLogLevel -p crawlercommons.urlfrontier.service -l DEBUG

will ask the Frontier to generate logs at level DEBUG for any class within the crawlercommons.urlfrontier.service package.

Multi-tenancy with crawlIDs

A Frontier instance can now support multi-tenancy in URLFrontier by introducing a concept of crawlID, therefore handling logical crawls separately e.g. generic crawl vs specific ones. This affects pretty much every endpoint in the API as well as the service implementation.

Please note that these changes are not backward compatible and as a result, an existing frontier generated with a version < 1.1 can be loaded with URLFrontier 1.1 and above.

Two new endpoints have been added to the API in order to deal with crawls as a whole:

1. ListCrawls
2. DeleteCrawl

... (truncated)

Commits

• 1c39a3c [maven-release-plugin] prepare release urlfrontier-1.2
• f308bb2 version 1.2
• 0ec2095 bugfix check for crawlID only if it has been set
• e6aca69 Update README.md
• 58fd56a Merge pull request #49 from crawler-commons/prometheus
• 5f6fd3d Bugfix don't delete elements from the queues if they don't hvae the right cra...
• d410489 Added comment in the README about Prometheus
• 3f1b9c9 Merge changes from master
• ca12756 Update README.md
• 417340b [maven-release-plugin] prepare for next development iteration
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)