Skip to content

Commit

Permalink
Merge pull request #2 from jbartece/pncClient
Browse files Browse the repository at this point in the history
Introduce PNC 2.0 Rest client
  • Loading branch information
dwalluck committed Apr 22, 2020
2 parents a26f439 + d4d08b0 commit a321d4a
Show file tree
Hide file tree
Showing 47 changed files with 1,728 additions and 3,575 deletions.
29 changes: 9 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,24 +7,24 @@
| Name | Description | Badge |
| --------- | -------------------------- | -------------------------- |
| Maven | Latest Release | [![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.redhat.red.build/koji-build-finder/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.redhat.red.build/koji-build-finder)
| AppVeyor | Build Status (Windows) | [![Build Status (AppVeyor)](https://ci.appveyor.com/api/projects/status/775lq2o1chu7abn5?svg=true)](https://ci.appveyor.com/project/dwalluck/koji-build-finder-6o7ag) |
| Travis CI | Build Status (Linux/OS X) | [![Build Status (Travis CI)](https://api.travis-ci.com/release-engineering/koji-build-finder.svg)](https://travis-ci.com/release-engineering/koji-build-finder) |
| Codecov | Code Coverage | [![Code Coverage](https://codecov.io/gh/release-engineering/koji-build-finder/graph/badge.svg)](https://codecov.io/gh/release-engineering/koji-build-finder) |
| Snyk | Known Vulnerabilities | [![Known Vulnerabilities](https://snyk.io/test/github/release-engineering/koji-build-finder/badge.svg)](https://snyk.io/test/github/release-engineering/koji-build-finder) |
| AppVeyor | Build Status (Windows) | [![Build Status (AppVeyor)](https://ci.appveyor.com/api/projects/status/g1bjwaun03fao5qm?svg=true)](https://ci.appveyor.com/project/dwalluck/build-finder) |
| Travis CI | Build Status (Linux/OS X) | [![Build Status (Travis CI)](https://api.travis-ci.com/project-ncl/build-finder.svg)](https://travis-ci.com/project-ncl/build-finder) |
| Codecov | Code Coverage | [![Code Coverage](https://codecov.io/gh/project-ncl/build-finder/graph/badge.svg)](https://codecov.io/gh/project-ncl/build-finder) |
| Snyk | Known Vulnerabilities | [![Known Vulnerabilities](https://snyk.io/test/github/project-ncl/build-finder/badge.svg)](https://snyk.io/test/github/project-ncl/build-finder) |

## Development

An example `codestyle-eclipse.xml` code formatting style is supplied for [Eclipse](https://www.eclipse.org/ide/). An example `codestyle-intellij.xml` code formatting style is supplied for [IntelliJ IDEA](https://www.jetbrains.com/idea/).
To format the code, run `net.revelc.code.formatter:formatter-maven-plugin:format`.

[Apache Maven](https://maven.apache.org/) is used for the building. The command `mvn clean install` will compile the code and run all of the unit tests.

## Operation

The support for various compressed archive types relies on [Apache Commons VFS](https://commons.apache.org/proper/commons-vfs/) and the compressor and archive formats that Commons VFS can open *automatically*. If an exception occurs while trying to open a file, then the file is considered to be a normal file and recursive processing of the file is aborted.

The default supported Koji archive types are `jar`, `xml`, `pom`, `so`, `dll`, and `dylib`. Koji Build Finder uses [Koji Java Interface](https://github.com/release-engineering/kojiji) for Koji support and asks for all known extensions for the given Koji archive type name. Note that if you specify no Koji archive types, Koji Build Finder will ask the Koji server for all known Koji archive types. The default set of types is meant to give a reasonable default, particularly for Java-based distributions.
The default supported Koji archive types are `jar`, `xml`, `pom`, `so`, `dll`, and `dylib`. Build Finder uses [Koji Java Interface](https://github.com/release-engineering/kojiji) for Koji support and asks for all known extensions for the given Koji archive type name. Note that if you specify no Koji archive types, Build Finder will ask the Koji server for all known Koji archive types. The default set of types is meant to give a reasonable default, particularly for Java-based distributions.

Koji Build Finder operates in three stages:
Build Finder operates in three stages:

1. Checksums are calculated offline for all files in the distribution, including files inside archives. Checksum information is stored in JSON format.

Expand All @@ -46,7 +46,7 @@ To see the available options, execute the command `java -jar target/build-finder
Default: [pnc, koji]
-c, --config=FILE Specify configuration file to use.
Default: ${user.home}.
build-finder\config.json
build-finder/config.json
--cache-lifespan=LONG Specify cache lifespan.
Default: 3600000
--cache-max-idle=LONG Specify cache maximum idle time.
Expand Down Expand Up @@ -77,15 +77,9 @@ To see the available options, execute the command `java -jar target/build-finder
-o, --output-directory=FILE
Set output directory.
Default: .
--pnc-connection-timeout=LONG
Set Pnc connection timeout.
Default: -1
--pnc-partition-size=INT
Set Pnc partition size.
Default: 18
--pnc-read-timeout=LONG
Set Pnc read timeout.
Default: -1
--pnc-url=URL Set Pnc URL.
-q, --quiet Disable all logging.
-t, --checksum-type=CHECKSUM
Expand Down Expand Up @@ -119,6 +113,7 @@ where `<version>` should be replaced with the current version of the software.

On the first run, Build Finder will write a starter configuration file. You may optionally edit this file by hand, but you do not need to create it ahead of time as Build Finder will create a default configuration file if none exists.


### Configuration file format

The configuration file is in JSON format. The default configuration file, `config.json`, is as follows.
Expand All @@ -137,9 +132,7 @@ The configuration file is in JSON format. The default configuration file, `confi
"koji-multicall-size" : 8,
"koji-num-threads" : 12,
"output-directory" : ".",
"pnc-connection-timeout" : -1,
"pnc-partition-size" : 18,
"pnc-read-timeout" : -1,
"use-builds-file" : false,
"use-checksums-file" : false
}
Expand Down Expand Up @@ -170,12 +163,8 @@ The `koji-num-threads` option sets the number of Koji threads.

The `koji-hub-url` and `koji-web-url` options must be set to valid URLs for your particular network.

The `pnc-connection-timeout` option sets the Pnc connection timeout.

The `pnc-partition-size` option sets the Pnc partition size.

The `pnc-read-timeout` option sets the Pnc read timeout.

The `pnc-url` option must be set to a valid URL for your particular network if you want Pnc support.

The `output-directory` option specifies the directory to use for output.
Expand Down
16 changes: 16 additions & 0 deletions cli/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,10 @@
<groupId>org.infinispan</groupId>
<artifactId>infinispan-core</artifactId>
</dependency>
<dependency>
<groupId>org.jboss.logmanager</groupId>
<artifactId>jboss-logmanager</artifactId>
</dependency>
<dependency>
<groupId>com.github.stefanbirkner</groupId>
<artifactId>system-rules</artifactId>
Expand Down Expand Up @@ -146,6 +150,18 @@
<include>**</include>
</includes>
</filter>
<filter>
<artifact>org.jboss.logmanager:jboss-logmanager</artifact>
<includes>
<include>**</include>
</includes>
</filter>
<filter>
<artifact>org.jboss.resteasy:resteasy-jaxrs</artifact>
<includes>
<include>**</include>
</includes>
</filter>
<filter>
<artifact>*:*</artifact>
<excludes>
Expand Down
21 changes: 4 additions & 17 deletions cli/src/main/java/org/jboss/pnc/build/finder/cli/Main.java
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,8 @@
import org.jboss.pnc.build.finder.koji.KojiBuild;
import org.jboss.pnc.build.finder.koji.KojiClientSession;
import org.jboss.pnc.build.finder.koji.KojiJSONUtils;
import org.jboss.pnc.build.finder.pnc.client.PncClient14;
import org.jboss.pnc.build.finder.pnc.client.HashMapCachingPncClient;
import org.jboss.pnc.build.finder.pnc.client.PncClient;
import org.jboss.pnc.build.finder.report.BuildStatisticsReport;
import org.jboss.pnc.build.finder.report.GAVReport;
import org.jboss.pnc.build.finder.report.HTMLReport;
Expand Down Expand Up @@ -184,15 +185,9 @@ public final class Main implements Callable<Void> {
@Option(names = { "-o", "--output-directory" }, paramLabel = "FILE", description = "Set output directory.")
private File outputDirectory = new File(ConfigDefaults.OUTPUT_DIR);

@Option(names = { "--pnc-connection-timeout" }, paramLabel = "LONG", description = "Set Pnc connection timeout.")
private Long pncConnectionTimeout = ConfigDefaults.PNC_CONNECTION_TIMEOUT;

@Option(names = { "--pnc-partition-size" }, paramLabel = "INT", description = "Set Pnc partition size.")
private Integer pncPartitionSize = ConfigDefaults.PNC_PARTITION_SIZE;

@Option(names = { "--pnc-read-timeout" }, paramLabel = "LONG", description = "Set Pnc read timeout.")
private Long pncReadTimeout = ConfigDefaults.PNC_READ_TIMEOUT;

@Option(names = { "--pnc-url" }, paramLabel = "URL", description = "Set Pnc URL.")
private URL pncURL = ConfigDefaults.PNC_URL;

Expand Down Expand Up @@ -391,18 +386,10 @@ private BuildConfig setupBuildConfig() throws IOException {
LOGGER.debug("Read Kerberos password");
}

if (commandSpec.commandLine().getParseResult().hasMatchedOption("--pnc-connection-timeout")) {
config.setPncConnectionTimeout(pncConnectionTimeout);
}

if (commandSpec.commandLine().getParseResult().hasMatchedOption("--pnc-partition-size")) {
config.setPncPartitionSize(pncPartitionSize);
}

if (commandSpec.commandLine().getParseResult().hasMatchedOption("--pnc-read-timeout")) {
config.setPncReadTimeout(pncReadTimeout);
}

if (commandSpec.commandLine().getParseResult().hasMatchedOption("--pnc-url")) {
config.setPncURL(pncURL);
}
Expand Down Expand Up @@ -724,7 +711,7 @@ public Void call() throws Exception {
analyzer.setChecksums(checksums);

if (config.getPncURL() != null) {
PncClient14 pncclient = new PncClient14(config);
PncClient pncclient = new HashMapCachingPncClient(config);
finder = new BuildFinder(session, config, analyzer, cacheManager, pncclient);
} else {
finder = new BuildFinder(session, config, analyzer, cacheManager);
Expand Down Expand Up @@ -775,7 +762,7 @@ public Void call() throws Exception {
}

if (config.getPncURL() != null) {
PncClient14 pncclient = new PncClient14(config);
PncClient pncclient = new HashMapCachingPncClient(config);
finder = new BuildFinder(session, config, analyzer, cacheManager, pncclient);
} else {
finder = new BuildFinder(session, config, analyzer, cacheManager);
Expand Down
Loading

0 comments on commit a321d4a

Please sign in to comment.