Optimize path joins with os.sep.join(...).

[bdice]

Description

Extends on work started in #511. In a few performance-critical locations that use internal paths (not user-provided paths), we can sacrifice the safety benefits of os.path.join(*paths) in favor of the speedier os.sep.join(paths).

This yields about a 20-25% speedup when iterating over a project without accessing job state points.

We can only apply this optimization in places that deal with internal paths, like "project_workspace / job_id" but not like job.fn(some_user_path). User paths must be accessed through os.path.join, which includes the project workspace (which is configurable). Regardless, we are able to safely handle the most important cases (the job workspace and the job state point filename) that are called O(N) times.

Benchmarks

Below I have benchmarks of the following script, which just iterates over the project without accessing any job properties. There are 1000 jobs in the workspace.

import signac

project = signac.get_project()

def no_load_data():
    data = [job for job in project]

no_load_data()

Before

The call to os.path.join (displayed as posixpath.py:71(join)) takes a chunk of time, which is eliminated in the "After".

After

The bulk of the iteration is stat calls, which cannot be further optimized. 👍

Motivation and Context

Seeking some final obvious optimizations for synced collections (#484).

Types of Changes

• Documentation update
• Bug fix
• New feature
• Breaking change¹

¹The change breaks (or has the potential to break) existing functionality.

Checklist:

• I am familiar with the Contributing Guidelines.
• I agree with the terms of the Contributor Agreement.
• My name is on the list of contributors.
• My code follows the code style guideline of this project.
• The changes introduced by this pull request are covered by existing or newly introduced tests.

If necessary:

• I have updated the API documentation as part of the package doc-strings.
• I have created a separate pull request to update the framework documentation on signac-docs and linked it here.
• I have updated the changelog and added all related issue and pull request numbers for future reference (if applicable). See example below.

[codecov]

Codecov Report

Merging #515 (54a8b9a) into master (29c0749) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master     #515   +/-   ##
=======================================
  Coverage   78.05%   78.05%           
=======================================
  Files          63       63           
  Lines        6986     6986           
  Branches     1310     1310           
=======================================
  Hits         5453     5453           
  Misses       1228     1228           
  Partials      305      305           

Impacted Files	Coverage Δ
signac/contrib/job.py	89.96% <100.00%> (ø)
signac/contrib/project.py	85.08% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 29c0749...54a8b9a. Read the comment docs.

[kidrahahjo]

This is an interesting find, the changes looks good!