Skip to content

Commit

Permalink
Merge branch 'master' into patch-1
Browse files Browse the repository at this point in the history
  • Loading branch information
smagdali committed Jul 17, 2023
2 parents ef40506 + 936f07f commit 72aaab3
Show file tree
Hide file tree
Showing 55 changed files with 3,723 additions and 70 deletions.
2 changes: 1 addition & 1 deletion .fleek.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"build": {
"image": "hugomrdias/puppeteer",
"image": "node:20.2",
"command": "npm install && npm run build",
"publicDir": "public"
}
Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ yarn.lock
package-lock.json
node_modules
resources
static/_gen
data/toc.json
api/dist
api/worker
.hugo_build.lock
assets/jsconfig.json
1 change: 1 addition & 0 deletions .nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
v20.2.0
42 changes: 31 additions & 11 deletions api/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -31,21 +31,41 @@ async function handleRequest(event) {
async function cov(event) {
const url = new URL(event.request.url)
// https://github.com/filecoin-project/lotus
const repo = url.searchParams.get('repo').split('/').slice(3).join('/')
const data = await get(event, {
url: `https://codecov.io/api/gh/${repo}`,
const [owner, repo] = url.searchParams.get('repo').split('/').slice(3)
const headers = {
'User-Agent': 'ianconsolata',
Accept: 'application/json',
Authorization: `Bearer ${CODECOV_TOKEN}`,
}
const repo_resp = await get(event, {
url: `https://api.codecov.io/api/v2/github/${owner}/repos/${repo}/`,
transform: (data) => {
const out = {
cov: dlv(data, 'commit.totals.c', 0),
ci: dlv(data, 'commit.ci_passed', false),
repo: dlv(data, 'repo.name', 'N/A'),
org: dlv(data, 'owner.username', 'N/A'),
lang: dlv(data, 'repo.language', 'N/A'),
branch: dlv(data, 'branch', 'master'),
lang: dlv(data, 'language', 'N/A'),
}
return out
},
headers,
})
return data
const repo_data = await repo_resp.json()

const cov_data = await get(event, {
url: `https://api.codecov.io/api/v2/github/${owner}/repos/${repo}/branches/${repo_data.branch}/`,
transform: (data) => {
const out = {
cov: dlv(data, 'head_commit.totals.coverage', 0),
ci: dlv(data, 'head_commit.ci_passed', false),
repo: repo,
org: owner,
lang: repo_data.lang,
branch: repo_data.branch,
}
return out
},
headers,
})
return cov_data
}

async function github(event) {
Expand All @@ -56,7 +76,7 @@ async function github(event) {
const path = file.slice(7).join('/')
const ref = file[6]
const headers = {
'User-Agent': 'hugomrdias',
'User-Agent': 'ianconsolata',
Authorization: `token ${GITHUB_TOKEN}`,
}

Expand All @@ -82,7 +102,7 @@ async function github(event) {

async function releases(event) {
const headers = {
'User-Agent': 'hugomrdias',
'User-Agent': 'ianconsolata',
Authorization: `token ${GITHUB_TOKEN}`,
}
const rsp = await get(event, {
Expand Down
3 changes: 2 additions & 1 deletion api/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
"dlv": "^1.1.3",
"merge-options": "^3.0.3",
"nanoid": "^3.1.16",
"p-map": "^4.0.0"
"p-map": "^4.0.0",
"wrangler": "^2.17.0"
}
}
8 changes: 3 additions & 5 deletions api/wrangler.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
name = "specs-api"
type = "webpack"
account_id = "fffa4b4363a7e5250af8357087263b3a"
account_id = "e85b6fdda17f8c464c343dc84d1373e8"
workers_dev = true
route = ""
zone_id = ""
webpack_config = "webpack.config.js"
main = "index.js"
compatibility_date = "2023-04-27"
6 changes: 5 additions & 1 deletion config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ canonifyurls = false
theme = "book"
# relativeURLs = true

# allow building even if getting remote JSON fails
# needed because rust-fil-proofs has no coverage
ignoreErrors = ["error-remote-getjson"]

# Book configuration
disablePathToLower = true
enableGitInfo = true
Expand Down Expand Up @@ -41,7 +45,7 @@ enableGitInfo = true
# weight = 10

[params]
API = 'https://specs-api.protocol-labs.workers.dev'
API = 'https://specs-api.filecoin-foundation.workers.dev'
# (Optional, default true) Controls table of contents visibility on right side of pages.
# Start and end levels can be controlled with markup.tableOfContents setting.
# You can also specify this parameter per page in front matter.
Expand Down
10 changes: 9 additions & 1 deletion content/algorithms/crypto/poseidon.md
Original file line number Diff line number Diff line change
Expand Up @@ -356,7 +356,15 @@ while providing security against known attacks (statistical, interpolation, and

<br/>

{{<plain>}}$\constb R_F, R_P = \texttt{calc\_round\_numbers}(p, M, t, \alpha)${{</plain>}}\
{{<plain>}}$\constb R_F, R_P = \texttt{calc\_round\_numbers}(p, M, t, c_{\alpha}})${{</plain>}}\
where the S-box case, {{<plain>}}$c_{\alpha}${{</plain>}}, is given by
{{<plain>}}
$c_{\alpha} = \begin{cases}
0 & \if \alpha = 3 \cr
1 & \if \alpha = 5 \cr
2 & \if \alpha = \neg 1
\end{cases}
{{</plain>}}\
The number of full and partial rounds, both are positive integers $R_F, R_P \typecolon \mathbb{Z}_{>0}$ and $R_F$ is even.

$R_F$ and $R_P$ are calculated using either the Python script [`calc_round_numbers.py`](https://extgit.iaik.tugraz.at/krypto/hadeshash/-/blob/9d80ec0473ad7cde5a12f3aac46439ad0da68c0a/code/scripts/calc_round_numbers.py) or the [`neptune`](https://github.com/filecoin-project/neptune) Rust library, denoted {{<plain>}}$\texttt{calc\_round\_numbers}${{</plain>}}. Both methods calculate the round numbers via brute-force; by iterating over all reasonable values for $R_F$ and $R_P$ and choosing the pair that satisfies the security inequalities (provided below) while minimizing the number of S-boxes.
Expand Down
10 changes: 5 additions & 5 deletions content/algorithms/expected_consensus/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,9 +173,9 @@ WinCount guarantees that a lucky single block will earn the same reward as the r

**Alternative Options for the Distribution/Sortition**

Bernouli, Binomial and Poisson distributions have been considered for the _WinCount_ of a miner with power `p` out of a total network power of `N`. There are the following options:
Bernoulli, Binomial and Poisson distributions have been considered for the _WinCount_ of a miner with power `p` out of a total network power of `N`. There are the following options:

- Option 1: WinCount(p,N) ~ Bernouli(pE/N)
- Option 1: WinCount(p,N) ~ Bernoulli(pE/N)
- Option 2: WinCount(p,N) ~ Binomial(E, p/N)
- Option 3: WinCount(p,N) ~ Binomial(p, E/N)
- Option 4: WinCount(p,N) ~ Binomial(p/M, ME/N)
Expand All @@ -189,7 +189,7 @@ In Option 5 (the one used in Filecoin Leader Election), the ticket targets for e

**Rationale for the Poisson Sortition choice**

- Option 1 - Bernouli(pE/N): this option is easy to implement, but comes with a drawback: if the miner's power exceeds 1/E, the miner's WinCount is always 1, but never higher than 1.
- Option 1 - Bernoulli(pE/N): this option is easy to implement, but comes with a drawback: if the miner's power exceeds 1/E, the miner's WinCount is always 1, but never higher than 1.
- Option 2 - Binomial(E, p/N): the expectation of WinCount stays the same irrespectively of whether the miner splits their power into more than one Sybil nodes, but the variance increases if they choose to Sybil. Risk-seeking miners will prefer to Sybil, while risk-averse miners will prefer to pool, none of which is a behaviour the protocol should encourage. This option is not computationally-expensive as it would involve calculation of factorials and fixed-point multiplications (or small integer exponents) only.
- Option 3 - Binomial(p, E/N): this option is computationally inefficient. It involves very large integer exponents.
- Option 4 - Binomial(p/M, ME/N): the complexity of this option depends on the value of M. A small M results in high computational cost, similarly to Option 3. A large M, on the other hand, leads to a situation similar to that of Option 2, where a risk-seeking miner is incentivized to Sybil. Clearly none of these are desirable properties.
Expand Down Expand Up @@ -342,7 +342,7 @@ The exact value for these parameters remain to be determined, but for testing pu
- `log2b(X) = floor(log2(x)) = (binary length of X) - 1` and `log2b(0) = 0`. Note that that special case should never be used (given it would mean an empty power table).

{{< hint warning >}}
**Note that if your implementation does not allow for rounding to the fourth decimal**, miners should apply the [tie-breaker below](selecting-between-tipsets-with-equal-weight). Weight changes will be on the order of single digit numbers on expectation, so this should not have an outsized impact on chain consensus across implementations.
**Note that if your implementation does not allow for rounding to the fourth decimal**, miners should apply the [tie-breaker below](#section-algorithms.expected_consensus.selecting-between-tipsets-with-equal-weight). Weight changes will be on the order of single digit numbers on expectation, so this should not have an outsized impact on chain consensus across implementations.
{{< /hint >}}

`ParentWeight` is the aggregate chain weight of a given block's parent set. It is calculated as
Expand Down Expand Up @@ -397,7 +397,7 @@ This is detectable when a given miner submits two blocks that satisfy any of the

A single consensus fault results into:

- miner termination and removal of power from the power table,
- miner suspension
- loss of all pledge collateral (which includes the initial pledge and blocks rewards yet to be vested)

### Detection and Reporting
Expand Down
2 changes: 1 addition & 1 deletion content/algorithms/pos/post.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Before continuing into more details of the _WinnningPoSt_ and _WindowPoSt_ algor

## WinningPoSt

At the beginning of each epoch, a small number of storage miners (up to five, currently) are elected to mine new blocks, by Filecoin's [Expected Consensus](algorithms#expected_consensus) algorithm. Recall that the Filecoin blockchain operates on the basis of _tipsets_, which are groups of up to five blocks. This means that multiple blocks (up to five) can be mined at the same height.
At the beginning of each epoch, a small number of storage miners are elected to mine new blocks, by Filecoin's [Expected Consensus](algorithms#expected_consensus) algorithm. Recall that the Filecoin blockchain operates on the basis of _tipsets_, therefore multiple blocks can be mined at the same height.

Each of the miners that are elected to mine a block have to submit a proof that they keep a sealed copy of the data which they have included in their proposed block, before the end of the current epoch. Successful submission of this proof is the _WinningPoSt_, which in turn grants the miner the Filecoin Block Reward, as well as the opportunity to charge other nodes fees in order to include their messages in the block. If a miner misses the epoch-end deadline, then the miner misses the opportunity to mine a block and get a Block Reward. No penalty is incurred in this case.

Expand Down
4 changes: 2 additions & 2 deletions content/algorithms/sdr/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -1156,8 +1156,8 @@ $\line{11}{}{\return \Sha{254}(\preimage) \as \Fq}$

**Implementation:**

- [`storage_proofs::porep::stacked::circuit::proof::StackedCircuit::synthesize()`](https://github.com/filecoin-project/rust-fil-proofs/blob/master/storage-proofs/porep/src/stacked/circuit/proof.rs#L77)
- [`storage_proofs::porep::stacked::circuit::params::Proof::synthesize()`](https://github.com/filecoin-project/rust-fil-proofs/blob/master/storage-proofs/porep/src/stacked/circuit/params.rs#L77)
- [`storage_proofs::porep::stacked::circuit::proof::StackedCircuit::synthesize()`](https://github.com/filecoin-project/rust-fil-proofs/blob/108517271f8166cabc23e1966d8e8f397ec75ae0/storage-proofs-porep/src/stacked/circuit/proof.rs#L64)
- [`storage_proofs::porep::stacked::circuit::params::Proof::synthesize()`](https://github.com/filecoin-project/rust-fil-proofs/blob/108517271f8166cabc23e1966d8e8f397ec75ae0/storage-proofs-porep/src/stacked/circuit/proof.rs#L92)

**Additional Notation:**

Expand Down
22 changes: 22 additions & 0 deletions content/appendix/audit_reports.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,17 @@ dashboardAudit: n/a

Security is a critical component in ensuring Filecoin can fulfill its mission to be the storage network for humanity. In addition to robust secure development processes, trainings, theory audits, and investing in external security research, the Filecoin project has engaged reputable third party auditing specialists to ensure that the theory behind the protocol and its implementation delivers the intended value, enabling Filecoin to be a safe and secure network. This section covers a selection of audit reports that have been published on Filecoin's theory and implementation.

## Filecoin Virtual Machine

### `2023-03-09` Filecoin EVM (FEVM)

- Report: [Filecoin EVM Audit](<https://github.com/oak-security/audit-reports/blob/master/Filecoin%20Foundation/2023-03-09%20Audit%20Report%20-%20Filecoin%20EVM%20(FEVM)%20v1.1.pdf>)
- Audit conducted by **Oak Security**

The audit covers the implementation of:

- FEVM's [builtin actors](https://github.com/filecoin-project/builtin-actors/tree/1b11df4b399550753a4105f45f58bc07015af2a3/actors/evm) out of which only [actors/evm](https://github.com/filecoin-project/builtin-actors/tree/1b11df4b399550753a4105f45f58bc07015af2a3/actors/evm) and [actors/eam](https://github.com/filecoin-project/builtin-actors/tree/1b11df4b399550753a4105f45f58bc07015af2a3/actors/eam) were included in scope along with code base of [ref-fvm](https://github.com/filecoin-project/ref-fvm). The report included auditing EVM runtime action and implementation, correctness of EVM opcodes, including Ethereum Address Manager(EAM). The report also included issues and enhancements methods for gas model and F4 addresses. The audit team also reviewed the message execution flow and kernel setup, WASM integration and FVM logs. All the valid issues raised by the audit were resolved and acknowledged including a few informational issues. More details on these issues are available in the report.

## Lotus

### `2020-10-20` Lotus Mainnet Ready Security Audit
Expand All @@ -22,6 +33,17 @@ The scope of this audit covered:
- The Lotus Daemon: Core component responsible for handling the Blockchain node logic by handling peer- to-peer networking, chain syncing, block validation, data retrieval and transfer, etc.
- The Lotus Storage Miner: Mining component used to manage a single storage miner by contributing to the network through Sector commitments and Proofs-of-Spacetime data proving it is storing the sectors it has committed to. This component communicates with the Lotus daemon via JSON-RPC API calls.

## Venus

### `2021-06-29` Venus Security Audit

- Report: [Venus Security Assessment](https://leastauthority.com/static/publications/LeastAuthority_Filecoin_Foundation_Venus_Final_Audit_Report.pdf)
- Audit conducted by: **Least Authority**

The scope of this audit covered:

- The Venus Daemon: Core component responsible for handling the Filecoin node logic by handling peer-to-peer networking, chain syncing, block validation, etc.

## Actors

### `2020-10-19` Actors Mainnet Ready Security Audit
Expand Down
2 changes: 1 addition & 1 deletion content/implementations/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Filecoin is targeting [multiple implementations](https://filecoin.io/blog/announ

- [Lotus](https://lotu.sh): the Go-based implementation, supported by [Protocol Labs](https://protocol.ai),
- [Forest](https://github.com/ChainSafe/forest): the Rust-based implementation, supported by [ChainSafe](https://chainsafe.io),
- [Fuhon](https://github.com/filecoin-project/cpp-filecoin): the C++-based implementation, supported by [Soramitsu](https://soramitsu.co.jp),
- [Fuhon (deprecated)](https://github.com/filecoin-project/cpp-filecoin): the C++-based implementation, supported by [Soramitsu](https://soramitsu.co.jp),
- [Venus](https://github.com/filecoin-project/go-filecoin): a second Go-based implementation of Filecoin, previously called `go-filecoin`, which is maintained by the [IPFS-Force Community](https://github.com/ipfs-force-community).

{{<dashboard-impl>}}
4 changes: 3 additions & 1 deletion content/implementations/fuhon.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Fuhon
title: Fuhon (deprecated)
weight: 4
dashboardWeight: 1
dashboardState: reliable
Expand All @@ -15,3 +15,5 @@ Fuhon is the C++ implementation of Filecoin. The implementation uses Rust librar
You can find the Fuhon codebase [here](https://github.com/filecoin-project/cpp-filecoin).

The Fuhon implementation of Filecoin is supported by [Soramitsu](https://soramitsu.co.jp/).

Since May 2022, this implementation has been deprecated and out of support. The existing code repositories will remain public.
2 changes: 1 addition & 1 deletion content/intro/systems/impl_systems.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ For this purpose, we use the `FilecoinNode` data structure, which is passed into
Further, Systems MUST abide by the following limitations:

- **Random crashes.** A Filecoin Node may crash at any moment. Systems must be secure and consistent through crashes.
This is primarily achived by limiting the use of persistent state, persisting such state through Ipld data structures,
This is primarily achieved by limiting the use of persistent state, persisting such state through Ipld data structures,
and through the use of initialization routines that check state, and perhaps correct errors.
- **Isolation.** Systems must communicate over well-defined, isolated interfaces. They must not build their critical
functionality over a shared memory space. (Note: for performance, shared memory abstractions can be used to power
Expand Down
6 changes: 3 additions & 3 deletions content/intro/systems/why_systems.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,12 @@ This decoupling is useful for:
- **Security Isolation:** some systems require higher operational security than others. System decoupling allows
implementations to meet their security and functionality needs. A good example of this is separating Blockchain
processing from Data Transfer.
- **Scalability:** systems, and various use cases, may drive different performance requirements for different opertators.
- **Scalability:** systems, and various use cases, may drive different performance requirements for different operators.
System decoupling makes it easier for operators to scale their deployments along system boundaries.

## Filecoin Nodes don't need all the systems

Filecoin Nodes vary significantly, and do not need all the systems.
Filecoin Nodes vary significantly and do not need all the systems.
Most systems are only needed for a subset of use cases.

For example, the Blockchain System is required for synchronizing the chain, participating in secure consensus,
Expand Down Expand Up @@ -68,7 +68,7 @@ Sometimes this is harder, and it requires detangling, adding, or removing abstra
example, the `StoragePowerActor` and the `StorageMarketActor` were a single `Actor` previously. This caused
a large coupling of functionality across `StorageDeal` making, the `StorageMarket`, markets in general, with
Storage Mining, Sector Sealing, PoSt Generation, and more. Detangling these two sets of related functionality
requried breaking apart the one actor into two.
required breaking apart the one actor into two.

## Decomposing within a System

Expand Down
Loading

0 comments on commit 72aaab3

Please sign in to comment.