Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Limit TXO set memory usage #228

Closed
doitian opened this issue Jan 26, 2019 · 0 comments
Closed

Limit TXO set memory usage #228

doitian opened this issue Jan 26, 2019 · 0 comments
Assignees
@yangby-cryptape

Comments

@doitian
Copy link
Member

doitian commented Jan 26, 2019

In #158, TXO stores all the history transaction, although each transaction only occupies about 1 byte, it will keep increasing.

At some time point, the general nodes will have no enough memory to store all the TXO spent flags. A simple optimization is pruning the fully spent transactions from the TXO set. It is OK to assume that the transaction using dead cells is orphan.

Even the number of transactions with unspent outputs is increasing. To make the node friendly to small memory devices, this should be hybrid storage, where the memory usage can be limited via config.

Originally posted by @doitian in #158
@doitian doitian added the s:triage Status: This issue is pending to decide the next step. label Jan 26, 2019
@doitian doitian added 500 and removed s:triage Status: This issue is pending to decide the next step. labels Apr 14, 2019
@doitian doitian removed the 500 label Apr 29, 2019
@doitian doitian modified the milestones: testnet, sprint/v0.12.0, backlog May 6, 2019
@doitian doitian modified the milestones: backlog, sprint/v0.12.0 May 9, 2019
@yangby-cryptape yangby-cryptape mentioned this issue May 19, 2019
Merged
This was referenced Jun 3, 2019
Closed
Merged
bors bot added a commit that referenced this issue Jul 24, 2019
@bors @dependabot-preview
Merge #1295
e6bdb38 
1295: chore(deps): bump backtrace from 0.3.20 to 0.3.33 r=doitian a=dependabot-preview[bot]

Bumps [backtrace](https://github.com/rust-lang/backtrace-rs) from 0.3.20 to 0.3.33.
<details>
<summary>Commits</summary>

- [`e2da24a`](rust-lang/backtrace-rs@e2da24a) Bump to 0.3.33
- [`eb7d74a`](rust-lang/backtrace-rs@eb7d74a) Bump backtrace-sys to 0.1.31
- [`7d0d81d`](rust-lang/backtrace-rs@7d0d81d) Bump version req of `cc` in `backtrace-sys`
- [`94ab4a7`](rust-lang/backtrace-rs@94ab4a7) Merge pull request [#228](https://github-redirect.dependabot.com/rust-lang/backtrace-rs/issues/228) from rust-lang/fix-android
- [`acf613e`](rust-lang/backtrace-rs@acf613e) android: Probe whether `dl_iterate_phdr` is defined
- [`7d93adb`](rust-lang/backtrace-rs@7d93adb) Merge pull request [#224](https://github-redirect.dependabot.com/rust-lang/backtrace-rs/issues/224) from rust-lang/run-all
- [`6482e2d`](rust-lang/backtrace-rs@6482e2d) gimli: Search object file symbol tables again
- [`3f854f7`](rust-lang/backtrace-rs@3f854f7) Bump backtrace-sys to 0.1.30
- [`19cfb49`](rust-lang/backtrace-rs@19cfb49) Merge pull request [#219](https://github-redirect.dependabot.com/rust-lang/backtrace-rs/issues/219) from mciantyre/master
- [`8e9980c`](rust-lang/backtrace-rs@8e9980c) Add prefix to one more macho_file_to_host symbol
- Additional commits viewable in [compare view](rust-lang/backtrace-rs@0.3.20...0.3.33)
</details>
<br />

[![Dependabot compatibility score](https://api.dependabot.com/badges/compatibility_score?dependency-name=backtrace&package-manager=cargo&previous-version=0.3.20&new-version=0.3.33)](https://dependabot.com/compatibility-score.html?dependency-name=backtrace&package-manager=cargo&previous-version=0.3.20&new-version=0.3.33)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot ignore this [patch|minor|major] version` will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it). To ignore the version in this PR you can just close it
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
- `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Automerge options (never/patch/minor, and dev/runtime dependencies)
- Pull request limits (per update run and/or open at any time)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)

Finally, you can contact us by mentioning @dependabot.

</details>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
bors bot added a commit that referenced this issue May 1, 2020
@bors @driftluo @yangby-cryptape
Merge #2050 #2053
f27ed20 
2050: chore: upgrade p2p/tokio r=TheWaWaR,quake a=driftluo

1. Upgrade p2p, use new `BlockingFlag` to optimize performance
2. Upgrade tokio, allow recursive call `block_in_place`

p2p release note:
```
Bug Fix

    Fix session proto open/close by user part(#220)

Features

    Replace unsplit with assignment(#225)
    Upgrade tokio util(#224)
    Avoids unnecessary session id checking(#223)
    Check all underscore(#228)
    Use flag to control how to use block_in_place(#226)
    More test(#227/#220)
```

tokio release note:
```
Fix
    rt: support block_in_place in more contexts
    stream: no panic in merge() and chain() when using size_hint()
    task: include visibility modifier when defining a task-local
    sync: broadcast closing the channel no longer requires capacity 

Added
    rt: runtime::Handle::block_on
```

2053: test: save integration test logs in a specified directory r=keroro520,doitian a=yangby-cryptape

Currently, it's too hard to distinguish between integration test logs and other temporary logs, especially when make integration test to be automated.

Co-authored-by: driftluo <[email protected]>
Co-authored-by: Boyu Yang <[email protected]>
bors bot added a commit that referenced this issue May 1, 2020
@bors @driftluo
Merge #2050
e492116 
2050: chore: upgrade p2p/tokio r=TheWaWaR,quake a=driftluo

1. Upgrade p2p, use new `BlockingFlag` to optimize performance
2. Upgrade tokio, allow recursive call `block_in_place`

p2p release note:
```
Bug Fix

    Fix session proto open/close by user part(#220)

Features

    Replace unsplit with assignment(#225)
    Upgrade tokio util(#224)
    Avoids unnecessary session id checking(#223)
    Check all underscore(#228)
    Use flag to control how to use block_in_place(#226)
    More test(#227/#220)
```

tokio release note:
```
Fix
    rt: support block_in_place in more contexts
    stream: no panic in merge() and chain() when using size_hint()
    task: include visibility modifier when defining a task-local
    sync: broadcast closing the channel no longer requires capacity 

Added
    rt: runtime::Handle::block_on
```

Co-authored-by: driftluo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yangby-cryptape yangby-cryptape

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@doitian @yangby-cryptape